2024-09-12T02:58:36.672 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T02:58:36.697 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:58:36.786 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901665' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+C0LSmtRkU1jiku0oD3WJiTKs/lYcYrhxlduYpG0qNwUDUdJGFpzaDLrmBH4mesHirTkpmeMP/wCnqJUqYVw0= smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVnf4qkPB13GcYX04TyVja37EJfIVTeqfIHirtxJY+cuS+kTdMDAkfmcrAXb46ifLlDih+xD9w5nw3g61dNWT0= smithi135.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwQ/lqAbAs+Z/x+6RU6n1hbwD7QU0NtQvYfpvKMqVh4mrO1Y3vcbrHPpt8rdLu2nDBTQX9MfPuwa5SnylCvV3Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T02:58:36.786 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T02:58:36.787 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T02:58:36.787 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T02:58:36.788 INFO:teuthology.task.internal:Checking packages... 2024-09-12T02:58:36.809 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T02:58:36.809 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T02:58:36.809 INFO:teuthology.packaging:ref: None 2024-09-12T02:58:36.809 INFO:teuthology.packaging:tag: None 2024-09-12T02:58:36.809 INFO:teuthology.packaging:branch: squid 2024-09-12T02:58:36.809 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:58:36.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T02:58:37.063 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T02:58:37.064 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T02:58:37.071 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T02:58:37.071 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T02:58:37.145 INFO:teuthology.task.internal:Saving configuration 2024-09-12T02:58:37.164 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T02:58:37.203 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T02:58:37.227 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:53:51.861850', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB+C0LSmtRkU1jiku0oD3WJiTKs/lYcYrhxlduYpG0qNwUDUdJGFpzaDLrmBH4mesHirTkpmeMP/wCnqJUqYVw0='} 2024-09-12T02:58:37.247 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:53:51.863215', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVnf4qkPB13GcYX04TyVja37EJfIVTeqfIHirtxJY+cuS+kTdMDAkfmcrAXb46ifLlDih+xD9w5nw3g61dNWT0='} 2024-09-12T02:58:37.273 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi135.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:53:51.860042', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwQ/lqAbAs+Z/x+6RU6n1hbwD7QU0NtQvYfpvKMqVh4mrO1Y3vcbrHPpt8rdLu2nDBTQX9MfPuwa5SnylCvV3Y='} 2024-09-12T02:58:37.274 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T02:58:37.280 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T02:58:37.280 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T02:58:37.280 INFO:teuthology.task.internal:roles: ubuntu@smithi135.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T02:58:37.280 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T02:58:37.383 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f0696df3be0>, signals=[15]) 2024-09-12T02:58:37.383 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T02:58:37.434 INFO:teuthology.task.internal:Opening connections... 2024-09-12T02:58:37.434 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T02:58:37.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:58:37.516 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-09-12T02:58:37.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:58:37.594 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi135.front.sepia.ceph.com 2024-09-12T02:58:37.595 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:58:37.671 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T02:58:37.677 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-09-12T02:58:37.695 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-09-12T02:58:37.696 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-09-12T02:58:37.753 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:58:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:58:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:58:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:58:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:58:37.754 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:58:37.755 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-09-12T02:58:37.772 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-09-12T02:58:37.791 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-09-12T02:58:37.791 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-09-12T02:58:37.849 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-09-12T02:58:37.849 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-09-12T02:58:37.849 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:58:37.850 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:58:37.851 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-09-12T02:58:37.873 DEBUG:teuthology.orchestra.run.smithi135:> uname -m 2024-09-12T02:58:37.891 INFO:teuthology.orchestra.run.smithi135.stdout:x86_64 2024-09-12T02:58:37.892 DEBUG:teuthology.orchestra.run.smithi135:> cat /etc/os-release 2024-09-12T02:58:37.950 INFO:teuthology.orchestra.run.smithi135.stdout:NAME="CentOS Stream" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION="9" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:ID="centos" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:VERSION_ID="9" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:ANSI_COLOR="0;31" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:58:37.951 INFO:teuthology.orchestra.run.smithi135.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:58:37.952 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:58:37.952 INFO:teuthology.orchestra.run.smithi135.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:58:37.952 INFO:teuthology.lock.ops:Updating smithi135.front.sepia.ceph.com on lock server 2024-09-12T02:58:37.972 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T02:58:37.984 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T02:58:38.017 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T02:58:38.017 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:58:38.020 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:58:38.022 DEBUG:teuthology.orchestra.run.smithi135:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:58:38.039 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T02:58:38.050 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T02:58:38.050 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:58:38.077 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:58:38.079 DEBUG:teuthology.orchestra.run.smithi135:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:58:38.176 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T02:58:38.257 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T02:58:38.272 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T02:58:38.272 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T02:58:38.272 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T02:58:38.273 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:58:38.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:58:38.291 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-12T02:58:38.347 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:58:38.348 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-503.el9.x86_64 2024-09-12T02:58:38.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-09-12T02:58:40.682 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-12T02:58:47.358 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 3.7 MB/s | 20 MB 00:05 2024-09-12T02:58:53.742 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.5 MB 00:02 2024-09-12T02:58:55.476 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-12T02:58:56.338 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-12T02:59:02.144 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-12T02:59:04.242 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:59:04.243 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:59:04.345 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:59:04.362 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 79 M 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 126 M 2024-09-12T02:59:04.363 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-12T02:59:04.754 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 198 kB/s | 23 kB 00:00 2024-09-12T02:59:11.145 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.4 MB/s | 16 MB 00:06 2024-09-12T02:59:11.629 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.9 MB/s | 35 MB 00:06 2024-09-12T02:59:18.162 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.1 MB/s | 29 MB 00:13 2024-09-12T02:59:18.162 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:59:18.162 INFO:teuthology.orchestra.run.smithi019.stdout:Total 5.7 MB/s | 79 MB 00:13 2024-09-12T02:59:18.874 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-12T02:59:19.255 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-12T02:59:19.255 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-12T02:59:20.252 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-12T02:59:20.252 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-12T02:59:22.854 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-12T02:59:23.548 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:59:23.661 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:59:26.449 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:59:26.574 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:59:31.703 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:59:31.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:59:36.880 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:00:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:00:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:00:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:00:16.337 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T03:00:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T03:00:16.338 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:00:17.173 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-12T03:00:17.499 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:00:17.634 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.634 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:00:17.634 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T03:00:17.634 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T03:00:17.634 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T03:00:17.634 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T03:00:17.634 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T03:00:17.635 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T03:00:17.693 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T03:00:17.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T03:00:17.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-09-12T03:00:17.737 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 82.el9 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8274670 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T03:00:17.738 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:00:17.739 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T03:00:17.740 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T03:00:17.740 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-505.el9.x86_64 2024-09-12T03:00:17.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T03:00:18.450 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-09-12T03:00:19.790 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T03:00:19.815 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-09-12T03:00:19.817 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-09-12T03:00:19.832 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.XPSR5SGg1x 2024-09-12T03:00:19.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XPSR5SGg1x 2024-09-12T03:00:19.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.XPSR5SGg1x 2024-09-12T03:00:20.034 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.XPSR5SGg1x is 6KB 2024-09-12T03:00:20.083 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.XPSR5SGg1x 2024-09-12T03:00:20.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T03:00:20.159 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T03:00:20.160 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T03:00:20.160 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T03:00:20.160 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T03:00:20.161 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T03:00:20.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-09-12T03:00:50.322 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T03:00:50.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:00:50.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:01:08.797 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-09-12T03:01:17.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:01:17.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:02:17.802 DEBUG:teuthology.orchestra.remote:timed out 2024-09-12T03:02:29.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:02:29.805 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:02:30.091 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-12T03:02:30.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:02:30.446 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T03:02:30.446 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-12T03:02:30.463 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-505.el9.x86_64 2024-09-12T03:02:30.464 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T03:02:30.464 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T03:02:30.464 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T03:02:31.465 DEBUG:teuthology.parallel:result is None 2024-09-12T03:02:31.465 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T03:02:31.473 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T03:02:31.473 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:02:31.476 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:02:31.478 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T03:02:31.497 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T03:02:31.503 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T03:02:31.544 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T03:02:31.545 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:02:31.547 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:02:31.550 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T03:02:31.583 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T03:02:31.590 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T03:02:31.590 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:02:31.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:31.633 DEBUG:teuthology.orchestra.run.smithi135:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T03:02:31.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:31.651 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:02:31.653 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:02:31.676 DEBUG:teuthology.orchestra.run.smithi135:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T03:02:31.735 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.739 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.749 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.751 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.758 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.773 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T03:02:31.776 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T03:02:31.782 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T03:02:31.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:02:31.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:02:31.788 DEBUG:teuthology.orchestra.run.smithi135:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T03:02:31.815 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T03:02:31.824 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T03:02:31.825 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:02:31.857 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:02:31.859 DEBUG:teuthology.orchestra.run.smithi135:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T03:02:31.876 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:31.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:32.008 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.136 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:02:32.136 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:02:32.204 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:32.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:32.309 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.394 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:02:32.395 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:02:32.461 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:32.507 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T03:02:32.566 DEBUG:teuthology.orchestra.run.smithi135:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.649 DEBUG:teuthology.orchestra.run.smithi135:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T03:02:32.692 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:02:32.692 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T03:02:32.760 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-09-12T03:02:32.763 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-09-12T03:02:32.765 DEBUG:teuthology.orchestra.run.smithi135:> sudo service rsyslog restart 2024-09-12T03:02:32.804 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:02:32.808 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:02:32.838 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:02:33.338 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T03:02:33.346 INFO:teuthology.task.internal:Starting timer... 2024-09-12T03:02:33.346 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T03:02:33.376 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T03:02:33.410 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T03:02:33.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-09-12T03:02:33.476 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-09-12T03:02:33.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-09-12T03:02:33.543 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-09-12T03:02:33.545 DEBUG:teuthology.orchestra.run.smithi135:> sudo service auditd rotate 2024-09-12T03:02:33.585 INFO:teuthology.orchestra.run.smithi135.stdout:Rotating logs: 2024-09-12T03:02:33.587 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T03:02:33.587 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-09-12T03:02:33.623 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-09-12T03:02:33.623 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-09-12T03:02:33.645 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-09-12T03:02:33.645 DEBUG:teuthology.orchestra.run.smithi135:> /usr/sbin/getenforce 2024-09-12T03:02:33.681 INFO:teuthology.orchestra.run.smithi135.stdout:Permissive 2024-09-12T03:02:33.681 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi121.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi135.front.sepia.ceph.com': 'permissive'} 2024-09-12T03:02:33.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:02:33.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:33.713 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:02:33.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:33.742 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:02:33.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:02:33.772 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T03:02:33.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:02:33.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:02:33.921 DEBUG:teuthology.orchestra.run.smithi135:> sudo /usr/sbin/setenforce permissive 2024-09-12T03:02:33.958 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T03:02:33.967 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T03:02:33.982 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T03:02:33.983 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi121.front.sepia.ceph.com,smithi135.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T03:06:39.371 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi121.front.sepia.ceph.com'), Remote(name='ubuntu@smithi135.front.sepia.ceph.com')] 2024-09-12T03:06:39.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:06:39.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:06:39.455 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-12T03:06:39.535 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-12T03:06:39.535 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-12T03:06:39.536 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:06:39.632 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-09-12T03:06:39.709 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-09-12T03:06:39.710 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-12T03:06:39.710 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi135.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T03:06:39.788 DEBUG:teuthology.orchestra.run.smithi135:> true 2024-09-12T03:06:39.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi135.front.sepia.ceph.com' 2024-09-12T03:06:39.873 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T03:06:39.883 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T03:06:39.883 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:06:39.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:06:39.886 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:06:39.886 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:06:39.889 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T03:06:39.889 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:06:39.917 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:06:39.920 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:06:39.932 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:06:39.936 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:06:39.941 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T03:06:39.957 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-09-12T03:06:39.957 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T03:06:39.964 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-09-12T03:06:39.968 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-09-12T03:06:39.976 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-09-12T03:06:39.982 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:06:39.983 INFO:teuthology.orchestra.run.smithi135.stderr:sudo: ntpd: command not found 2024-09-12T03:06:39.992 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:06:39.995 INFO:teuthology.orchestra.run.smithi135.stdout:506 Cannot talk to daemon 2024-09-12T03:06:39.996 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:06:40.007 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:06:40.010 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T03:06:40.026 INFO:teuthology.orchestra.run.smithi135.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T03:06:40.050 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.054 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.073 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.076 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.086 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.089 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:06:40.090 INFO:teuthology.run_tasks:Running task install... 2024-09-12T03:06:40.099 DEBUG:teuthology.task.install:project ceph 2024-09-12T03:06:40.099 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T03:06:40.099 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T03:06:40.100 INFO:teuthology.task.install:Using flavor: default 2024-09-12T03:06:40.106 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T03:06:40.106 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T03:06:40.107 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:06:40.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:06:40.110 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:06:40.110 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:06:40.111 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T03:06:40.111 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:06:40.253 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:06:40.254 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:06:40.263 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:06:40.263 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:06:40.268 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T03:06:40.269 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T03:06:40.362 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:06:40.363 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:06:40.363 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:06:40.373 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:06:40.373 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:06:40.373 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:06:40.378 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T03:06:40.378 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:06:40.378 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T03:06:40.390 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:06:40.391 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:06:40.399 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:06:40.399 DEBUG:teuthology.orchestra.run.smithi135:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:06:40.404 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T03:06:40.404 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T03:06:40.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:06:40.465 DEBUG:teuthology.orchestra.run.smithi135:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:06:40.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T03:06:40.537 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:06:40.540 DEBUG:teuthology.orchestra.run.smithi135:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:06:40.549 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T03:06:40.602 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-09-12T03:06:40.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-09-12T03:06:40.605 INFO:teuthology.orchestra.run.smithi135.stdout:check_obsoletes = 1 2024-09-12T03:06:40.606 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum clean all 2024-09-12T03:06:40.614 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-09-12T03:06:40.615 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-09-12T03:06:40.830 INFO:teuthology.orchestra.run.smithi135.stdout:45 files removed 2024-09-12T03:06:40.840 INFO:teuthology.orchestra.run.smithi121.stdout:45 files removed 2024-09-12T03:06:40.842 INFO:teuthology.orchestra.run.smithi019.stdout:45 files removed 2024-09-12T03:06:40.857 DEBUG:teuthology.orchestra.run.smithi135:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:06:40.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:06:40.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T03:06:41.402 INFO:teuthology.orchestra.run.smithi135.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-12T03:06:41.426 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-12T03:06:41.445 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T03:06:41.686 INFO:teuthology.orchestra.run.smithi135.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T03:06:41.710 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T03:06:41.729 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-12T03:06:41.945 INFO:teuthology.orchestra.run.smithi135.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-12T03:06:41.993 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-12T03:06:42.004 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T03:06:42.795 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-12T03:06:43.377 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.2 MB 00:01 2024-09-12T03:06:43.463 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.2 MB 00:01 2024-09-12T03:06:44.579 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-12T03:06:47.728 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 6.3 MB/s | 20 MB 00:03 2024-09-12T03:06:49.847 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 3.9 MB/s | 20 MB 00:05 2024-09-12T03:06:50.072 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.5 MB 00:01 2024-09-12T03:06:52.237 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-12T03:06:52.381 INFO:teuthology.orchestra.run.smithi135.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-12T03:06:53.452 INFO:teuthology.orchestra.run.smithi135.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-12T03:06:53.704 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 68 kB/s | 19 kB 00:00 2024-09-12T03:06:54.575 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T03:06:54.640 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.5 MB 00:00 2024-09-12T03:06:57.157 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-12T03:06:58.269 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-12T03:06:58.991 INFO:teuthology.orchestra.run.smithi135.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T03:07:00.064 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T03:07:01.285 INFO:teuthology.orchestra.run.smithi135.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:01.285 INFO:teuthology.orchestra.run.smithi135.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:01.296 INFO:teuthology.orchestra.run.smithi135.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:07:01.297 INFO:teuthology.orchestra.run.smithi135.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:07:01.298 INFO:teuthology.orchestra.run.smithi135.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:07:01.299 INFO:teuthology.orchestra.run.smithi135.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:07:01.300 INFO:teuthology.orchestra.run.smithi135.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:07:01.303 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:07:01.303 INFO:teuthology.orchestra.run.smithi135.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:07:01.365 INFO:teuthology.orchestra.run.smithi135.stdout:Dependencies resolved. 2024-09-12T03:07:01.385 INFO:teuthology.orchestra.run.smithi135.stdout:====================================================================================== 2024-09-12T03:07:01.385 INFO:teuthology.orchestra.run.smithi135.stdout: Package Arch Version Repository Size 2024-09-12T03:07:01.385 INFO:teuthology.orchestra.run.smithi135.stdout:====================================================================================== 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout:Installing: 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:07:01.386 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:07:01.387 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:07:01.388 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:07:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:07:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:07:01.389 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrading: 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout:Installing dependencies: 2024-09-12T03:07:01.390 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:07:01.391 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:07:01.392 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:07:01.393 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:07:01.394 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:07:01.395 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:07:01.396 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:07:01.397 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:07:01.398 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:07:01.399 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout:Installing weak dependencies: 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction Summary 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout:====================================================================================== 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout:Install 148 Packages 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout:Upgrade 8 Packages 2024-09-12T03:07:01.400 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:01.401 INFO:teuthology.orchestra.run.smithi135.stdout:Total download size: 226 M 2024-09-12T03:07:01.401 INFO:teuthology.orchestra.run.smithi135.stdout:Downloading Packages: 2024-09-12T03:07:02.042 INFO:teuthology.orchestra.run.smithi135.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-09-12T03:07:02.366 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:02.366 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:02.378 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:07:02.379 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:07:02.380 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:07:02.381 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:07:02.382 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:07:02.385 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:07:02.386 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:07:02.448 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-09-12T03:07:02.462 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-12T03:07:02.462 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:07:02.463 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:07:02.464 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:07:02.465 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:07:02.466 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:07:02.467 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T03:07:02.468 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:07:02.469 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:07:02.470 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:07:02.471 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:07:02.472 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:07:02.473 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:07:02.474 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:07:02.475 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================== 2024-09-12T03:07:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:Install 148 Packages 2024-09-12T03:07:02.477 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 8 Packages 2024-09-12T03:07:02.477 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:02.477 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 226 M 2024-09-12T03:07:02.477 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-09-12T03:07:02.501 INFO:teuthology.orchestra.run.smithi135.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.9 MB/s | 897 kB 00:00 2024-09-12T03:07:02.576 INFO:teuthology.orchestra.run.smithi135.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.9 MB/s | 145 kB 00:00 2024-09-12T03:07:02.985 INFO:teuthology.orchestra.run.smithi135.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-09-12T03:07:03.107 INFO:teuthology.orchestra.run.smithi121.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2024-09-12T03:07:03.243 INFO:teuthology.orchestra.run.smithi135.stdout:(5/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 5.4 MB 00:01 2024-09-12T03:07:03.285 INFO:teuthology.orchestra.run.smithi135.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-12T03:07:03.507 INFO:teuthology.orchestra.run.smithi121.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.2 MB/s | 897 kB 00:00 2024-09-12T03:07:03.860 INFO:teuthology.orchestra.run.smithi135.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 7.6 MB/s | 4.7 MB 00:00 2024-09-12T03:07:03.926 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T03:07:04.141 INFO:teuthology.orchestra.run.smithi121.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.2 MB/s | 5.4 MB 00:01 2024-09-12T03:07:04.166 INFO:teuthology.orchestra.run.smithi121.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 220 kB/s | 145 kB 00:00 2024-09-12T03:07:04.333 INFO:teuthology.orchestra.run.smithi121.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 2.4 MB 00:00 2024-09-12T03:07:04.583 INFO:teuthology.orchestra.run.smithi121.stdout:(6/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 21 MB 00:01 2024-09-12T03:07:04.677 INFO:teuthology.orchestra.run.smithi135.stdout:(8/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9. 13 MB/s | 11 MB 00:00 2024-09-12T03:07:04.717 INFO:teuthology.orchestra.run.smithi121.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 12 MB/s | 4.7 MB 00:00 2024-09-12T03:07:04.752 INFO:teuthology.orchestra.run.smithi135.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 336 kB/s | 25 kB 00:00 2024-09-12T03:07:05.192 INFO:teuthology.orchestra.run.smithi121.stdout:(8/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 1.5 MB/s | 1.5 MB 00:01 2024-09-12T03:07:05.284 INFO:teuthology.orchestra.run.smithi121.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 275 kB/s | 25 kB 00:00 2024-09-12T03:07:05.411 INFO:teuthology.orchestra.run.smithi135.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 5.9 MB/s | 21 MB 00:03 2024-09-12T03:07:05.470 INFO:teuthology.orchestra.run.smithi135.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T03:07:05.518 INFO:teuthology.orchestra.run.smithi121.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 13 MB/s | 11 MB 00:00 2024-09-12T03:07:05.545 INFO:teuthology.orchestra.run.smithi135.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 362 kB/s | 27 kB 00:00 2024-09-12T03:07:05.601 INFO:teuthology.orchestra.run.smithi121.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 2.7 MB/s | 228 kB 00:00 2024-09-12T03:07:05.620 INFO:teuthology.orchestra.run.smithi135.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 423 kB/s | 32 kB 00:00 2024-09-12T03:07:05.677 INFO:teuthology.orchestra.run.smithi121.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 361 kB/s | 27 kB 00:00 2024-09-12T03:07:05.854 INFO:teuthology.orchestra.run.smithi135.stdout:(14/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 6.6 MB/s | 17 MB 00:02 2024-09-12T03:07:05.860 INFO:teuthology.orchestra.run.smithi121.stdout:(13/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 13 MB/s | 17 MB 00:01 2024-09-12T03:07:05.877 INFO:teuthology.orchestra.run.smithi121.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 158 kB/s | 32 kB 00:00 2024-09-12T03:07:05.887 INFO:teuthology.orchestra.run.smithi135.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 2.7 MB/s | 752 kB 00:00 2024-09-12T03:07:05.929 INFO:teuthology.orchestra.run.smithi135.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T03:07:05.952 INFO:teuthology.orchestra.run.smithi121.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.0 MB/s | 752 kB 00:00 2024-09-12T03:07:05.963 INFO:teuthology.orchestra.run.smithi135.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T03:07:05.978 INFO:teuthology.orchestra.run.smithi121.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.6 MB/s | 163 kB 00:00 2024-09-12T03:07:06.028 INFO:teuthology.orchestra.run.smithi121.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T03:07:06.055 INFO:teuthology.orchestra.run.smithi135.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 4.0 MB/s | 506 kB 00:00 2024-09-12T03:07:06.061 INFO:teuthology.orchestra.run.smithi121.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.9 MB/s | 506 kB 00:00 2024-09-12T03:07:06.137 INFO:teuthology.orchestra.run.smithi121.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 602 kB/s | 45 kB 00:00 2024-09-12T03:07:06.138 INFO:teuthology.orchestra.run.smithi135.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 542 kB/s | 45 kB 00:00 2024-09-12T03:07:06.222 INFO:teuthology.orchestra.run.smithi135.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.6 MB/s | 136 kB 00:00 2024-09-12T03:07:06.229 INFO:teuthology.orchestra.run.smithi121.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.4 MB/s | 136 kB 00:00 2024-09-12T03:07:06.292 INFO:teuthology.orchestra.run.smithi135.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.4 MB/s | 162 kB 00:00 2024-09-12T03:07:06.312 INFO:teuthology.orchestra.run.smithi121.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 162 kB 00:00 2024-09-12T03:07:06.364 INFO:teuthology.orchestra.run.smithi135.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 4.2 MB/s | 321 kB 00:00 2024-09-12T03:07:06.381 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:06.382 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T03:07:06.396 INFO:teuthology.orchestra.run.smithi019.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T03:07:06.396 INFO:teuthology.orchestra.run.smithi121.stdout:(22/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-12T03:07:06.397 INFO:teuthology.orchestra.run.smithi019.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T03:07:06.398 INFO:teuthology.orchestra.run.smithi019.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T03:07:06.399 INFO:teuthology.orchestra.run.smithi019.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T03:07:06.401 INFO:teuthology.orchestra.run.smithi019.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T03:07:06.404 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T03:07:06.405 INFO:teuthology.orchestra.run.smithi019.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T03:07:06.421 INFO:teuthology.orchestra.run.smithi121.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 2.9 MB/s | 321 kB 00:00 2024-09-12T03:07:06.423 INFO:teuthology.orchestra.run.smithi135.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.1 MB/s | 301 kB 00:00 2024-09-12T03:07:06.455 INFO:teuthology.orchestra.run.smithi121.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:07:06.472 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-12T03:07:06.483 INFO:teuthology.orchestra.run.smithi121.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.6 MB/s | 100 kB 00:00 2024-09-12T03:07:06.490 INFO:teuthology.orchestra.run.smithi135.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 100 kB 00:00 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T03:07:06.493 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T03:07:06.494 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T03:07:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T03:07:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T03:07:06.496 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T03:07:06.497 INFO:teuthology.orchestra.run.smithi019.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T03:07:06.498 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-12T03:07:06.499 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T03:07:06.500 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T03:07:06.501 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T03:07:06.502 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T03:07:06.503 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T03:07:06.504 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T03:07:06.505 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T03:07:06.506 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T03:07:06.507 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T03:07:06.508 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:====================================================================================== 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:Install 148 Packages 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 8 Packages 2024-09-12T03:07:06.509 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:06.510 INFO:teuthology.orchestra.run.smithi121.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.6 MB/s | 85 kB 00:00 2024-09-12T03:07:06.510 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 226 M 2024-09-12T03:07:06.510 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-12T03:07:06.565 INFO:teuthology.orchestra.run.smithi135.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 8.9 MB/s | 5.3 MB 00:00 2024-09-12T03:07:06.575 INFO:teuthology.orchestra.run.smithi121.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T03:07:06.591 INFO:teuthology.orchestra.run.smithi135.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 851 kB/s | 85 kB 00:00 2024-09-12T03:07:06.649 INFO:teuthology.orchestra.run.smithi135.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T03:07:06.667 INFO:teuthology.orchestra.run.smithi121.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-12T03:07:06.684 INFO:teuthology.orchestra.run.smithi121.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 293 kB/s | 32 kB 00:00 2024-09-12T03:07:06.691 INFO:teuthology.orchestra.run.smithi135.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 765 kB/s | 32 kB 00:00 2024-09-12T03:07:06.735 INFO:teuthology.orchestra.run.smithi121.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.1 MB/s | 146 kB 00:00 2024-09-12T03:07:06.767 INFO:teuthology.orchestra.run.smithi135.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T03:07:06.866 INFO:teuthology.orchestra.run.smithi135.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 10 MB/s | 3.1 MB 00:00 2024-09-12T03:07:06.935 INFO:teuthology.orchestra.run.smithi121.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 14 MB/s | 3.6 MB 00:00 2024-09-12T03:07:07.027 INFO:teuthology.orchestra.run.smithi121.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.7 MB/s | 249 kB 00:00 2024-09-12T03:07:07.112 INFO:teuthology.orchestra.run.smithi121.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 591 kB/s | 49 kB 00:00 2024-09-12T03:07:07.177 INFO:teuthology.orchestra.run.smithi121.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 247 kB/s | 16 kB 00:00 2024-09-12T03:07:07.269 INFO:teuthology.orchestra.run.smithi121.stdout:(35/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-12T03:07:07.295 INFO:teuthology.orchestra.run.smithi121.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.4 MB/s | 285 kB 00:00 2024-09-12T03:07:07.327 INFO:teuthology.orchestra.run.smithi019.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-12T03:07:07.353 INFO:teuthology.orchestra.run.smithi121.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-12T03:07:07.358 INFO:teuthology.orchestra.run.smithi135.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 6.0 MB/s | 3.6 MB 00:00 2024-09-12T03:07:07.370 INFO:teuthology.orchestra.run.smithi121.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 242 kB/s | 18 kB 00:00 2024-09-12T03:07:07.521 INFO:teuthology.orchestra.run.smithi121.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 478 kB/s | 80 kB 00:00 2024-09-12T03:07:07.561 INFO:teuthology.orchestra.run.smithi019.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-12T03:07:07.604 INFO:teuthology.orchestra.run.smithi121.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-12T03:07:07.638 INFO:teuthology.orchestra.run.smithi121.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.9 MB/s | 801 kB 00:00 2024-09-12T03:07:07.663 INFO:teuthology.orchestra.run.smithi121.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-12T03:07:07.703 INFO:teuthology.orchestra.run.smithi019.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-09-12T03:07:07.728 INFO:teuthology.orchestra.run.smithi019.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 870 kB/s | 145 kB 00:00 2024-09-12T03:07:07.738 INFO:teuthology.orchestra.run.smithi121.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-12T03:07:07.772 INFO:teuthology.orchestra.run.smithi121.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-12T03:07:07.797 INFO:teuthology.orchestra.run.smithi121.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T03:07:07.803 INFO:teuthology.orchestra.run.smithi019.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 24 MB/s | 2.4 MB 00:00 2024-09-12T03:07:07.831 INFO:teuthology.orchestra.run.smithi121.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-12T03:07:07.850 INFO:teuthology.orchestra.run.smithi135.stdout:(32/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 16 MB/s | 49 MB 00:03 2024-09-12T03:07:07.856 INFO:teuthology.orchestra.run.smithi121.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-12T03:07:07.884 INFO:teuthology.orchestra.run.smithi135.stdout:(33/156): ceph-mgr-modules-core-19.1.1-298.gb53 475 kB/s | 249 kB 00:00 2024-09-12T03:07:07.898 INFO:teuthology.orchestra.run.smithi121.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-09-12T03:07:07.957 INFO:teuthology.orchestra.run.smithi121.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 9.2 MB/s | 943 kB 00:00 2024-09-12T03:07:07.982 INFO:teuthology.orchestra.run.smithi121.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-09-12T03:07:07.993 INFO:teuthology.orchestra.run.smithi135.stdout:(34/156): ceph-mgr-diskprediction-local-19.1.1- 6.6 MB/s | 7.4 MB 00:01 2024-09-12T03:07:08.007 INFO:teuthology.orchestra.run.smithi121.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-12T03:07:08.018 INFO:teuthology.orchestra.run.smithi135.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 123 kB/s | 16 kB 00:00 2024-09-12T03:07:08.020 INFO:teuthology.orchestra.run.smithi019.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 5.2 MB/s | 1.5 MB 00:00 2024-09-12T03:07:08.041 INFO:teuthology.orchestra.run.smithi121.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-12T03:07:08.043 INFO:teuthology.orchestra.run.smithi135.stdout:(36/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 257 kB/s | 49 kB 00:00 2024-09-12T03:07:08.067 INFO:teuthology.orchestra.run.smithi121.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-12T03:07:08.093 INFO:teuthology.orchestra.run.smithi135.stdout:(37/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.8 MB/s | 285 kB 00:00 2024-09-12T03:07:08.104 INFO:teuthology.orchestra.run.smithi019.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 16 MB/s | 4.7 MB 00:00 2024-09-12T03:07:08.117 INFO:teuthology.orchestra.run.smithi121.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-12T03:07:08.118 INFO:teuthology.orchestra.run.smithi135.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 242 kB/s | 18 kB 00:00 2024-09-12T03:07:08.200 INFO:teuthology.orchestra.run.smithi121.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-09-12T03:07:08.210 INFO:teuthology.orchestra.run.smithi135.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-12T03:07:08.218 INFO:teuthology.orchestra.run.smithi121.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-12T03:07:08.244 INFO:teuthology.orchestra.run.smithi135.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-12T03:07:08.259 INFO:teuthology.orchestra.run.smithi121.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T03:07:08.277 INFO:teuthology.orchestra.run.smithi121.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-09-12T03:07:08.327 INFO:teuthology.orchestra.run.smithi121.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-12T03:07:08.328 INFO:teuthology.orchestra.run.smithi135.stdout:(41/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-12T03:07:08.353 INFO:teuthology.orchestra.run.smithi135.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-12T03:07:08.360 INFO:teuthology.orchestra.run.smithi121.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T03:07:08.371 INFO:teuthology.orchestra.run.smithi019.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 21 MB 00:01 2024-09-12T03:07:08.377 INFO:teuthology.orchestra.run.smithi121.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 486 kB/s | 24 kB 00:00 2024-09-12T03:07:08.387 INFO:teuthology.orchestra.run.smithi135.stdout:(43/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-12T03:07:08.402 INFO:teuthology.orchestra.run.smithi121.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-12T03:07:08.412 INFO:teuthology.orchestra.run.smithi019.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 608 kB/s | 25 kB 00:00 2024-09-12T03:07:08.421 INFO:teuthology.orchestra.run.smithi135.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T03:07:08.436 INFO:teuthology.orchestra.run.smithi121.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-12T03:07:08.446 INFO:teuthology.orchestra.run.smithi135.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-12T03:07:08.461 INFO:teuthology.orchestra.run.smithi121.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-12T03:07:08.471 INFO:teuthology.orchestra.run.smithi135.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:07:08.478 INFO:teuthology.orchestra.run.smithi121.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 920 kB/s | 38 kB 00:00 2024-09-12T03:07:08.505 INFO:teuthology.orchestra.run.smithi135.stdout:(47/156): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.2 MB 00:00 2024-09-12T03:07:08.530 INFO:teuthology.orchestra.run.smithi135.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T03:07:08.679 INFO:teuthology.orchestra.run.smithi121.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-09-12T03:07:08.687 INFO:teuthology.orchestra.run.smithi019.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 18 MB/s | 11 MB 00:00 2024-09-12T03:07:08.696 INFO:teuthology.orchestra.run.smithi135.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 642 kB/s | 107 kB 00:00 2024-09-12T03:07:08.722 INFO:teuthology.orchestra.run.smithi135.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2024-09-12T03:07:08.737 INFO:teuthology.orchestra.run.smithi121.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-12T03:07:08.738 INFO:teuthology.orchestra.run.smithi019.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 4.5 MB/s | 228 kB 00:00 2024-09-12T03:07:08.747 INFO:teuthology.orchestra.run.smithi135.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-12T03:07:08.779 INFO:teuthology.orchestra.run.smithi121.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:07:08.789 INFO:teuthology.orchestra.run.smithi135.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-12T03:07:08.813 INFO:teuthology.orchestra.run.smithi019.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 361 kB/s | 27 kB 00:00 2024-09-12T03:07:08.814 INFO:teuthology.orchestra.run.smithi135.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-12T03:07:08.821 INFO:teuthology.orchestra.run.smithi121.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-12T03:07:08.839 INFO:teuthology.orchestra.run.smithi135.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-12T03:07:08.847 INFO:teuthology.orchestra.run.smithi121.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:07:08.865 INFO:teuthology.orchestra.run.smithi135.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-12T03:07:08.880 INFO:teuthology.orchestra.run.smithi019.stdout:(13/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 476 kB/s | 32 kB 00:00 2024-09-12T03:07:08.890 INFO:teuthology.orchestra.run.smithi135.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-12T03:07:08.915 INFO:teuthology.orchestra.run.smithi135.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T03:07:08.922 INFO:teuthology.orchestra.run.smithi121.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T03:07:08.950 INFO:teuthology.orchestra.run.smithi135.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-09-12T03:07:08.964 INFO:teuthology.orchestra.run.smithi019.stdout:(14/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 8.8 MB/s | 752 kB 00:00 2024-09-12T03:07:08.981 INFO:teuthology.orchestra.run.smithi121.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T03:07:09.008 INFO:teuthology.orchestra.run.smithi135.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-09-12T03:07:09.033 INFO:teuthology.orchestra.run.smithi135.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-12T03:07:09.039 INFO:teuthology.orchestra.run.smithi019.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T03:07:09.051 INFO:teuthology.orchestra.run.smithi135.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 585 kB/s | 24 kB 00:00 2024-09-12T03:07:09.068 INFO:teuthology.orchestra.run.smithi135.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-12T03:07:09.101 INFO:teuthology.orchestra.run.smithi135.stdout:(63/156): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-12T03:07:09.114 INFO:teuthology.orchestra.run.smithi121.stdout:(73/156): postgresql-server-13.16-1.el9.x86_64. 20 MB/s | 5.8 MB 00:00 2024-09-12T03:07:09.114 INFO:teuthology.orchestra.run.smithi019.stdout:(16/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.6 MB/s | 127 kB 00:00 2024-09-12T03:07:09.126 INFO:teuthology.orchestra.run.smithi135.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-12T03:07:09.152 INFO:teuthology.orchestra.run.smithi135.stdout:(65/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-12T03:07:09.177 INFO:teuthology.orchestra.run.smithi135.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 512 kB/s | 38 kB 00:00 2024-09-12T03:07:09.181 INFO:teuthology.orchestra.run.smithi121.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 919 kB/s | 61 kB 00:00 2024-09-12T03:07:09.206 INFO:teuthology.orchestra.run.smithi019.stdout:(17/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T03:07:09.261 INFO:teuthology.orchestra.run.smithi135.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-09-12T03:07:09.265 INFO:teuthology.orchestra.run.smithi121.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-12T03:07:09.302 INFO:teuthology.orchestra.run.smithi135.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-12T03:07:09.336 INFO:teuthology.orchestra.run.smithi135.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-12T03:07:09.348 INFO:teuthology.orchestra.run.smithi121.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-12T03:07:09.374 INFO:teuthology.orchestra.run.smithi121.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-12T03:07:09.395 INFO:teuthology.orchestra.run.smithi135.stdout:(70/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-12T03:07:09.399 INFO:teuthology.orchestra.run.smithi121.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-12T03:07:09.425 INFO:teuthology.orchestra.run.smithi121.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-12T03:07:09.436 INFO:teuthology.orchestra.run.smithi135.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-12T03:07:09.458 INFO:teuthology.orchestra.run.smithi121.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-12T03:07:09.470 INFO:teuthology.orchestra.run.smithi135.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T03:07:09.484 INFO:teuthology.orchestra.run.smithi121.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-12T03:07:09.495 INFO:teuthology.orchestra.run.smithi135.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T03:07:09.537 INFO:teuthology.orchestra.run.smithi135.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T03:07:09.542 INFO:teuthology.orchestra.run.smithi121.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T03:07:09.601 INFO:teuthology.orchestra.run.smithi121.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T03:07:09.613 INFO:teuthology.orchestra.run.smithi135.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-09-12T03:07:09.623 INFO:teuthology.orchestra.run.smithi019.stdout:(18/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 5.3 MB 00:00 2024-09-12T03:07:09.638 INFO:teuthology.orchestra.run.smithi135.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-12T03:07:09.663 INFO:teuthology.orchestra.run.smithi135.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-12T03:07:09.668 INFO:teuthology.orchestra.run.smithi121.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-09-12T03:07:09.689 INFO:teuthology.orchestra.run.smithi135.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-12T03:07:09.693 INFO:teuthology.orchestra.run.smithi121.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 839 kB/s | 77 kB 00:00 2024-09-12T03:07:09.719 INFO:teuthology.orchestra.run.smithi121.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-09-12T03:07:09.760 INFO:teuthology.orchestra.run.smithi121.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-09-12T03:07:09.780 INFO:teuthology.orchestra.run.smithi135.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-12T03:07:09.785 INFO:teuthology.orchestra.run.smithi121.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-12T03:07:09.798 INFO:teuthology.orchestra.run.smithi019.stdout:(19/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 9.5 MB/s | 17 MB 00:01 2024-09-12T03:07:09.814 INFO:teuthology.orchestra.run.smithi135.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-12T03:07:09.819 INFO:teuthology.orchestra.run.smithi121.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-09-12T03:07:09.824 INFO:teuthology.orchestra.run.smithi019.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 225 kB/s | 45 kB 00:00 2024-09-12T03:07:09.839 INFO:teuthology.orchestra.run.smithi135.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-09-12T03:07:09.844 INFO:teuthology.orchestra.run.smithi121.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 868 kB/s | 51 kB 00:00 2024-09-12T03:07:09.849 INFO:teuthology.orchestra.run.smithi019.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.6 MB/s | 136 kB 00:00 2024-09-12T03:07:09.870 INFO:teuthology.orchestra.run.smithi121.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:07:09.874 INFO:teuthology.orchestra.run.smithi019.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T03:07:09.899 INFO:teuthology.orchestra.run.smithi019.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 6.3 MB/s | 321 kB 00:00 2024-09-12T03:07:09.920 INFO:teuthology.orchestra.run.smithi121.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-09-12T03:07:09.923 INFO:teuthology.orchestra.run.smithi135.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-12T03:07:09.933 INFO:teuthology.orchestra.run.smithi019.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 5.0 MB/s | 301 kB 00:00 2024-09-12T03:07:09.948 INFO:teuthology.orchestra.run.smithi135.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-12T03:07:09.950 INFO:teuthology.orchestra.run.smithi019.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T03:07:09.973 INFO:teuthology.orchestra.run.smithi135.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-12T03:07:09.976 INFO:teuthology.orchestra.run.smithi019.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.0 MB/s | 85 kB 00:00 2024-09-12T03:07:09.979 INFO:teuthology.orchestra.run.smithi121.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-09-12T03:07:09.999 INFO:teuthology.orchestra.run.smithi135.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-12T03:07:10.024 INFO:teuthology.orchestra.run.smithi135.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-12T03:07:10.034 INFO:teuthology.orchestra.run.smithi019.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.9 MB/s | 171 kB 00:00 2024-09-12T03:07:10.037 INFO:teuthology.orchestra.run.smithi121.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T03:07:10.050 INFO:teuthology.orchestra.run.smithi135.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-12T03:07:10.075 INFO:teuthology.orchestra.run.smithi135.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-12T03:07:10.087 INFO:teuthology.orchestra.run.smithi121.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 721 kB/s | 36 kB 00:00 2024-09-12T03:07:10.100 INFO:teuthology.orchestra.run.smithi135.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-12T03:07:10.101 INFO:teuthology.orchestra.run.smithi019.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 479 kB/s | 32 kB 00:00 2024-09-12T03:07:10.125 INFO:teuthology.orchestra.run.smithi135.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-12T03:07:10.138 INFO:teuthology.orchestra.run.smithi121.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-09-12T03:07:10.150 INFO:teuthology.orchestra.run.smithi135.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T03:07:10.176 INFO:teuthology.orchestra.run.smithi019.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.9 MB/s | 146 kB 00:00 2024-09-12T03:07:10.205 INFO:teuthology.orchestra.run.smithi121.stdout:(97/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-12T03:07:10.234 INFO:teuthology.orchestra.run.smithi135.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-12T03:07:10.243 INFO:teuthology.orchestra.run.smithi019.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-12T03:07:10.259 INFO:teuthology.orchestra.run.smithi135.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 384 kB/s | 42 kB 00:00 2024-09-12T03:07:10.284 INFO:teuthology.orchestra.run.smithi135.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 887 kB/s | 44 kB 00:00 2024-09-12T03:07:10.309 INFO:teuthology.orchestra.run.smithi135.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-12T03:07:10.335 INFO:teuthology.orchestra.run.smithi135.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-12T03:07:10.360 INFO:teuthology.orchestra.run.smithi135.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-09-12T03:07:10.364 INFO:teuthology.orchestra.run.smithi121.stdout:(98/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-12T03:07:10.381 INFO:teuthology.orchestra.run.smithi121.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 329 kB/s | 58 kB 00:00 2024-09-12T03:07:10.385 INFO:teuthology.orchestra.run.smithi135.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-12T03:07:10.411 INFO:teuthology.orchestra.run.smithi135.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-12T03:07:10.423 INFO:teuthology.orchestra.run.smithi121.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T03:07:10.436 INFO:teuthology.orchestra.run.smithi135.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-12T03:07:10.448 INFO:teuthology.orchestra.run.smithi121.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-09-12T03:07:10.461 INFO:teuthology.orchestra.run.smithi135.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-12T03:07:10.473 INFO:teuthology.orchestra.run.smithi121.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-12T03:07:10.486 INFO:teuthology.orchestra.run.smithi135.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-12T03:07:10.503 INFO:teuthology.orchestra.run.smithi135.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 533 kB/s | 22 kB 00:00 2024-09-12T03:07:10.532 INFO:teuthology.orchestra.run.smithi121.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-09-12T03:07:10.554 INFO:teuthology.orchestra.run.smithi135.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-12T03:07:10.582 INFO:teuthology.orchestra.run.smithi121.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T03:07:10.596 INFO:teuthology.orchestra.run.smithi135.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-12T03:07:10.613 INFO:teuthology.orchestra.run.smithi135.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-12T03:07:10.624 INFO:teuthology.orchestra.run.smithi121.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 7.2 MB/s | 309 kB 00:00 2024-09-12T03:07:10.646 INFO:teuthology.orchestra.run.smithi135.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 3.3 MB/s | 309 kB 00:00 2024-09-12T03:07:10.649 INFO:teuthology.orchestra.run.smithi121.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 775 kB/s | 19 kB 00:00 2024-09-12T03:07:10.672 INFO:teuthology.orchestra.run.smithi135.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T03:07:10.691 INFO:teuthology.orchestra.run.smithi121.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-12T03:07:10.697 INFO:teuthology.orchestra.run.smithi135.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:07:10.722 INFO:teuthology.orchestra.run.smithi135.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T03:07:10.748 INFO:teuthology.orchestra.run.smithi135.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-12T03:07:10.792 INFO:teuthology.orchestra.run.smithi121.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-09-12T03:07:10.802 INFO:teuthology.orchestra.run.smithi019.stdout:(31/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 20 MB/s | 49 MB 00:02 2024-09-12T03:07:10.806 INFO:teuthology.orchestra.run.smithi135.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-12T03:07:10.817 INFO:teuthology.orchestra.run.smithi121.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:07:10.842 INFO:teuthology.orchestra.run.smithi121.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T03:07:10.848 INFO:teuthology.orchestra.run.smithi135.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T03:07:10.852 INFO:teuthology.orchestra.run.smithi019.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T03:07:10.874 INFO:teuthology.orchestra.run.smithi135.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:07:10.899 INFO:teuthology.orchestra.run.smithi135.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T03:07:10.909 INFO:teuthology.orchestra.run.smithi121.stdout:(111/156): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-12T03:07:10.910 INFO:teuthology.orchestra.run.smithi019.stdout:(33/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 4.8 MB/s | 3.6 MB 00:00 2024-09-12T03:07:10.928 INFO:teuthology.orchestra.run.smithi019.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 658 kB/s | 49 kB 00:00 2024-09-12T03:07:10.935 INFO:teuthology.orchestra.run.smithi121.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-09-12T03:07:10.953 INFO:teuthology.orchestra.run.smithi019.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 395 kB/s | 16 kB 00:00 2024-09-12T03:07:10.978 INFO:teuthology.orchestra.run.smithi019.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T03:07:10.985 INFO:teuthology.orchestra.run.smithi121.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-12T03:07:11.010 INFO:teuthology.orchestra.run.smithi121.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:07:11.058 INFO:teuthology.orchestra.run.smithi135.stdout:(116/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2024-09-12T03:07:11.060 INFO:teuthology.orchestra.run.smithi121.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 647 kB/s | 32 kB 00:00 2024-09-12T03:07:11.083 INFO:teuthology.orchestra.run.smithi135.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 77 kB/s | 14 kB 00:00 2024-09-12T03:07:11.086 INFO:teuthology.orchestra.run.smithi121.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 561 kB/s | 14 kB 00:00 2024-09-12T03:07:11.095 INFO:teuthology.orchestra.run.smithi019.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 8.7 MB/s | 7.4 MB 00:00 2024-09-12T03:07:11.112 INFO:teuthology.orchestra.run.smithi019.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 137 kB/s | 18 kB 00:00 2024-09-12T03:07:11.119 INFO:teuthology.orchestra.run.smithi121.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-09-12T03:07:11.133 INFO:teuthology.orchestra.run.smithi135.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 8.4 MB/s | 358 kB 00:00 2024-09-12T03:07:11.145 INFO:teuthology.orchestra.run.smithi121.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T03:07:11.145 INFO:teuthology.orchestra.run.smithi019.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 3.9 MB/s | 765 kB 00:00 2024-09-12T03:07:11.167 INFO:teuthology.orchestra.run.smithi135.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 9.4 MB/s | 303 kB 00:00 2024-09-12T03:07:11.178 INFO:teuthology.orchestra.run.smithi121.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-12T03:07:11.204 INFO:teuthology.orchestra.run.smithi121.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 8.6 MB/s | 220 kB 00:00 2024-09-12T03:07:11.237 INFO:teuthology.orchestra.run.smithi121.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-12T03:07:11.250 INFO:teuthology.orchestra.run.smithi135.stdout:(120/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-12T03:07:11.254 INFO:teuthology.orchestra.run.smithi019.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-09-12T03:07:11.271 INFO:teuthology.orchestra.run.smithi121.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T03:07:11.275 INFO:teuthology.orchestra.run.smithi135.stdout:(121/156): python3-cheroot-8.6.0-4.el9.noarch.r 790 kB/s | 172 kB 00:00 2024-09-12T03:07:11.296 INFO:teuthology.orchestra.run.smithi121.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-12T03:07:11.300 INFO:teuthology.orchestra.run.smithi135.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.6 MB/s | 220 kB 00:00 2024-09-12T03:07:11.321 INFO:teuthology.orchestra.run.smithi121.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 706 kB/s | 18 kB 00:00 2024-09-12T03:07:11.326 INFO:teuthology.orchestra.run.smithi135.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-12T03:07:11.338 INFO:teuthology.orchestra.run.smithi019.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 996 kB/s | 191 kB 00:00 2024-09-12T03:07:11.347 INFO:teuthology.orchestra.run.smithi121.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-09-12T03:07:11.351 INFO:teuthology.orchestra.run.smithi135.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-12T03:07:11.363 INFO:teuthology.orchestra.run.smithi019.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-12T03:07:11.372 INFO:teuthology.orchestra.run.smithi121.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-12T03:07:11.376 INFO:teuthology.orchestra.run.smithi135.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-12T03:07:11.388 INFO:teuthology.orchestra.run.smithi019.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-12T03:07:11.397 INFO:teuthology.orchestra.run.smithi121.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 788 kB/s | 20 kB 00:00 2024-09-12T03:07:11.401 INFO:teuthology.orchestra.run.smithi135.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-09-12T03:07:11.414 INFO:teuthology.orchestra.run.smithi019.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-12T03:07:11.427 INFO:teuthology.orchestra.run.smithi135.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-09-12T03:07:11.440 INFO:teuthology.orchestra.run.smithi121.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-12T03:07:11.440 INFO:teuthology.orchestra.run.smithi019.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T03:07:11.452 INFO:teuthology.orchestra.run.smithi135.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-12T03:07:11.464 INFO:teuthology.orchestra.run.smithi019.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T03:07:11.464 INFO:teuthology.orchestra.run.smithi121.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:07:11.478 INFO:teuthology.orchestra.run.smithi135.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T03:07:11.490 INFO:teuthology.orchestra.run.smithi121.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:07:11.503 INFO:teuthology.orchestra.run.smithi135.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:07:11.515 INFO:teuthology.orchestra.run.smithi121.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-12T03:07:11.523 INFO:teuthology.orchestra.run.smithi019.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T03:07:11.528 INFO:teuthology.orchestra.run.smithi135.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-12T03:07:11.549 INFO:teuthology.orchestra.run.smithi121.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T03:07:11.562 INFO:teuthology.orchestra.run.smithi135.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-12T03:07:11.574 INFO:teuthology.orchestra.run.smithi121.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 653 kB/s | 16 kB 00:00 2024-09-12T03:07:11.587 INFO:teuthology.orchestra.run.smithi135.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 666 kB/s | 16 kB 00:00 2024-09-12T03:07:11.600 INFO:teuthology.orchestra.run.smithi121.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T03:07:11.612 INFO:teuthology.orchestra.run.smithi135.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T03:07:11.614 INFO:teuthology.orchestra.run.smithi019.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-09-12T03:07:11.617 INFO:teuthology.orchestra.run.smithi121.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-12T03:07:11.637 INFO:teuthology.orchestra.run.smithi135.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-12T03:07:11.648 INFO:teuthology.orchestra.run.smithi019.stdout:(49/156): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-09-12T03:07:11.659 INFO:teuthology.orchestra.run.smithi121.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-09-12T03:07:11.671 INFO:teuthology.orchestra.run.smithi135.stdout:(136/156): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-09-12T03:07:11.673 INFO:teuthology.orchestra.run.smithi019.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-12T03:07:11.684 INFO:teuthology.orchestra.run.smithi121.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-09-12T03:07:11.696 INFO:teuthology.orchestra.run.smithi135.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-09-12T03:07:11.709 INFO:teuthology.orchestra.run.smithi121.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:07:11.722 INFO:teuthology.orchestra.run.smithi135.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:07:11.732 INFO:teuthology.orchestra.run.smithi019.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-12T03:07:11.734 INFO:teuthology.orchestra.run.smithi121.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T03:07:11.760 INFO:teuthology.orchestra.run.smithi121.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T03:07:11.763 INFO:teuthology.orchestra.run.smithi135.stdout:(139/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-12T03:07:11.766 INFO:teuthology.orchestra.run.smithi019.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-09-12T03:07:11.783 INFO:teuthology.orchestra.run.smithi019.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-09-12T03:07:11.789 INFO:teuthology.orchestra.run.smithi135.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-12T03:07:11.793 INFO:teuthology.orchestra.run.smithi121.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-12T03:07:11.808 INFO:teuthology.orchestra.run.smithi019.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.3 MB/s | 313 kB 00:00 2024-09-12T03:07:11.814 INFO:teuthology.orchestra.run.smithi135.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 389 kB/s | 36 kB 00:00 2024-09-12T03:07:11.819 INFO:teuthology.orchestra.run.smithi121.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:07:11.833 INFO:teuthology.orchestra.run.smithi019.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-12T03:07:11.839 INFO:teuthology.orchestra.run.smithi135.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T03:07:11.844 INFO:teuthology.orchestra.run.smithi121.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-12T03:07:11.858 INFO:teuthology.orchestra.run.smithi019.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-12T03:07:11.865 INFO:teuthology.orchestra.run.smithi135.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T03:07:11.869 INFO:teuthology.orchestra.run.smithi121.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-09-12T03:07:11.884 INFO:teuthology.orchestra.run.smithi019.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-12T03:07:11.890 INFO:teuthology.orchestra.run.smithi135.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T03:07:11.903 INFO:teuthology.orchestra.run.smithi121.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-12T03:07:11.909 INFO:teuthology.orchestra.run.smithi019.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-09-12T03:07:11.915 INFO:teuthology.orchestra.run.smithi135.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-12T03:07:11.928 INFO:teuthology.orchestra.run.smithi121.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-12T03:07:11.941 INFO:teuthology.orchestra.run.smithi135.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T03:07:11.966 INFO:teuthology.orchestra.run.smithi135.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T03:07:11.984 INFO:teuthology.orchestra.run.smithi019.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-09-12T03:07:11.987 INFO:teuthology.orchestra.run.smithi121.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T03:07:12.016 INFO:teuthology.orchestra.run.smithi135.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T03:07:12.035 INFO:teuthology.orchestra.run.smithi019.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 4.5 MB/s | 581 kB 00:00 2024-09-12T03:07:12.060 INFO:teuthology.orchestra.run.smithi019.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 324 kB/s | 24 kB 00:00 2024-09-12T03:07:12.085 INFO:teuthology.orchestra.run.smithi019.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-09-12T03:07:12.160 INFO:teuthology.orchestra.run.smithi019.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-12T03:07:12.186 INFO:teuthology.orchestra.run.smithi019.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-09-12T03:07:12.191 INFO:teuthology.orchestra.run.smithi135.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-12T03:07:12.211 INFO:teuthology.orchestra.run.smithi019.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 765 kB/s | 38 kB 00:00 2024-09-12T03:07:12.258 INFO:teuthology.orchestra.run.smithi135.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-12T03:07:12.304 INFO:teuthology.orchestra.run.smithi121.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.1 MB/s | 3.4 MB 00:00 2024-09-12T03:07:12.308 INFO:teuthology.orchestra.run.smithi135.stdout:(151/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 9.2 MB/s | 3.2 MB 00:00 2024-09-12T03:07:12.334 INFO:teuthology.orchestra.run.smithi135.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 542 kB/s | 41 kB 00:00 2024-09-12T03:07:12.363 INFO:teuthology.orchestra.run.smithi121.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-12T03:07:12.409 INFO:teuthology.orchestra.run.smithi135.stdout:(153/156): librados2-19.1.1-298.gb53b6d3e.el9.x 7.3 MB/s | 3.4 MB 00:00 2024-09-12T03:07:12.444 INFO:teuthology.orchestra.run.smithi135.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-12T03:07:12.468 INFO:teuthology.orchestra.run.smithi135.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.5 MB/s | 249 kB 00:00 2024-09-12T03:07:12.495 INFO:teuthology.orchestra.run.smithi019.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.1 MB/s | 3.0 MB 00:00 2024-09-12T03:07:12.593 INFO:teuthology.orchestra.run.smithi135.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-09-12T03:07:12.596 INFO:teuthology.orchestra.run.smithi121.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 7.9 MB/s | 2.3 MB 00:00 2024-09-12T03:07:12.600 INFO:teuthology.orchestra.run.smithi135.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:07:12.600 INFO:teuthology.orchestra.run.smithi135.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-12T03:07:12.655 INFO:teuthology.orchestra.run.smithi121.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 6.2 MB/s | 1.8 MB 00:00 2024-09-12T03:07:12.680 INFO:teuthology.orchestra.run.smithi121.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 488 kB/s | 41 kB 00:00 2024-09-12T03:07:12.714 INFO:teuthology.orchestra.run.smithi121.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-12T03:07:12.781 INFO:teuthology.orchestra.run.smithi121.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-12T03:07:12.864 INFO:teuthology.orchestra.run.smithi121.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-12T03:07:12.928 INFO:teuthology.orchestra.run.smithi019.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-09-12T03:07:13.195 INFO:teuthology.orchestra.run.smithi019.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-12T03:07:13.264 INFO:teuthology.orchestra.run.smithi019.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-12T03:07:13.472 INFO:teuthology.orchestra.run.smithi019.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-09-12T03:07:13.613 INFO:teuthology.orchestra.run.smithi019.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.5 MB/s | 4.9 MB 00:01 2024-09-12T03:07:13.671 INFO:teuthology.orchestra.run.smithi019.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-12T03:07:13.697 INFO:teuthology.orchestra.run.smithi019.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-09-12T03:07:13.738 INFO:teuthology.orchestra.run.smithi019.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-12T03:07:13.817 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction check 2024-09-12T03:07:13.822 INFO:teuthology.orchestra.run.smithi019.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.9 MB/s | 245 kB 00:00 2024-09-12T03:07:13.914 INFO:teuthology.orchestra.run.smithi019.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-09-12T03:07:13.956 INFO:teuthology.orchestra.run.smithi019.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 616 kB/s | 26 kB 00:00 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction check succeeded. 2024-09-12T03:07:13.966 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction test 2024-09-12T03:07:13.998 INFO:teuthology.orchestra.run.smithi019.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-09-12T03:07:14.073 INFO:teuthology.orchestra.run.smithi019.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-12T03:07:14.115 INFO:teuthology.orchestra.run.smithi019.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-09-12T03:07:14.323 INFO:teuthology.orchestra.run.smithi121.stdout:(156/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x 5.4 MB/s | 49 MB 00:09 2024-09-12T03:07:14.329 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:07:14.329 INFO:teuthology.orchestra.run.smithi121.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T03:07:15.023 INFO:teuthology.orchestra.run.smithi019.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.3 MB/s | 5.8 MB 00:01 2024-09-12T03:07:15.115 INFO:teuthology.orchestra.run.smithi019.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-09-12T03:07:15.207 INFO:teuthology.orchestra.run.smithi019.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-12T03:07:15.266 INFO:teuthology.orchestra.run.smithi019.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T03:07:15.308 INFO:teuthology.orchestra.run.smithi019.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-12T03:07:15.416 INFO:teuthology.orchestra.run.smithi019.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-12T03:07:15.517 INFO:teuthology.orchestra.run.smithi019.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-09-12T03:07:15.527 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-09-12T03:07:15.567 INFO:teuthology.orchestra.run.smithi019.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-12T03:07:15.601 INFO:teuthology.orchestra.run.smithi019.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-12T03:07:15.679 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-09-12T03:07:15.679 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-09-12T03:07:15.692 INFO:teuthology.orchestra.run.smithi019.stdout:(90/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.2 MB/s | 5.1 MB 00:01 2024-09-12T03:07:15.709 INFO:teuthology.orchestra.run.smithi019.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 493 kB/s | 54 kB 00:00 2024-09-12T03:07:15.735 INFO:teuthology.orchestra.run.smithi019.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.3 MB/s | 279 kB 00:00 2024-09-12T03:07:15.752 INFO:teuthology.orchestra.run.smithi019.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-09-12T03:07:15.762 INFO:teuthology.orchestra.run.smithi135.stdout:Transaction test succeeded. 2024-09-12T03:07:15.764 INFO:teuthology.orchestra.run.smithi135.stdout:Running transaction 2024-09-12T03:07:15.769 INFO:teuthology.orchestra.run.smithi019.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-12T03:07:15.802 INFO:teuthology.orchestra.run.smithi019.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 1.1 MB/s | 36 kB 00:00 2024-09-12T03:07:15.844 INFO:teuthology.orchestra.run.smithi019.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-09-12T03:07:15.870 INFO:teuthology.orchestra.run.smithi019.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 707 kB/s | 47 kB 00:00 2024-09-12T03:07:15.912 INFO:teuthology.orchestra.run.smithi019.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-12T03:07:15.954 INFO:teuthology.orchestra.run.smithi019.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 2.2 MB/s | 96 kB 00:00 2024-09-12T03:07:15.987 INFO:teuthology.orchestra.run.smithi019.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-12T03:07:16.013 INFO:teuthology.orchestra.run.smithi019.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T03:07:16.038 INFO:teuthology.orchestra.run.smithi019.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-09-12T03:07:16.130 INFO:teuthology.orchestra.run.smithi019.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-09-12T03:07:16.222 INFO:teuthology.orchestra.run.smithi019.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-12T03:07:16.255 INFO:teuthology.orchestra.run.smithi019.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.0 MB/s | 309 kB 00:00 2024-09-12T03:07:16.289 INFO:teuthology.orchestra.run.smithi019.stdout:(106/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.4 MB/s | 863 kB 00:00 2024-09-12T03:07:16.323 INFO:teuthology.orchestra.run.smithi019.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-12T03:07:16.348 INFO:teuthology.orchestra.run.smithi019.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-12T03:07:16.373 INFO:teuthology.orchestra.run.smithi019.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T03:07:16.398 INFO:teuthology.orchestra.run.smithi019.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-12T03:07:16.424 INFO:teuthology.orchestra.run.smithi019.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-09-12T03:07:16.466 INFO:teuthology.orchestra.run.smithi019.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-12T03:07:16.516 INFO:teuthology.orchestra.run.smithi019.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-12T03:07:16.541 INFO:teuthology.orchestra.run.smithi019.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T03:07:16.567 INFO:teuthology.orchestra.run.smithi019.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T03:07:16.592 INFO:teuthology.orchestra.run.smithi019.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-12T03:07:16.617 INFO:teuthology.orchestra.run.smithi019.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-12T03:07:16.651 INFO:teuthology.orchestra.run.smithi019.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2024-09-12T03:07:16.710 INFO:teuthology.orchestra.run.smithi019.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-12T03:07:16.744 INFO:teuthology.orchestra.run.smithi019.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-12T03:07:16.769 INFO:teuthology.orchestra.run.smithi019.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T03:07:16.802 INFO:teuthology.orchestra.run.smithi019.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T03:07:16.828 INFO:teuthology.orchestra.run.smithi019.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-12T03:07:16.853 INFO:teuthology.orchestra.run.smithi019.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-12T03:07:16.995 INFO:teuthology.orchestra.run.smithi019.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 164 kB/s | 23 kB 00:00 2024-09-12T03:07:17.020 INFO:teuthology.orchestra.run.smithi019.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 781 kB/s | 19 kB 00:00 2024-09-12T03:07:17.045 INFO:teuthology.orchestra.run.smithi019.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-12T03:07:17.104 INFO:teuthology.orchestra.run.smithi019.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-09-12T03:07:17.121 INFO:teuthology.orchestra.run.smithi019.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-09-12T03:07:17.146 INFO:teuthology.orchestra.run.smithi019.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T03:07:17.180 INFO:teuthology.orchestra.run.smithi019.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-09-12T03:07:17.197 INFO:teuthology.orchestra.run.smithi019.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-12T03:07:17.214 INFO:teuthology.orchestra.run.smithi019.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 1.0 MB/s | 16 kB 00:00 2024-09-12T03:07:17.239 INFO:teuthology.orchestra.run.smithi019.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T03:07:17.264 INFO:teuthology.orchestra.run.smithi019.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-12T03:07:17.289 INFO:teuthology.orchestra.run.smithi019.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-12T03:07:17.315 INFO:teuthology.orchestra.run.smithi019.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-12T03:07:17.340 INFO:teuthology.orchestra.run.smithi019.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-12T03:07:17.374 INFO:teuthology.orchestra.run.smithi019.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-09-12T03:07:17.399 INFO:teuthology.orchestra.run.smithi019.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 611 kB/s | 36 kB 00:00 2024-09-12T03:07:17.424 INFO:teuthology.orchestra.run.smithi019.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-12T03:07:17.450 INFO:teuthology.orchestra.run.smithi019.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-12T03:07:17.475 INFO:teuthology.orchestra.run.smithi019.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-12T03:07:17.500 INFO:teuthology.orchestra.run.smithi019.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 800 kB/s | 20 kB 00:00 2024-09-12T03:07:17.526 INFO:teuthology.orchestra.run.smithi019.stdout:(145/156): re2-20211101-20.el9.x86_64.rpm 7.6 MB/s | 191 kB 00:00 2024-09-12T03:07:17.551 INFO:teuthology.orchestra.run.smithi019.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-09-12T03:07:17.601 INFO:teuthology.orchestra.run.smithi019.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-12T03:07:17.647 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-09-12T03:07:17.647 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-09-12T03:07:17.843 INFO:teuthology.orchestra.run.smithi019.stdout:(148/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T03:07:17.910 INFO:teuthology.orchestra.run.smithi019.stdout:(149/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 10 MB/s | 3.2 MB 00:00 2024-09-12T03:07:18.377 INFO:teuthology.orchestra.run.smithi019.stdout:(150/156): NetworkManager-1.49.90-1.el9.x86_64. 4.3 MB/s | 2.3 MB 00:00 2024-09-12T03:07:18.427 INFO:teuthology.orchestra.run.smithi019.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 818 kB/s | 41 kB 00:00 2024-09-12T03:07:18.503 INFO:teuthology.orchestra.run.smithi019.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-12T03:07:18.597 INFO:teuthology.orchestra.run.smithi019.stdout:(153/156): NetworkManager-libnm-1.49.90-1.el9.x 2.6 MB/s | 1.8 MB 00:00 2024-09-12T03:07:18.736 INFO:teuthology.orchestra.run.smithi019.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-12T03:07:19.053 INFO:teuthology.orchestra.run.smithi019.stdout:(155/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.8 MB/s | 16 MB 00:03 2024-09-12T03:07:19.329 INFO:teuthology.orchestra.run.smithi019.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 3.3 MB/s | 2.4 MB 00:00 2024-09-12T03:07:19.333 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:07:19.333 INFO:teuthology.orchestra.run.smithi019.stdout:Total 18 MB/s | 226 MB 00:12 2024-09-12T03:07:20.588 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-12T03:07:20.756 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-12T03:07:20.756 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-12T03:07:22.032 INFO:teuthology.orchestra.run.smithi135.stdout: Preparing : 1/1 2024-09-12T03:07:22.121 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:07:22.253 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:07:22.433 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:07:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:07:22.499 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:07:22.543 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:07:22.872 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:07:23.016 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:23.075 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-12T03:07:23.075 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-12T03:07:23.121 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:23.150 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:23.242 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:23.271 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:07:23.417 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:07:23.476 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:07:23.504 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:23.607 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:23.630 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:23.692 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:23.715 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:23.736 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:23.878 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-09-12T03:07:23.915 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:24.009 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:07:24.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:07:24.069 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:24.109 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:07:24.151 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:07:24.171 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:07:24.177 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:07:24.219 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:07:24.285 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:24.356 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:24.542 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:07:24.631 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:24.662 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:24.740 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:07:24.775 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:24.813 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:24.834 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:07:24.869 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:07:24.903 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:24.904 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:07:24.942 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:07:24.978 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:07:25.022 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:07:25.062 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:07:25.088 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:07:25.106 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:07:25.138 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:07:25.139 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:07:25.166 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:07:25.167 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:25.236 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:07:25.253 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:25.277 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:25.289 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:07:25.319 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:07:25.339 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:25.361 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:25.363 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:07:25.385 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:25.439 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:07:25.481 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:07:25.516 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:07:25.545 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:07:25.562 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:25.615 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:07:25.663 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:07:25.691 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:25.738 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:07:25.815 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:07:25.870 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:07:25.904 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:25.923 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:07:25.973 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:26.189 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:26.295 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:07:26.392 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:07:26.420 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:07:26.446 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:07:26.463 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:07:26.528 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:07:26.557 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:07:26.572 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:07:26.599 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:07:26.604 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:07:26.638 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:07:26.656 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:07:26.679 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:07:26.705 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:07:26.733 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:07:26.802 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:07:26.844 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:07:26.856 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:07:26.887 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:07:26.899 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:07:26.931 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:07:26.953 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:07:27.005 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:07:27.027 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:07:27.041 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:07:27.058 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:07:27.084 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:07:27.094 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:07:27.121 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:07:27.141 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:07:27.177 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:07:27.189 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:07:27.228 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:07:27.245 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:07:27.270 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:07:27.317 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:07:27.406 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:07:27.449 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:07:27.456 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:07:27.497 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:07:27.505 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:07:27.599 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:07:27.707 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:07:27.872 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:07:27.905 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T03:07:27.929 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:07:28.006 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:07:28.021 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:07:28.102 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:07:28.130 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:07:28.141 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:07:28.178 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:07:28.183 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:07:28.206 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:07:28.221 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:07:28.240 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:07:28.263 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:07:28.279 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:07:28.311 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:07:28.341 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:07:28.375 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:07:28.428 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:07:28.459 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:07:28.492 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:07:28.493 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:07:28.546 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:07:28.579 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:07:28.620 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:07:28.632 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:07:28.660 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:07:28.663 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:07:28.695 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:07:28.742 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:07:28.766 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:07:28.778 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:07:28.814 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:07:28.830 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:07:28.880 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:07:29.014 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:07:29.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:07:29.098 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:07:29.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:07:29.483 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:07:29.534 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:07:29.578 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:07:29.582 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:07:29.611 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:07:29.613 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-12T03:07:29.615 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T03:07:29.640 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:07:29.713 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:07:29.716 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T03:07:29.719 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:07:29.756 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T03:07:29.769 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:07:29.813 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:07:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T03:07:29.851 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:07:29.859 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T03:07:29.884 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T03:07:29.893 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:07:29.905 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:07:29.924 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:07:29.933 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:07:29.940 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T03:07:29.967 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:07:30.015 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:07:30.046 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:07:30.076 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:07:30.111 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:07:30.115 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:07:30.138 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:07:30.144 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:07:30.185 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:07:30.207 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:07:30.220 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:07:30.308 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:07:30.353 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:07:30.381 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T03:07:30.384 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:07:30.485 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:07:30.493 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:30.543 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:07:30.620 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:07:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:30.721 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:07:30.749 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:07:30.848 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:07:30.930 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T03:07:30.962 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:07:30.988 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T03:07:31.016 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:07:31.150 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:31.213 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T03:07:31.250 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:07:31.251 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:31.271 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T03:07:31.290 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:07:31.324 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:07:31.421 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:07:31.472 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:31.479 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:07:31.533 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:07:31.610 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:07:31.644 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:07:31.651 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T03:07:31.766 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T03:07:31.826 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:07:31.867 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:07:31.879 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:31.943 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:07:31.950 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:32.180 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:07:32.184 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T03:07:32.267 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:07:32.295 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:07:32.301 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T03:07:32.328 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:07:32.406 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T03:07:32.444 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T03:07:32.483 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T03:07:32.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T03:07:32.583 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:07:32.618 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T03:07:32.637 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:07:32.666 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T03:07:32.682 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:32.697 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:07:32.709 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T03:07:32.742 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T03:07:32.774 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:32.778 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T03:07:32.804 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:07:32.840 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:07:32.848 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T03:07:32.901 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T03:07:32.931 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T03:07:32.977 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T03:07:33.062 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T03:07:33.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T03:07:33.146 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T03:07:33.174 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T03:07:33.248 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T03:07:33.289 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T03:07:33.319 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:07:33.366 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T03:07:33.385 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:33.457 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:33.487 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:07:33.502 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T03:07:33.559 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T03:07:33.909 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:07:33.996 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:07:34.023 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:07:34.057 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:07:34.120 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T03:07:34.206 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T03:07:34.241 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T03:07:34.292 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T03:07:34.338 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T03:07:34.375 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:07:34.385 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T03:07:34.420 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:34.494 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:34.533 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:07:34.541 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T03:07:34.569 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:07:34.600 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T03:07:34.656 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T03:07:34.730 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T03:07:34.767 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T03:07:34.803 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T03:07:34.852 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T03:07:34.895 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T03:07:34.938 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T03:07:34.981 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T03:07:35.049 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:07:35.082 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:35.124 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T03:07:35.152 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:35.175 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:07:35.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T03:07:35.432 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T03:07:35.538 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T03:07:35.581 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T03:07:35.681 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T03:07:35.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T03:07:35.913 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T03:07:36.040 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T03:07:36.066 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:36.101 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:36.182 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T03:07:36.220 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T03:07:36.249 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T03:07:36.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T03:07:36.330 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T03:07:36.370 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T03:07:36.400 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T03:07:36.435 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T03:07:36.468 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T03:07:36.510 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T03:07:36.547 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T03:07:36.638 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T03:07:36.645 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:36.687 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:36.720 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T03:07:36.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T03:07:36.779 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:36.779 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:07:36.779 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:07:36.779 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:36.813 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:36.859 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T03:07:36.919 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T03:07:37.475 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:37.690 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T03:07:37.699 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:37.732 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T03:07:37.766 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T03:07:37.868 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T03:07:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T03:07:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T03:07:38.069 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T03:07:38.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T03:07:38.245 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:38.283 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:38.293 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T03:07:38.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T03:07:38.376 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:38.376 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:07:38.376 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:07:38.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:38.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T03:07:38.410 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:39.102 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T03:07:39.221 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T03:07:40.558 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T03:07:40.639 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T03:07:40.667 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T03:07:40.700 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T03:07:41.027 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T03:07:41.071 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:41.171 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T03:07:41.201 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T03:07:41.246 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T03:07:41.751 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T03:07:41.782 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:41.854 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T03:07:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T03:07:44.258 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:44.297 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:44.879 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T03:07:44.917 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:44.991 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T03:07:44.992 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T03:07:44.992 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T03:07:44.992 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:45.035 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:48.726 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /sys 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /proc 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /mnt 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /var/tmp 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /home 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /root 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout:skipping the directory /tmp 2024-09-12T03:07:48.727 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:48.908 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:48.925 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:48.925 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:48.925 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:07:48.925 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:50.261 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:50.261 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-09-12T03:07:50.261 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-09-12T03:07:50.262 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:50.395 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:50.422 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:50.430 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:50.446 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:50.447 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:50.447 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:07:50.447 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:50.645 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:50.669 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:51.112 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:07:51.136 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:07:51.137 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:51.137 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:07:51.137 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:07:51.137 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:07:51.137 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:51.242 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:07:51.299 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:07:51.354 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:07:51.541 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:07:51.588 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:07:51.666 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:07:51.754 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:07:51.901 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:07:51.934 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:51.946 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:07:51.960 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:51.960 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:51.960 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:07:51.960 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:51.960 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:51.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:51.986 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:07:52.014 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:07:52.184 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:52.207 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:52.207 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:52.208 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:07:52.208 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:52.208 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:52.208 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:52.466 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:07:52.496 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:07:52.528 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:07:52.590 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:07:52.627 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:07:52.643 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:07:52.666 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:07:52.666 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:52.667 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:07:52.667 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:07:52.667 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:07:52.667 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:52.765 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:07:52.839 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:07:52.894 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:07:53.078 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:07:53.127 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:07:53.205 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:07:53.301 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:07:53.441 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:07:53.495 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:07:53.526 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:07:53.554 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:07:53.673 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:07:53.704 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:07:53.938 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:07:53.978 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:07:54.009 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:07:54.072 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:07:54.101 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:07:54.667 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:07:54.696 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:07:54.799 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:07:54.958 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:07:54.990 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:07:55.014 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:07:55.014 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:55.015 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:07:55.015 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:07:55.015 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:07:55.015 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:55.066 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:07:55.077 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:07:55.154 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:07:55.175 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:07:55.186 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:07:55.230 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:07:56.152 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:07:56.156 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:07:56.178 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:07:56.185 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:07:56.208 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:07:56.208 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:56.208 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:07:56.208 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:07:56.209 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:07:56.209 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:56.252 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:07:56.275 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:07:56.276 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:56.276 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:07:56.276 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:56.279 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:07:56.319 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:07:56.343 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:56.441 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:07:56.481 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:07:56.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:56.548 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:07:56.559 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:07:56.645 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:07:56.658 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:07:56.668 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:07:56.669 INFO:teuthology.orchestra.run.smithi135.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:56.669 INFO:teuthology.orchestra.run.smithi135.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:07:56.669 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:07:56.669 INFO:teuthology.orchestra.run.smithi135.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:07:56.669 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:07:56.704 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-09-12T03:07:57.574 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-09-12T03:07:57.575 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-09-12T03:07:57.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:57.629 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:07:57.659 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:07:57.682 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:07:57.683 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:57.683 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:07:57.683 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:07:57.683 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:07:57.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:57.726 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:07:57.749 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:07:57.749 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:57.749 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:07:57.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:57.763 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:57.781 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T03:07:57.781 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:57.781 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T03:07:57.781 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:57.793 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:07:57.816 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:07:57.816 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:57.816 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:07:57.817 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:07:57.817 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:07:57.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:58.119 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:07:58.142 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:07:59.317 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T03:07:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T03:07:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T03:07:59.346 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:07:59.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T03:07:59.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:00.068 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T03:08:00.093 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:00.207 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T03:08:00.265 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T03:08:00.329 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T03:08:00.533 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T03:08:00.586 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T03:08:00.668 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T03:08:00.782 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T03:08:00.932 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T03:08:00.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T03:08:01.009 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T03:08:01.037 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T03:08:01.216 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:08:01.257 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:08:01.297 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:08:01.329 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:08:01.357 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:08:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:08:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T03:08:01.539 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:08:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T03:08:01.578 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:08:01.609 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:08:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T03:08:01.638 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:08:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:08:01.685 INFO:teuthology.orchestra.run.smithi135.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:08:02.264 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:02.299 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:02.320 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:02.731 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:08:02.763 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T03:08:02.781 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:08:02.802 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:08:02.821 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:08:02.853 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:08:02.880 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:08:03.060 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:08:03.102 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:08:03.132 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:08:03.159 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:08:03.162 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:08:03.200 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:08:03.237 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:08:03.277 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:08:03.358 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:08:03.398 INFO:teuthology.orchestra.run.smithi135.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:08:03.423 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:08:03.449 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:08:03.449 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:03.534 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:03.534 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:03.601 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:03.626 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:08:03.631 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:03.631 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:03.761 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:03.798 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:03.803 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:03.803 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:03.818 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:03.819 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T03:08:03.843 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:08:03.887 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:03.949 INFO:teuthology.orchestra.run.smithi135.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:03.953 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T03:08:04.130 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T03:08:04.162 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T03:08:04.188 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:04.238 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:08:04.250 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T03:08:04.351 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T03:08:04.416 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T03:08:04.649 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:08:04.737 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:08:04.777 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:08:04.858 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:08:04.890 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:08:04.915 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:08:04.941 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:08:04.941 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:05.017 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:05.084 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:05.109 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:08:05.114 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:05.114 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:05.303 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:05.303 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:05.370 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T03:08:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:08:05.433 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T03:08:05.458 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:05.499 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:08:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T03:08:05.524 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:05.524 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T03:08:05.524 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:05.583 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:08:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T03:08:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T03:08:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:08:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T03:08:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:05.942 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:08:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T03:08:05.968 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T03:08:05.968 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T03:08:05.969 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:08:05.969 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T03:08:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:09.927 INFO:teuthology.orchestra.run.smithi135.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:09.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:08:09.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:08:09.927 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:08:09.928 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:08:09.929 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:08:09.930 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:08:09.931 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:08:09.932 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:08:09.933 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:08:09.934 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:08:09.935 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:08:09.936 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:08:09.937 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:08:09.938 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:08:09.939 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:08:09.940 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:08:09.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:08:09.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:08:09.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:08:09.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:08:09.941 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:08:10.796 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T03:08:10.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T03:08:10.879 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T03:08:10.911 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T03:08:10.938 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T03:08:11.145 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T03:08:11.197 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:08:11.206 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:08:11.207 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:08:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:08:11.209 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:08:11.210 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:08:11.211 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:08:11.212 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:08:11.213 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:08:11.214 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:08:11.215 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:08:11.216 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:08:11.217 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:08:11.218 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:08:11.219 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:08:11.223 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T03:08:11.253 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T03:08:11.307 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T03:08:11.973 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:12.013 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:12.035 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout:Upgraded: 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.653 INFO:teuthology.orchestra.run.smithi135.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout:Installed: 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.654 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.655 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:08:12.656 INFO:teuthology.orchestra.run.smithi135.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:08:12.657 INFO:teuthology.orchestra.run.smithi135.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:08:12.658 INFO:teuthology.orchestra.run.smithi135.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:08:12.659 INFO:teuthology.orchestra.run.smithi135.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:08:12.660 INFO:teuthology.orchestra.run.smithi135.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:08:12.661 INFO:teuthology.orchestra.run.smithi135.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:08:12.662 INFO:teuthology.orchestra.run.smithi135.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:08:12.663 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:08:12.664 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:08:12.665 INFO:teuthology.orchestra.run.smithi135.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:08:12.666 INFO:teuthology.orchestra.run.smithi135.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:08:12.666 INFO:teuthology.orchestra.run.smithi135.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:08:12.666 INFO:teuthology.orchestra.run.smithi135.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:08:12.666 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:08:12.666 INFO:teuthology.orchestra.run.smithi135.stdout:Complete! 2024-09-12T03:08:12.941 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T03:08:13.034 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T03:08:13.055 DEBUG:teuthology.parallel:result is None 2024-09-12T03:08:13.087 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T03:08:13.185 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T03:08:13.221 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T03:08:13.247 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T03:08:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T03:08:13.281 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:13.373 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T03:08:13.373 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:13.458 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T03:08:13.482 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T03:08:13.489 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:13.489 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:13.676 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T03:08:13.676 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:13.732 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:08:13.733 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.734 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:08:13.735 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.736 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.737 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:08:13.738 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.739 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:08:13.740 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:08:13.741 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:08:13.742 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:08:13.743 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.744 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:08:13.745 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:08:13.746 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-09-12T03:08:13.769 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T03:08:13.837 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:14.165 DEBUG:teuthology.parallel:result is None 2024-09-12T03:08:19.409 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:19.409 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T03:08:19.409 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T03:08:19.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T03:08:19.411 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T03:08:19.412 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T03:08:19.413 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T03:08:19.414 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T03:08:19.415 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T03:08:19.416 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T03:08:19.417 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T03:08:19.418 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T03:08:19.419 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T03:08:19.420 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T03:08:19.421 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T03:08:19.422 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T03:08:19.422 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T03:08:19.422 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T03:08:19.422 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T03:08:19.422 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T03:08:22.184 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T03:08:22.184 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:22.185 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.187 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T03:08:22.188 INFO:teuthology.orchestra.run.smithi019.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T03:08:22.189 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T03:08:22.190 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T03:08:22.191 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T03:08:22.192 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T03:08:22.193 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T03:08:22.194 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.195 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T03:08:22.196 INFO:teuthology.orchestra.run.smithi019.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T03:08:22.197 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:08:22.198 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-12T03:08:22.619 DEBUG:teuthology.parallel:result is None 2024-09-12T03:08:22.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:22.761 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:08:22.784 INFO:teuthology.orchestra.run.smithi019.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:22.788 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:22.788 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:08:22.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:22.937 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:08:22.963 INFO:teuthology.orchestra.run.smithi121.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:22.964 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:22.964 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:08:22.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:23.113 DEBUG:teuthology.orchestra.run.smithi135:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T03:08:23.140 INFO:teuthology.orchestra.run.smithi135.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:23.140 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T03:08:23.140 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T03:08:23.142 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T03:08:23.142 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:08:23.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:08:23.174 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:08:23.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:08:23.207 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:08:23.208 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T03:08:23.240 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T03:08:23.240 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:08:23.240 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:08:23.270 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:08:23.340 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:08:23.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:08:23.370 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:08:23.440 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:08:23.440 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T03:08:23.469 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T03:08:23.536 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T03:08:23.536 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:08:23.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:08:23.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:08:23.636 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:08:23.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:08:23.665 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:08:23.735 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:08:23.735 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T03:08:23.764 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T03:08:23.831 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T03:08:23.832 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:08:23.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:08:23.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:08:23.932 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:08:23.932 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:08:23.963 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:08:24.032 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:08:24.032 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T03:08:24.061 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T03:08:24.129 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T03:08:24.229 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2}} 2024-09-12T03:08:24.229 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:24.229 INFO:tasks.cephadm:Cluster fsid is 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:08:24.230 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T03:08:24.230 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.19', 'mon.b': '172.21.15.121', 'mon.c': '172.21.15.135'} 2024-09-12T03:08:24.230 INFO:tasks.cephadm:First mon is mon.a on smithi019 2024-09-12T03:08:24.230 INFO:tasks.cephadm:First mgr is x 2024-09-12T03:08:24.230 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T03:08:24.230 DEBUG:teuthology.orchestra.run.smithi019:> sudo hostname $(hostname -s) 2024-09-12T03:08:24.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo hostname $(hostname -s) 2024-09-12T03:08:24.291 DEBUG:teuthology.orchestra.run.smithi135:> sudo hostname $(hostname -s) 2024-09-12T03:08:24.319 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T03:08:24.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:24.456 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T03:08:24.596 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:08:24.598 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:08:24.599 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T03:08:24.599 DEBUG:teuthology.orchestra.run.smithi019:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.011 INFO:teuthology.orchestra.run.smithi019.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:08 /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.011 DEBUG:teuthology.orchestra.run.smithi121:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.401 INFO:teuthology.orchestra.run.smithi121.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:08 /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.402 DEBUG:teuthology.orchestra.run.smithi135:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.798 INFO:teuthology.orchestra.run.smithi135.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 03:08 /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.798 DEBUG:teuthology.orchestra.run.smithi019:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.819 DEBUG:teuthology.orchestra.run.smithi121:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.839 DEBUG:teuthology.orchestra.run.smithi135:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:25.874 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T03:08:25.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:08:25.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:08:25.881 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T03:08:27.491 INFO:teuthology.orchestra.run.smithi019.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:08:27.507 INFO:teuthology.orchestra.run.smithi135.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:08:27.507 INFO:teuthology.orchestra.run.smithi121.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:08:54.042 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:08:54.042 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi019.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi019.stdout: "repo_digests": [ 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi019.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:08:54.043 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:09:00.654 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-12T03:09:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:09:00.654 INFO:teuthology.orchestra.run.smithi135.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:09:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: "repo_digests": [ 2024-09-12T03:09:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:09:00.655 INFO:teuthology.orchestra.run.smithi135.stdout: ] 2024-09-12T03:09:00.655 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-12T03:09:03.385 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-12T03:09:03.385 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T03:09:03.385 INFO:teuthology.orchestra.run.smithi121.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T03:09:03.385 INFO:teuthology.orchestra.run.smithi121.stdout: "repo_digests": [ 2024-09-12T03:09:03.386 INFO:teuthology.orchestra.run.smithi121.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T03:09:03.386 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-09-12T03:09:03.386 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-12T03:09:03.412 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph 2024-09-12T03:09:03.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph 2024-09-12T03:09:03.477 DEBUG:teuthology.orchestra.run.smithi135:> sudo mkdir -p /etc/ceph 2024-09-12T03:09:03.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /etc/ceph 2024-09-12T03:09:03.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /etc/ceph 2024-09-12T03:09:03.566 DEBUG:teuthology.orchestra.run.smithi135:> sudo chmod 777 /etc/ceph 2024-09-12T03:09:03.596 INFO:tasks.cephadm:Writing seed config... 2024-09-12T03:09:03.597 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T03:09:03.598 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T03:09:03.599 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T03:09:03.600 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T03:09:03.601 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T03:09:03.602 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:09:03.602 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T03:09:03.620 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 45fb56a2-70b4-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T03:09:03.621 DEBUG:teuthology.orchestra.run.smithi019:mon.a> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T03:09:03.664 DEBUG:teuthology.orchestra.run.smithi019:mgr.x> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T03:09:03.707 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T03:09:03.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.19 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:09:03.921 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-12T03:09:03.921 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '45fb56a2-70b4-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.19', '--skip-admin-label'] 2024-09-12T03:09:03.921 INFO:teuthology.orchestra.run.smithi019.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T03:09:03.922 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying podman|docker is present... 2024-09-12T03:09:03.945 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:09:03.946 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying lvm2 is present... 2024-09-12T03:09:03.946 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying time synchronization is in place... 2024-09-12T03:09:03.953 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:09:03.953 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:09:03.960 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:09:03.960 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:09:03.967 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-09-12T03:09:03.973 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-09-12T03:09:03.974 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:09:03.974 INFO:teuthology.orchestra.run.smithi019.stdout:Repeating the final host check... 2024-09-12T03:09:03.998 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 5.2.2 2024-09-12T03:09:03.998 INFO:teuthology.orchestra.run.smithi019.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T03:09:03.998 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl is present 2024-09-12T03:09:03.998 INFO:teuthology.orchestra.run.smithi019.stdout:lvcreate is present 2024-09-12T03:09:04.004 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T03:09:04.004 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T03:09:04.010 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T03:09:04.010 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:09:04.017 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout enabled 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout active 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:Unit chronyd.service is enabled and running 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:Host looks OK 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:Cluster fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:Acquiring lock 140442785113088 on /run/cephadm/45fb56a2-70b4-11ef-bcea-c7b262605968.lock 2024-09-12T03:09:04.024 INFO:teuthology.orchestra.run.smithi019.stdout:Lock 140442785113088 acquired on /run/cephadm/45fb56a2-70b4-11ef-bcea-c7b262605968.lock 2024-09-12T03:09:04.025 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 3300 ... 2024-09-12T03:09:04.025 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying IP 172.21.15.19 port 6789 ... 2024-09-12T03:09:04.025 INFO:teuthology.orchestra.run.smithi019.stdout:Base mon IP(s) is [172.21.15.19:3300, 172.21.15.19:6789], mon addrv is [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-12T03:09:04.028 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.19 metric 100 2024-09-12T03:09:04.029 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.19 metric 100 2024-09-12T03:09:04.031 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T03:09:04.031 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T03:09:04.031 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1787sec hoplimit 64 pref medium 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-09-12T03:09:04.034 INFO:teuthology.orchestra.run.smithi019.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T03:09:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-09-12T03:09:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:Mon IP `172.21.15.19` is in CIDR network `172.21.0.0/20` 2024-09-12T03:09:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T03:09:04.035 INFO:teuthology.orchestra.run.smithi019.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T03:09:04.036 INFO:teuthology.orchestra.run.smithi019.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T03:09:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T03:09:05.465 INFO:teuthology.orchestra.run.smithi019.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:09:05.465 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T03:09:05.465 INFO:teuthology.orchestra.run.smithi019.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T03:09:06.413 INFO:teuthology.orchestra.run.smithi019.stdout:stat: stdout 167 167 2024-09-12T03:09:06.413 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial keys... 2024-09-12T03:09:07.369 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBSW+JmgGsfLRAA5SUmUldjEiQXg64iEAishA== 2024-09-12T03:09:08.292 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBTW+Jmh7f0KhAA4ItVPGe2UD3kdKIaOqtY2w== 2024-09-12T03:09:09.322 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph-authtool: stdout AQBUW+JmALpGJhAA745qKcg4AjOF7jjOhFCRMA== 2024-09-12T03:09:09.323 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial monmap... 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool for a [v2:172.21.15.19:3300,v1:172.21.15.19:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: set fsid to 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:09:10.405 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mon... 2024-09-12T03:09:16.225 INFO:teuthology.orchestra.run.smithi019.stdout:create mon.a on 2024-09-12T03:09:16.788 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T03:09:17.292 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T03:09:17.882 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target. 2024-09-12T03:09:17.882 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target. 2024-09-12T03:09:18.746 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:09:18.746 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T03:09:18.954 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target.wants/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@.service. 2024-09-12T03:09:19.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:19 smithi019 podman[30567]: 2024-09-12 03:09:19.108683564 +0000 UTC m=+0.016778828 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:09:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:20 smithi019 podman[30567]: 2024-09-12 03:09:20.194094331 +0000 UTC m=+1.102189586 container create cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:09:21.598 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-12T03:09:21.606 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:09:21.606 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:09:21.606 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-12T03:09:21.606 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:09:21.607 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon to start... 2024-09-12T03:09:21.607 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mon... 2024-09-12T03:09:21.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 podman[30567]: 2024-09-12 03:09:21.515560423 +0000 UTC m=+2.423655677 container init cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T03:09:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 podman[30567]: 2024-09-12 03:09:21.519817003 +0000 UTC m=+2.427912258 container start cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:09:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 ceph-mon[30586]: mkfs 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 ceph-mon[30586]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:09:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 bash[30567]: cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 2024-09-12T03:09:21.826 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:21 smithi019 systemd[1]: Started Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:09:22.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:09:22.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: monmap epoch 1 2024-09-12T03:09:22.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:22.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: last_changed 2024-09-12T03:09:09.637034+0000 2024-09-12T03:09:22.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: min_mon_release 19 (squid) 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: election_strategy: 1 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: fsmap 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: mgrmap e1: no daemons active 2024-09-12T03:09:22.656 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:22 smithi019 ceph-mon[30586]: from='client.? 172.21.15.19:0/1175265956' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-12T03:09:22.658 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T03:09:22.664 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout id: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout services: 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.565569s) 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout data: 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T03:09:22.665 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f954d2e4640 1 Processor -- start 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f954d2e4640 1 -- start start 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f954d2e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f954d2e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9548108550 con 0x7f9548107b80 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f9546ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f9546ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57190/0 (socket says 172.21.15.19:57190) 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.057+0000 7f9546ffd640 1 -- 172.21.15.19:0/3492371482 learned_addr learned my addr 172.21.15.19:0/3492371482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.058+0000 7f9546ffd640 1 -- 172.21.15.19:0/3492371482 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9548106830 con 0x7f9548107b80 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.060+0000 7f9546ffd640 1 --2- 172.21.15.19:0/3492371482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9538009820 tx=0x7f953802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fb9e0c46d81f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.060+0000 7f9545ffb640 1 -- 172.21.15.19:0/3492371482 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9538037070 con 0x7f9548107b80 2024-09-12T03:09:22.666 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.060+0000 7f9545ffb640 1 -- 172.21.15.19:0/3492371482 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9538032440 con 0x7f9548107b80 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.060+0000 7f9545ffb640 1 -- 172.21.15.19:0/3492371482 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9538032740 con 0x7f9548107b80 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- 172.21.15.19:0/3492371482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 msgr2=0x7f9548107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 --2- 172.21.15.19:0/3492371482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9538009820 tx=0x7f953802a270 comp rx=0 tx=0).stop 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- 172.21.15.19:0/3492371482 shutdown_connections 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 --2- 172.21.15.19:0/3492371482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f9548107f80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- 172.21.15.19:0/3492371482 >> 172.21.15.19:0/3492371482 conn(0x7f9548101550 msgr2=0x7f9548103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- 172.21.15.19:0/3492371482 shutdown_connections 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- 172.21.15.19:0/3492371482 wait complete. 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 Processor -- start 2024-09-12T03:09:22.667 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.061+0000 7f954d2e4640 1 -- start start 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f954d2e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f954d2e4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9548106b80 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f9546ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f9546ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57198/0 (socket says 172.21.15.19:57198) 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f9546ffd640 1 -- 172.21.15.19:0/1175265956 learned_addr learned my addr 172.21.15.19:0/1175265956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f9546ffd640 1 -- 172.21.15.19:0/1175265956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f954818f740 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f9546ffd640 1 --2- 172.21.15.19:0/1175265956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f95380090b0 tx=0x7f95380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9538041070 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f953803c7a0 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.062+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f954818d930 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.063+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9538037040 con 0x7f9548107b80 2024-09-12T03:09:22.668 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.063+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f954818de70 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.063+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9538044070 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.063+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f953803d3f0 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.064+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9548108000 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.066+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f9538046450 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.127+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f954818e5d0 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.127+0000 7f952ffff640 1 -- 172.21.15.19:0/1175265956 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f9538046630 con 0x7f9548107b80 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 msgr2=0x7f954818f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 --2- 172.21.15.19:0/1175265956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f95380090b0 tx=0x7f95380046c0 comp rx=0 tx=0).stop 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 shutdown_connections 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 --2- 172.21.15.19:0/1175265956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9548107b80 0x7f954818f200 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 >> 172.21.15.19:0/1175265956 conn(0x7f9548101550 msgr2=0x7f9548105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:22.669 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 shutdown_connections 2024-09-12T03:09:22.670 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:22.128+0000 7f954d2e4640 1 -- 172.21.15.19:0/1175265956 wait complete. 2024-09-12T03:09:22.670 INFO:teuthology.orchestra.run.smithi019.stdout:mon is available 2024-09-12T03:09:22.670 INFO:teuthology.orchestra.run.smithi019.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T03:09:23.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:23 smithi019 ceph-mon[30586]: from='client.? 172.21.15.19:0/258751803' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:09:23.770 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:23 smithi019 ceph-mon[30586]: from='client.? 172.21.15.19:0/258751803' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:09:23.779 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0da2750640 1 Processor -- start 2024-09-12T03:09:23.780 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0da2750640 1 -- start start 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0da2750640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0da2750640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c108530 con 0x7f0d9c107b60 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57204/0 (socket says 172.21.15.19:57204) 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0d9bfff640 1 -- 172.21.15.19:0/4101030348 learned_addr learned my addr 172.21.15.19:0/4101030348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.086+0000 7f0d9bfff640 1 -- 172.21.15.19:0/4101030348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c106770 con 0x7f0d9c107b60 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.087+0000 7f0d9bfff640 1 --2- 172.21.15.19:0/4101030348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009820 tx=0x7f0d8c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32799c18d2441fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.087+0000 7f0d9affd640 1 -- 172.21.15.19:0/4101030348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c037070 con 0x7f0d9c107b60 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.087+0000 7f0d9affd640 1 -- 172.21.15.19:0/4101030348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c032440 con 0x7f0d9c107b60 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.087+0000 7f0d9affd640 1 -- 172.21.15.19:0/4101030348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c032740 con 0x7f0d9c107b60 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 -- 172.21.15.19:0/4101030348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 msgr2=0x7f0d9c107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:23.781 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 --2- 172.21.15.19:0/4101030348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009820 tx=0x7f0d8c02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 -- 172.21.15.19:0/4101030348 shutdown_connections 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 --2- 172.21.15.19:0/4101030348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c107f60 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 -- 172.21.15.19:0/4101030348 >> 172.21.15.19:0/4101030348 conn(0x7f0d9c101550 msgr2=0x7f0d9c103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 -- 172.21.15.19:0/4101030348 shutdown_connections 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.088+0000 7f0da2750640 1 -- 172.21.15.19:0/4101030348 wait complete. 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0da2750640 1 Processor -- start 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0da2750640 1 -- start start 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0da2750640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0da2750640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d9c106ca0 con 0x7f0d9c107b60 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0d9bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57210/0 (socket says 172.21.15.19:57210) 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.089+0000 7f0d9bfff640 1 -- 172.21.15.19:0/258751803 learned_addr learned my addr 172.21.15.19:0/258751803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:23.782 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d9bfff640 1 -- 172.21.15.19:0/258751803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9c19b320 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d9bfff640 1 --2- 172.21.15.19:0/258751803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0090b0 tx=0x7f0d8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c041070 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c03c7a0 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c037040 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9c19b5b0 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d9c19e2a0 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0d8c044070 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.090+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0d8c03d3f0 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.091+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d9c1072f0 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.093+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0d8c046450 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.154+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f0d9c02d910 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.157+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f0d8c046630 con 0x7f0d9c107b60 2024-09-12T03:09:23.783 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.157+0000 7f0d997fa640 1 -- 172.21.15.19:0/258751803 <== mon.0 v2:172.21.15.19:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f0d8c0584b0 con 0x7f0d9c107b60 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 msgr2=0x7f0d9c19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 --2- 172.21.15.19:0/258751803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0090b0 tx=0x7f0d8c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 shutdown_connections 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 --2- 172.21.15.19:0/258751803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d9c107b60 0x7f0d9c19ade0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 >> 172.21.15.19:0/258751803 conn(0x7f0d9c101550 msgr2=0x7f0d9c191ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 shutdown_connections 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:23.158+0000 7f0da2750640 1 -- 172.21.15.19:0/258751803 wait complete. 2024-09-12T03:09:23.784 INFO:teuthology.orchestra.run.smithi019.stdout:Generating new minimal ceph.conf... 2024-09-12T03:09:24.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:24 smithi019 ceph-mon[30586]: from='client.? 172.21.15.19:0/1243571315' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:09:25.326 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec73ec640 1 Processor -- start 2024-09-12T03:09:25.326 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec73ec640 1 -- start start 2024-09-12T03:09:25.326 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec73ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:25.326 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec73ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdec0108110 con 0x7fdec0107740 2024-09-12T03:09:25.326 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec5188640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec5188640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57216/0 (socket says 172.21.15.19:57216) 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.166+0000 7fdec5188640 1 -- 172.21.15.19:0/3289748657 learned_addr learned my addr 172.21.15.19:0/3289748657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.167+0000 7fdec5188640 1 -- 172.21.15.19:0/3289748657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01088a0 con 0x7fdec0107740 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.167+0000 7fdec5188640 1 --2- 172.21.15.19:0/3289748657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009820 tx=0x7fdebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482467c67ba1e37a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.167+0000 7fdeb7fff640 1 -- 172.21.15.19:0/3289748657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdebc037070 con 0x7fdec0107740 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdeb7fff640 1 -- 172.21.15.19:0/3289748657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdebc032900 con 0x7fdec0107740 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdeb7fff640 1 -- 172.21.15.19:0/3289748657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdebc030850 con 0x7fdec0107740 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 -- 172.21.15.19:0/3289748657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 msgr2=0x7fdec0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 --2- 172.21.15.19:0/3289748657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009820 tx=0x7fdebc02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 -- 172.21.15.19:0/3289748657 shutdown_connections 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 --2- 172.21.15.19:0/3289748657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec0107b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 -- 172.21.15.19:0/3289748657 >> 172.21.15.19:0/3289748657 conn(0x7fdec0102eb0 msgr2=0x7fdec0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:25.327 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 -- 172.21.15.19:0/3289748657 shutdown_connections 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.168+0000 7fdec73ec640 1 -- 172.21.15.19:0/3289748657 wait complete. 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec73ec640 1 Processor -- start 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec73ec640 1 -- start start 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec73ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec73ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdec010bc50 con 0x7fdec0107740 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec5188640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec5188640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57220/0 (socket says 172.21.15.19:57220) 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.169+0000 7fdec5188640 1 -- 172.21.15.19:0/1243571315 learned_addr learned my addr 172.21.15.19:0/1243571315 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdec5188640 1 -- 172.21.15.19:0/1243571315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec019bbf0 con 0x7fdec0107740 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdec5188640 1 --2- 172.21.15.19:0/1243571315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009940 tx=0x7fdebc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdebc042070 con 0x7fdec0107740 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdebc032be0 con 0x7fdec0107740 2024-09-12T03:09:25.328 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec019be80 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdebc037040 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec019eb70 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdebc045070 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.170+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdebc047a30 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.171+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec0107bc0 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.173+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdebc032d90 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.234+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdec019ed40 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.234+0000 7fdeb67fc640 1 -- 172.21.15.19:0/1243571315 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fdebc03d430 con 0x7fdec0107740 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 msgr2=0x7fdec019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 --2- 172.21.15.19:0/1243571315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009940 tx=0x7fdebc0326e0 comp rx=0 tx=0).stop 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 shutdown_connections 2024-09-12T03:09:25.329 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 --2- 172.21.15.19:0/1243571315 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdec0107740 0x7fdec019b6b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:25.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 >> 172.21.15.19:0/1243571315 conn(0x7fdec0102eb0 msgr2=0x7fdec0103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:25.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 shutdown_connections 2024-09-12T03:09:25.330 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:24.235+0000 7fdec73ec640 1 -- 172.21.15.19:0/1243571315 wait complete. 2024-09-12T03:09:25.330 INFO:teuthology.orchestra.run.smithi019.stdout:Restarting the monitor... 2024-09-12T03:09:25.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:25 smithi019 systemd[1]: Stopping Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:09:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a[30582]: 2024-09-12T03:09:25.605+0000 7f94d5a04640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:09:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a[30582]: 2024-09-12T03:09:25.605+0000 7f94d5a04640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T03:09:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:25 smithi019 podman[30828]: 2024-09-12 03:09:25.852541062 +0000 UTC m=+0.336542457 container died cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:09:26.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 podman[30828]: 2024-09-12 03:09:26.189012409 +0000 UTC m=+0.673013796 container cleanup cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:09:26.364 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 bash[30828]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:09:26.620 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 podman[30840]: 2024-09-12 03:09:26.365076113 +0000 UTC m=+0.508686481 container remove cd10b5199186f8a55150db29cbb5b1a9dcd5bfb9c2e6609ed3813e59bea98ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-12T03:09:26.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:09:26.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 systemd[1]: Stopped Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:09:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 systemd[1]: Starting Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:09:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:26 smithi019 podman[30935]: 2024-09-12 03:09:26.739233333 +0000 UTC m=+0.016688151 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T03:09:28.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:27 smithi019 podman[30935]: 2024-09-12 03:09:27.670645182 +0000 UTC m=+0.948099985 container create 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:09:29.824 INFO:teuthology.orchestra.run.smithi019.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 podman[30935]: 2024-09-12 03:09:29.677301759 +0000 UTC m=+2.954756561 container init 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 podman[30935]: 2024-09-12 03:09:29.681522465 +0000 UTC m=+2.958977268 container start 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: pidfile_write: ignore empty --pid-file 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: load: jerasure load: lrc 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Git sha 0 2024-09-12T03:09:29.952 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: DB SUMMARY 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: DB Session ID: TCS4EN4AO8IUMDCFQ4X4 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105374 ; 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:09:29.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.env: 0x562e6dd53940 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.info_log: 0x562e7004f6e0 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:09:29.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.statistics: (nil) 2024-09-12T03:09:29.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.use_fsync: 0 2024-09-12T03:09:29.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:09:29.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.db_log_dir: 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.wal_dir: 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:09:29.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.write_buffer_manager: 0x562e7005edc0 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:09:29.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.unordered_write: 0 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.row_cache: None 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.wal_filter: None 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:09:29.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.wal_compression: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:09:29.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:09:29.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_open_files: -1 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Compression algorithms supported: 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kZSTD supported: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:09:29.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.merge_operator: 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_filter: None 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:09:29.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562e7004f320) 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_type: 0 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_index_type: 0 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_shortening: 1 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: checksum: 4 2024-09-12T03:09:29.963 INFO:journalctl@ceph.mon.a.smithi019.stdout: no_block_cache: 0 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache: 0x562e700431f0 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_options: 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: capacity : 536870912 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_shard_bits : 4 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: strict_capacity_limit : 0 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_cache_compressed: (nil) 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: persistent_cache: (nil) 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size: 4096 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_size_deviation: 10 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_restart_interval: 16 2024-09-12T03:09:29.964 INFO:journalctl@ceph.mon.a.smithi019.stdout: index_block_restart_interval: 1 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: metadata_block_size: 4096 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: partition_filters: 0 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: use_delta_encoding: 1 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: filter_policy: bloomfilter 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: whole_key_filtering: 1 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: verify_compression: 0 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: format_version: 5 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_index_compression: 1 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: block_align: 0 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: max_auto_readahead_size: 262144 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: prepopulate_block_cache: 0 2024-09-12T03:09:29.965 INFO:journalctl@ceph.mon.a.smithi019.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression: NoCompression 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.num_levels: 7 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:09:29.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:09:29.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:09:29.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:09:29.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:09:29.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.ttl: 2592000 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:09:29.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b7cd7ca0-3bcc-44fe-9849-98df06f577c0 2024-09-12T03:09:29.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110569712858, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110569714135, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100447, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98331, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14312, "raw_average_key_size": 49, "raw_value_size": 90577, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726110569, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b7cd7ca0-3bcc-44fe-9849-98df06f577c0", "db_session_id": "TCS4EN4AO8IUMDCFQ4X4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110569714256, "job": 1, "event": "recovery_finished"} 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562e70070e00 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: DB pointer 0x562e7016c000 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** DB Stats ** 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:09:29.973 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:09:29.974 INFO:journalctl@ceph.mon.a.smithi019.stdout: L0 2/0 99.95 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Sum 2/0 99.95 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 118.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** Compaction Stats [default] ** 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 118.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:09:29.975 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: Cumulative compaction: 0.00 GB write, 23.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: Interval compaction: 0.00 GB write, 23.68 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache BinnedLRUCache@0x562e700431f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: starting mon.a rank 0 at public addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] at bind addrs [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:29.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???) e1 preinit fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).mds e1 new map 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).mds e1 print_map 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: e1 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: btime 2024-09-12T03:09:21:563894+0000 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: legacy client fscid: -1 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout: No filesystems configured 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:09:29.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: monmap epoch 1 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: last_changed 2024-09-12T03:09:09.637034+0000 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: min_mon_release 19 (squid) 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: election_strategy: 1 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: fsmap 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 ceph-mon[30953]: mgrmap e1: no daemons active 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 bash[30935]: 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 2024-09-12T03:09:29.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:29 smithi019 systemd[1]: Started Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.267+0000 7faf658fe640 1 Processor -- start 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.267+0000 7faf658fe640 1 -- start start 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.267+0000 7faf658fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.267+0000 7faf658fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf6007e010 con 0x7faf6007c2e0 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.268+0000 7faf5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.268+0000 7faf5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57236/0 (socket says 172.21.15.19:57236) 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.268+0000 7faf5effd640 1 -- 172.21.15.19:0/784358343 learned_addr learned my addr 172.21.15.19:0/784358343 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.268+0000 7faf5effd640 1 -- 172.21.15.19:0/784358343 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6007cc20 con 0x7faf6007c2e0 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.270+0000 7faf5effd640 1 --2- 172.21.15.19:0/784358343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e80 tx=0x7faf5002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f81f7fc7d854a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.270+0000 7faf5dffb640 1 -- 172.21.15.19:0/784358343 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf50036070 con 0x7faf6007c2e0 2024-09-12T03:09:30.882 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.270+0000 7faf5dffb640 1 -- 172.21.15.19:0/784358343 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf500318c0 con 0x7faf6007c2e0 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.270+0000 7faf5dffb640 1 -- 172.21.15.19:0/784358343 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf50007ab0 con 0x7faf6007c2e0 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 -- 172.21.15.19:0/784358343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 msgr2=0x7faf6007c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 --2- 172.21.15.19:0/784358343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7faf50009e80 tx=0x7faf5002a7a0 comp rx=0 tx=0).stop 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 -- 172.21.15.19:0/784358343 shutdown_connections 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 --2- 172.21.15.19:0/784358343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6007c6e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 -- 172.21.15.19:0/784358343 >> 172.21.15.19:0/784358343 conn(0x7faf60102dc0 msgr2=0x7faf601051e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 -- 172.21.15.19:0/784358343 shutdown_connections 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.271+0000 7faf658fe640 1 -- 172.21.15.19:0/784358343 wait complete. 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.272+0000 7faf658fe640 1 Processor -- start 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.272+0000 7faf658fe640 1 -- start start 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.272+0000 7faf658fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.272+0000 7faf658fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf6010dd60 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf5effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57246/0 (socket says 172.21.15.19:57246) 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf5effd640 1 -- 172.21.15.19:0/4081084102 learned_addr learned my addr 172.21.15.19:0/4081084102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf5effd640 1 -- 172.21.15.19:0/4081084102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6019de40 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf5effd640 1 --2- 172.21.15.19:0/4081084102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf50031ca0 tx=0x7faf50031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf50036050 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf50042070 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6019e0d0 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf5003d820 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.273+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf601a0dc0 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.274+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf5003dad0 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.274+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faf50007e00 con 0x7faf6007c2e0 2024-09-12T03:09:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.274+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf6007c760 con 0x7faf6007c2e0 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.277+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7faf500072f0 con 0x7faf6007c2e0 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.337+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7faf601a0f90 con 0x7faf6007c2e0 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf648fc640 1 -- 172.21.15.19:0/4081084102 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7faf50047460 con 0x7faf6007c2e0 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 msgr2=0x7faf6019d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 --2- 172.21.15.19:0/4081084102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7faf50031ca0 tx=0x7faf50031cd0 comp rx=0 tx=0).stop 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 shutdown_connections 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 --2- 172.21.15.19:0/4081084102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf6007c2e0 0x7faf6019d900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 >> 172.21.15.19:0/4081084102 conn(0x7faf60102dc0 msgr2=0x7faf60103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.339+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 shutdown_connections 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:30.340+0000 7faf658fe640 1 -- 172.21.15.19:0/4081084102 wait complete. 2024-09-12T03:09:30.885 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T03:09:30.907 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:09:30.907 INFO:teuthology.orchestra.run.smithi019.stdout:Creating mgr... 2024-09-12T03:09:30.908 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T03:09:30.908 INFO:teuthology.orchestra.run.smithi019.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T03:09:31.172 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:09:31.172 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Failed to reset failed state of unit ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T03:09:31.332 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968.target.wants/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@.service. 2024-09-12T03:09:31.649 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:31 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4081084102' entity='client.admin' 2024-09-12T03:09:31.828 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:09:31.828 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-12T03:09:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:09:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:09:31.836 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-12T03:09:31.837 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T03:09:31.846 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:09:31.846 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-12T03:09:31.855 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:09:31.856 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:09:31.856 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-12T03:09:31.856 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T03:09:31.856 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr to start... 2024-09-12T03:09:31.856 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr... 2024-09-12T03:09:31.914 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:31.877+0000 7f0aa4725040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:09:32.195 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:31.941+0000 7f0aa4725040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:09:32.195 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.098+0000 7f0aa4725040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:09:32.195 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.168+0000 7f0aa4725040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:09:32.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1258406339' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:09:32.479 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.327+0000 7f0aa4725040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:09:32.479 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.477+0000 7f0aa4725040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:09:32.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:09:32.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:09:32.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:32.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:09:32.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:09:32.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:09:32.901 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:09:21:563894+0000", 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:09:32.902 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:09:21.564686+0000", 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:32.903 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.272+0000 7f46dff3e640 1 Processor -- start 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.272+0000 7f46dff3e640 1 -- start start 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.272+0000 7f46dff3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.272+0000 7f46dff3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46d807e170 con 0x7f46d807dd70 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.273+0000 7f46ddcda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.273+0000 7f46ddcda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40728/0 (socket says 172.21.15.19:40728) 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.273+0000 7f46ddcda640 1 -- 172.21.15.19:0/351128324 learned_addr learned my addr 172.21.15.19:0/351128324 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.273+0000 7f46ddcda640 1 -- 172.21.15.19:0/351128324 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d807c820 con 0x7f46d807dd70 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46ddcda640 1 --2- 172.21.15.19:0/351128324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009820 tx=0x7f46d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b8683358a0e3c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dccd8640 1 -- 172.21.15.19:0/351128324 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46d4037070 con 0x7f46d807dd70 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dccd8640 1 -- 172.21.15.19:0/351128324 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46d4032900 con 0x7f46d807dd70 2024-09-12T03:09:32.904 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dccd8640 1 -- 172.21.15.19:0/351128324 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46d4030850 con 0x7f46d807dd70 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dff3e640 1 -- 172.21.15.19:0/351128324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 msgr2=0x7f46d807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dff3e640 1 --2- 172.21.15.19:0/351128324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f46d4009820 tx=0x7f46d402a270 comp rx=0 tx=0).stop 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dff3e640 1 -- 172.21.15.19:0/351128324 shutdown_connections 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.274+0000 7f46dff3e640 1 --2- 172.21.15.19:0/351128324 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d807c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 -- 172.21.15.19:0/351128324 >> 172.21.15.19:0/351128324 conn(0x7f46d8102eb0 msgr2=0x7f46d81052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 -- 172.21.15.19:0/351128324 shutdown_connections 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 -- 172.21.15.19:0/351128324 wait complete. 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 Processor -- start 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 -- start start 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.275+0000 7f46dff3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810dba0 con 0x7f46d807dd70 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ddcda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:32.905 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ddcda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40734/0 (socket says 172.21.15.19:40734) 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ddcda640 1 -- 172.21.15.19:0/1258406339 learned_addr learned my addr 172.21.15.19:0/1258406339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ddcda640 1 -- 172.21.15.19:0/1258406339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d819dd00 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ddcda640 1 --2- 172.21.15.19:0/1258406339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46d40097f0 tx=0x7f46d4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46d4037040 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f46d4040070 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d819df90 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46d403a390 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.276+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d81a0c80 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.277+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f46d403a5e0 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.277+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f46d4045950 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.277+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d807c360 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.279+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f46d4045460 con 0x7f46d807dd70 2024-09-12T03:09:32.906 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.342+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f46d81a0e50 con 0x7f46d807dd70 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.342+0000 7f46ceffd640 1 -- 172.21.15.19:0/1258406339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f46d403b430 con 0x7f46d807dd70 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.343+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 msgr2=0x7f46d819d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.343+0000 7f46dff3e640 1 --2- 172.21.15.19:0/1258406339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f46d40097f0 tx=0x7f46d4030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.344+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 shutdown_connections 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.344+0000 7f46dff3e640 1 --2- 172.21.15.19:0/1258406339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f46d807dd70 0x7f46d819d7c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.344+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 >> 172.21.15.19:0/1258406339 conn(0x7f46d8102eb0 msgr2=0x7f46d8103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.344+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 shutdown_connections 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:32.344+0000 7f46dff3e640 1 -- 172.21.15.19:0/1258406339 wait complete. 2024-09-12T03:09:32.907 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (1/15)... 2024-09-12T03:09:33.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.812+0000 7f0aa4725040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:09:33.070 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:32.875+0000 7f0aa4725040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:09:33.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:33.069+0000 7f0aa4725040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:09:34.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:33.995+0000 7f0aa4725040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:09:34.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:34.065+0000 7f0aa4725040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:09:35.095 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:34.822+0000 7f0aa4725040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:09:35.096 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:34.946+0000 7f0aa4725040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:09:35.096 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.095+0000 7f0aa4725040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:09:35.393 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.162+0000 7f0aa4725040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:09:35.393 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.319+0000 7f0aa4725040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:09:35.393 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.392+0000 7f0aa4725040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:09:35.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/219481650' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:09:35.664 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.461+0000 7f0aa4725040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:09:35.665 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.634+0000 7f0aa4725040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:09:35.983 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.702+0000 7f0aa4725040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:09:35.984 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.772+0000 7f0aa4725040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:09:35.984 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.858+0000 7f0aa4725040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:09:35.984 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.920+0000 7f0aa4725040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:09:35.984 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:35.983+0000 7f0aa4725040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:09:36.089 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:09:36.090 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:09:36.091 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:09:21:563894+0000", 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:09:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:09:21.564686+0000", 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:36.093 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.325+0000 7fbd5d00f640 1 Processor -- start 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.325+0000 7fbd5d00f640 1 -- start start 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.325+0000 7fbd5d00f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.325+0000 7fbd5d00f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5807be30 con 0x7fbd58108110 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.326+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.326+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40736/0 (socket says 172.21.15.19:40736) 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.326+0000 7fbd56d9d640 1 -- 172.21.15.19:0/151434353 learned_addr learned my addr 172.21.15.19:0/151434353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.326+0000 7fbd56d9d640 1 -- 172.21.15.19:0/151434353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5810aa40 con 0x7fbd58108110 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd56d9d640 1 --2- 172.21.15.19:0/151434353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009820 tx=0x7fbd4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0efbed8b3f8a65a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd55d9b640 1 -- 172.21.15.19:0/151434353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd48037070 con 0x7fbd58108110 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd55d9b640 1 -- 172.21.15.19:0/151434353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd48032900 con 0x7fbd58108110 2024-09-12T03:09:36.094 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd55d9b640 1 -- 172.21.15.19:0/151434353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd48030850 con 0x7fbd58108110 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd5d00f640 1 -- 172.21.15.19:0/151434353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 msgr2=0x7fbd5810a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd5d00f640 1 --2- 172.21.15.19:0/151434353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009820 tx=0x7fbd4802a270 comp rx=0 tx=0).stop 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.327+0000 7fbd5d00f640 1 -- 172.21.15.19:0/151434353 shutdown_connections 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.328+0000 7fbd5d00f640 1 --2- 172.21.15.19:0/151434353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5810a500 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.328+0000 7fbd5d00f640 1 -- 172.21.15.19:0/151434353 >> 172.21.15.19:0/151434353 conn(0x7fbd58101db0 msgr2=0x7fbd581041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.328+0000 7fbd5d00f640 1 -- 172.21.15.19:0/151434353 shutdown_connections 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.328+0000 7fbd5d00f640 1 -- 172.21.15.19:0/151434353 wait complete. 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd5d00f640 1 Processor -- start 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd5d00f640 1 -- start start 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd5d00f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd5d00f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5810ba20 con 0x7fbd58108110 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:36.095 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd56d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40740/0 (socket says 172.21.15.19:40740) 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.329+0000 7fbd56d9d640 1 -- 172.21.15.19:0/219481650 learned_addr learned my addr 172.21.15.19:0/219481650 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd56d9d640 1 -- 172.21.15.19:0/219481650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5819bb60 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd56d9d640 1 --2- 172.21.15.19:0/219481650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbd480097f0 tx=0x7fbd48030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd48037040 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbd48040070 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd4803a390 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5819bdf0 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5819eae0 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd4803a5e0 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.330+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd48045950 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.331+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd58108570 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.333+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbd48045460 con 0x7fbd58108110 2024-09-12T03:09:36.096 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.399+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fbd5819ecb0 con 0x7fbd58108110 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.399+0000 7fbd3ffff640 1 -- 172.21.15.19:0/219481650 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fbd4803b430 con 0x7fbd58108110 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 msgr2=0x7fbd5819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 --2- 172.21.15.19:0/219481650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbd480097f0 tx=0x7fbd48030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 shutdown_connections 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 --2- 172.21.15.19:0/219481650 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbd58108110 0x7fbd5819b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 >> 172.21.15.19:0/219481650 conn(0x7fbd58101db0 msgr2=0x7fbd581028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 shutdown_connections 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:35.401+0000 7fbd5d00f640 1 -- 172.21.15.19:0/219481650 wait complete. 2024-09-12T03:09:36.097 INFO:teuthology.orchestra.run.smithi019.stdout:mgr not available, waiting (2/15)... 2024-09-12T03:09:37.164 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:36.813+0000 7f0aa4725040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: Activating manager daemon x 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: mgrmap e2: x(active, starting, since 0.00303093s) 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: Manager daemon x is now available 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:09:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' 2024-09-12T03:09:37.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:09:37.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' 2024-09-12T03:09:37.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:36 smithi019 ceph-mon[30953]: from='mgr.14100 172.21.15.19:0/2358486730' entity='mgr.x' 2024-09-12T03:09:39.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:38 smithi019 ceph-mon[30953]: mgrmap e3: x(active, since 1.00723s) 2024-09-12T03:09:39.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3227840862' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T03:09:39.270 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 0 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T03:09:39.272 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T03:09:39.281 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T03:09:21:563894+0000", 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.282 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ], 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T03:09:21.564686+0000", 2024-09-12T03:09:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout }, 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.482+0000 7f4d71eb3640 1 Processor -- start 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.482+0000 7f4d71eb3640 1 -- start start 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.482+0000 7f4d71eb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.482+0000 7f4d71eb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c108110 con 0x7f4d6c107740 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.482+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.483+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40796/0 (socket says 172.21.15.19:40796) 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.483+0000 7f4d6b7fe640 1 -- 172.21.15.19:0/1110399201 learned_addr learned my addr 172.21.15.19:0/1110399201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.483+0000 7f4d6b7fe640 1 -- 172.21.15.19:0/1110399201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c1088a0 con 0x7f4d6c107740 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.484+0000 7f4d6b7fe640 1 --2- 172.21.15.19:0/1110399201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009820 tx=0x7f4d5c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc5d9db2fafcf4f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:39.284 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.484+0000 7f4d6a7fc640 1 -- 172.21.15.19:0/1110399201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c037070 con 0x7f4d6c107740 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.484+0000 7f4d6a7fc640 1 -- 172.21.15.19:0/1110399201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d5c032900 con 0x7f4d6c107740 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.484+0000 7f4d6a7fc640 1 -- 172.21.15.19:0/1110399201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c030850 con 0x7f4d6c107740 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 -- 172.21.15.19:0/1110399201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 msgr2=0x7f4d6c107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/1110399201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009820 tx=0x7f4d5c02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 -- 172.21.15.19:0/1110399201 shutdown_connections 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/1110399201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c107b40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 -- 172.21.15.19:0/1110399201 >> 172.21.15.19:0/1110399201 conn(0x7f4d6c102eb0 msgr2=0x7f4d6c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 -- 172.21.15.19:0/1110399201 shutdown_connections 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.485+0000 7f4d71eb3640 1 -- 172.21.15.19:0/1110399201 wait complete. 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d71eb3640 1 Processor -- start 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d71eb3640 1 -- start start 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d71eb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:39.285 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d71eb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6c10bc50 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d6b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40800/0 (socket says 172.21.15.19:40800) 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.486+0000 7f4d6b7fe640 1 -- 172.21.15.19:0/3227840862 learned_addr learned my addr 172.21.15.19:0/3227840862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d6b7fe640 1 -- 172.21.15.19:0/3227840862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d6c07e0b0 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d6b7fe640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0097f0 tx=0x7f4d5c030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c042070 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4d5c032ca0 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d6c07c2a0 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d5c037040 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.487+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d6c07c7e0 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.488+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f4d5c045070 con 0x7f4d6c107740 2024-09-12T03:09:39.286 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.488+0000 7f4d68ff9640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 0x7f4d5403b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.488+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d6c107bc0 con 0x7f4d6c107740 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.488+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d5c06cd00 con 0x7f4d6c107740 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.489+0000 7f4d6affd640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 0x7f4d5403b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.489+0000 7f4d6affd640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 0x7f4d5403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098c0 tx=0x7f4d60006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.494+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4d5c03ccd0 con 0x7f4d6c107740 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.738+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4d6c07cc30 con 0x7f4d6c107740 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.739+0000 7f4d68ff9640 1 -- 172.21.15.19:0/3227840862 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f4d5c03c7b0 con 0x7f4d6c107740 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 msgr2=0x7f4d5403b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 0x7f4d5403b2d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4d600098c0 tx=0x7f4d60006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 msgr2=0x7f4d6c07db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0097f0 tx=0x7f4d5c030ec0 comp rx=0 tx=0).stop 2024-09-12T03:09:39.287 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 shutdown_connections 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f4d54038e10 0x7f4d5403b2d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 --2- 172.21.15.19:0/3227840862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d6c107740 0x7f4d6c07db70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 >> 172.21.15.19:0/3227840862 conn(0x7f4d6c102eb0 msgr2=0x7f4d6c103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 shutdown_connections 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:38.741+0000 7f4d71eb3640 1 -- 172.21.15.19:0/3227840862 wait complete. 2024-09-12T03:09:39.288 INFO:teuthology.orchestra.run.smithi019.stdout:mgr is available 2024-09-12T03:09:40.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:39 smithi019 ceph-mon[30953]: mgrmap e4: x(active, since 2s) 2024-09-12T03:09:40.416 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout fsid = 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.19:3300,v1:172.21.15.19:6789] 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T03:09:40.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.655+0000 7f159ccef640 1 Processor -- start 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.655+0000 7f159ccef640 1 -- start start 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159ccef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:40.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159ccef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f159807beb0 con 0x7f1598108090 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40810/0 (socket says 172.21.15.19:40810) 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159659c640 1 -- 172.21.15.19:0/4165142855 learned_addr learned my addr 172.21.15.19:0/4165142855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.656+0000 7f159659c640 1 -- 172.21.15.19:0/4165142855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f159810a9c0 con 0x7f1598108090 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.657+0000 7f159659c640 1 --2- 172.21.15.19:0/4165142855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1588009820 tx=0x7f158802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c5f8c19b873e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.657+0000 7f159559a640 1 -- 172.21.15.19:0/4165142855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1588037070 con 0x7f1598108090 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.657+0000 7f159559a640 1 -- 172.21.15.19:0/4165142855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1588032900 con 0x7f1598108090 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.657+0000 7f159559a640 1 -- 172.21.15.19:0/4165142855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1588030850 con 0x7f1598108090 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 -- 172.21.15.19:0/4165142855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 msgr2=0x7f159810a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 --2- 172.21.15.19:0/4165142855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1588009820 tx=0x7f158802a270 comp rx=0 tx=0).stop 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 -- 172.21.15.19:0/4165142855 shutdown_connections 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 --2- 172.21.15.19:0/4165142855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f159810a480 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:40.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 -- 172.21.15.19:0/4165142855 >> 172.21.15.19:0/4165142855 conn(0x7f1598101db0 msgr2=0x7f15981041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 -- 172.21.15.19:0/4165142855 shutdown_connections 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.658+0000 7f159ccef640 1 -- 172.21.15.19:0/4165142855 wait complete. 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159ccef640 1 Processor -- start 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159ccef640 1 -- start start 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159ccef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159ccef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f159810ba60 con 0x7f1598108090 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40816/0 (socket says 172.21.15.19:40816) 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159659c640 1 -- 172.21.15.19:0/3450009215 learned_addr learned my addr 172.21.15.19:0/3450009215 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.659+0000 7f159659c640 1 -- 172.21.15.19:0/3450009215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1598107e50 con 0x7f1598108090 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f159659c640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15880097f0 tx=0x7f1588030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:40.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1588037040 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1588040070 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f158803a390 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1598106040 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.660+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1598106520 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.661+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15981084f0 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.661+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f158803a5e0 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.661+0000 7f15877fe640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 0x7f158003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.661+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f158806c210 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.661+0000 7f1595d9b640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 0x7f158003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.662+0000 7f1595d9b640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 0x7f158003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f158c0098c0 tx=0x7f158c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.670+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f158803a980 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.846+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f15981067a0 con 0x7f1598108090 2024-09-12T03:09:40.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.847+0000 7f15877fe640 1 -- 172.21.15.19:0/3450009215 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f158806bf70 con 0x7f1598108090 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 msgr2=0x7f158003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 0x7f158003b430 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f158c0098c0 tx=0x7f158c006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 msgr2=0x7f1598107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15880097f0 tx=0x7f1588030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 shutdown_connections 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f1580038f70 0x7f158003b430 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 --2- 172.21.15.19:0/3450009215 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1598108090 0x7f1598107910 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 >> 172.21.15.19:0/3450009215 conn(0x7f1598101db0 msgr2=0x7f15981028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 shutdown_connections 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:39.849+0000 7f159ccef640 1 -- 172.21.15.19:0/3450009215 wait complete. 2024-09-12T03:09:40.428 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling cephadm module... 2024-09-12T03:09:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:40 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3450009215' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T03:09:42.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:41 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4137445248' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T03:09:42.165 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ignoring --setuser ceph since I am not root 2024-09-12T03:09:42.165 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ignoring --setgroup ceph since I am not root 2024-09-12T03:09:42.165 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:41.976+0000 7fd3797bc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:09:42.165 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.036+0000 7fd3797bc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:09:42.429 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.191+0000 7fd3797bc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:09:42.429 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.260+0000 7fd3797bc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:09:42.429 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.414+0000 7fd3797bc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:09:42.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.798+0000 7f38af744640 1 Processor -- start 2024-09-12T03:09:42.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.798+0000 7f38af744640 1 -- start start 2024-09-12T03:09:42.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38af744640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:42.436 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38af744640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38a8108110 con 0x7f38a8107740 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38ad4e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38ad4e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56838/0 (socket says 172.21.15.19:56838) 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38ad4e0640 1 -- 172.21.15.19:0/1690722381 learned_addr learned my addr 172.21.15.19:0/1690722381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.799+0000 7f38ad4e0640 1 -- 172.21.15.19:0/1690722381 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a81088a0 con 0x7f38a8107740 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.800+0000 7f38ad4e0640 1 --2- 172.21.15.19:0/1690722381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009820 tx=0x7f38a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f77ae8c36ea72300 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.800+0000 7f389ffff640 1 -- 172.21.15.19:0/1690722381 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a4037070 con 0x7f38a8107740 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.800+0000 7f389ffff640 1 -- 172.21.15.19:0/1690722381 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38a4032900 con 0x7f38a8107740 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.800+0000 7f389ffff640 1 -- 172.21.15.19:0/1690722381 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a4030850 con 0x7f38a8107740 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- 172.21.15.19:0/1690722381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 msgr2=0x7f38a8107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 --2- 172.21.15.19:0/1690722381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38a4009820 tx=0x7f38a402a270 comp rx=0 tx=0).stop 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- 172.21.15.19:0/1690722381 shutdown_connections 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 --2- 172.21.15.19:0/1690722381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a8107b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:42.437 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- 172.21.15.19:0/1690722381 >> 172.21.15.19:0/1690722381 conn(0x7f38a8102eb0 msgr2=0x7f38a8105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- 172.21.15.19:0/1690722381 shutdown_connections 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- 172.21.15.19:0/1690722381 wait complete. 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 Processor -- start 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.801+0000 7f38af744640 1 -- start start 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38af744640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38af744640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38a810bc50 con 0x7f38a8107740 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38ad4e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38ad4e0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56846/0 (socket says 172.21.15.19:56846) 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38ad4e0640 1 -- 172.21.15.19:0/4137445248 learned_addr learned my addr 172.21.15.19:0/4137445248 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38ad4e0640 1 -- 172.21.15.19:0/4137445248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a819bbc0 con 0x7f38a8107740 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38ad4e0640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f38a40097f0 tx=0x7f38a4030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:42.438 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a4042070 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38a4032ca0 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a819be50 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38a4037040 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.802+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a819eb40 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.803+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f38a4045070 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.803+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a8107bc0 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.803+0000 7f389e7fc640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 0x7f389403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.804+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38a406bfe0 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.804+0000 7f38accdf640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 0x7f389403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.804+0000 7f38accdf640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 0x7f389403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38980098c0 tx=0x7f3898006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:40.808+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38a403be10 con 0x7f38a8107740 2024-09-12T03:09:42.439 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.031+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f38a819f1c0 con 0x7f38a8107740 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.824+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f38a403ccd0 con 0x7f38a8107740 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.826+0000 7f389e7fc640 1 -- 172.21.15.19:0/4137445248 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f38a403c7b0 con 0x7f38a8107740 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 msgr2=0x7f389403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 0x7f389403b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f38980098c0 tx=0x7f3898006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 msgr2=0x7f38a819b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f38a40097f0 tx=0x7f38a4030ec0 comp rx=0 tx=0).stop 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 shutdown_connections 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f3894038f20 0x7f389403b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 --2- 172.21.15.19:0/4137445248 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38a8107740 0x7f38a819b680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 >> 172.21.15.19:0/4137445248 conn(0x7f38a8102eb0 msgr2=0x7f38a8103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 shutdown_connections 2024-09-12T03:09:42.440 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:41.828+0000 7f38af744640 1 -- 172.21.15.19:0/4137445248 wait complete. 2024-09-12T03:09:42.732 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.568+0000 7fd3797bc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:09:43.017 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4137445248' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T03:09:43.017 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-mon[30953]: mgrmap e5: x(active, since 5s) 2024-09-12T03:09:43.017 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:42.966+0000 7fd3797bc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:09:43.306 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:43.072+0000 7fd3797bc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:09:43.664 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:43.305+0000 7fd3797bc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:09:43.800 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:09:43.800 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8107ac7640 1 Processor -- start 2024-09-12T03:09:43.801 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8107ac7640 1 -- start start 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8107ac7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8107ac7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f810007be30 con 0x7f8100108110 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8105863640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8105863640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56882/0 (socket says 172.21.15.19:56882) 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.810+0000 7f8105863640 1 -- 172.21.15.19:0/2295716514 learned_addr learned my addr 172.21.15.19:0/2295716514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.811+0000 7f8105863640 1 -- 172.21.15.19:0/2295716514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810010aa40 con 0x7f8100108110 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.811+0000 7f8105863640 1 --2- 172.21.15.19:0/2295716514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009820 tx=0x7f80fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8d439f6072e7c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.811+0000 7f8104861640 1 -- 172.21.15.19:0/2295716514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc037070 con 0x7f8100108110 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8104861640 1 -- 172.21.15.19:0/2295716514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80fc032900 con 0x7f8100108110 2024-09-12T03:09:43.802 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8104861640 1 -- 172.21.15.19:0/2295716514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc030850 con 0x7f8100108110 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 -- 172.21.15.19:0/2295716514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 msgr2=0x7f810010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 --2- 172.21.15.19:0/2295716514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009820 tx=0x7f80fc02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 -- 172.21.15.19:0/2295716514 shutdown_connections 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 --2- 172.21.15.19:0/2295716514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810010a500 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 -- 172.21.15.19:0/2295716514 >> 172.21.15.19:0/2295716514 conn(0x7f8100101d30 msgr2=0x7f8100104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 -- 172.21.15.19:0/2295716514 shutdown_connections 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.812+0000 7f8107ac7640 1 -- 172.21.15.19:0/2295716514 wait complete. 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8107ac7640 1 Processor -- start 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8107ac7640 1 -- start start 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8107ac7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8107ac7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f810010ba20 con 0x7f8100108110 2024-09-12T03:09:43.803 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8105863640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8105863640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56892/0 (socket says 172.21.15.19:56892) 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8105863640 1 -- 172.21.15.19:0/3593546066 learned_addr learned my addr 172.21.15.19:0/3593546066 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.813+0000 7f8105863640 1 -- 172.21.15.19:0/3593546066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810019dd70 con 0x7f8100108110 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f8105863640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097f0 tx=0x7f80fc030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc037040 con 0x7f8100108110 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f80fc040070 con 0x7f8100108110 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80fc03a390 con 0x7f8100108110 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810019e000 con 0x7f8100108110 2024-09-12T03:09:43.804 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.814+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81001a0cf0 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f80fc03b430 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8100108570 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f80f6ffd640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 0x7f80ec03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f80fc06c120 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f8105062640 1 -- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 msgr2=0x7f80ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.815+0000 7f8105062640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 0x7f80ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:42.820+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f80fc06be80 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.016+0000 7f8105062640 1 -- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 msgr2=0x7f80ec03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.016+0000 7f8105062640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 0x7f80ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.224+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f81001a12c0 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.225+0000 7f80f6ffd640 1 -- 172.21.15.19:0/3593546066 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f80fc032900 con 0x7f8100108110 2024-09-12T03:09:43.805 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 msgr2=0x7f80ec03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 0x7f80ec03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 msgr2=0x7f810019d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0097f0 tx=0x7f80fc030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 shutdown_connections 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f80ec038fc0 0x7f80ec03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 --2- 172.21.15.19:0/3593546066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8100108110 0x7f810019d830 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 >> 172.21.15.19:0/3593546066 conn(0x7f8100101d30 msgr2=0x7f8100102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 shutdown_connections 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:43.227+0000 7f8107ac7640 1 -- 172.21.15.19:0/3593546066 wait complete. 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-09-12T03:09:43.806 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 5... 2024-09-12T03:09:44.036 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:43 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3593546066' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:09:44.403 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:44.245+0000 7fd3797bc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:09:44.403 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:44.316+0000 7fd3797bc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:09:45.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.068+0000 7fd3797bc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:09:45.340 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.189+0000 7fd3797bc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:09:45.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.339+0000 7fd3797bc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:09:45.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.403+0000 7fd3797bc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:09:45.605 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.559+0000 7fd3797bc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:09:45.860 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.630+0000 7fd3797bc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:09:45.860 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.693+0000 7fd3797bc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:09:45.860 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.859+0000 7fd3797bc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:09:46.146 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.927+0000 7fd3797bc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:09:46.147 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:45.996+0000 7fd3797bc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:09:46.147 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:46.083+0000 7fd3797bc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:09:46.147 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:46.145+0000 7fd3797bc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:09:46.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:46.207+0000 7fd3797bc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:09:47.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:47.036+0000 7fd3797bc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:09:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: Active manager daemon x restarted 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: Activating manager daemon x 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: mgrmap e6: x(active, starting, since 0.00328279s) 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: Manager daemon x is now available 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:47.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:09:47.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:47 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:09:48.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:48 smithi019 ceph-mon[30953]: Found migration_current of "None". Setting to last migration. 2024-09-12T03:09:48.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:48 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:48.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:48 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:48.369 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:48 smithi019 ceph-mon[30953]: mgrmap e7: x(active, since 1.0057s) 2024-09-12T03:09:48.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:09:48.620 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.194+0000 7f7f51e21640 1 Processor -- start 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.194+0000 7f7f51e21640 1 -- start start 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f51e21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f51e21640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c07be20 con 0x7f7f4c1080c0 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56894/0 (socket says 172.21.15.19:56894) 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f4b7fe640 1 -- 172.21.15.19:0/1566781632 learned_addr learned my addr 172.21.15.19:0/1566781632 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.195+0000 7f7f4b7fe640 1 -- 172.21.15.19:0/1566781632 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c10a9f0 con 0x7f7f4c1080c0 2024-09-12T03:09:48.621 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.196+0000 7f7f4b7fe640 1 --2- 172.21.15.19:0/1566781632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009820 tx=0x7f7f3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4170c8aaa14a8a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.196+0000 7f7f4affd640 1 -- 172.21.15.19:0/1566781632 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c037070 con 0x7f7f4c1080c0 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.196+0000 7f7f4affd640 1 -- 172.21.15.19:0/1566781632 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c032900 con 0x7f7f4c1080c0 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.196+0000 7f7f4affd640 1 -- 172.21.15.19:0/1566781632 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c030850 con 0x7f7f4c1080c0 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 -- 172.21.15.19:0/1566781632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 msgr2=0x7f7f4c10a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 --2- 172.21.15.19:0/1566781632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009820 tx=0x7f7f3c02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 -- 172.21.15.19:0/1566781632 shutdown_connections 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 --2- 172.21.15.19:0/1566781632 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c10a4b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 -- 172.21.15.19:0/1566781632 >> 172.21.15.19:0/1566781632 conn(0x7f7f4c101ca0 msgr2=0x7f7f4c1040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 -- 172.21.15.19:0/1566781632 shutdown_connections 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.197+0000 7f7f51e21640 1 -- 172.21.15.19:0/1566781632 wait complete. 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.198+0000 7f7f51e21640 1 Processor -- start 2024-09-12T03:09:48.622 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.198+0000 7f7f51e21640 1 -- start start 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.198+0000 7f7f51e21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.198+0000 7f7f51e21640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c10b9d0 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56902/0 (socket says 172.21.15.19:56902) 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f4b7fe640 1 -- 172.21.15.19:0/3414078128 learned_addr learned my addr 172.21.15.19:0/3414078128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f4b7fe640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c19bb00 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f4b7fe640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c032ce0 tx=0x7f7f3c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c042070 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c0308c0 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c19bd90 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f3c037040 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.199+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c19ea80 con 0x7f7f4c1080c0 2024-09-12T03:09:48.623 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.200+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7f3c045070 con 0x7f7f4c1080c0 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.201+0000 7f7f497fa640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.201+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7f3803bb90 con 0x7f7f38038fc0 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.201+0000 7f7f42ffd640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.201+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7f3c06c0f0 con 0x7f7f4c1080c0 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.201+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.402+0000 7f7f42ffd640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.402+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.803+0000 7f7f42ffd640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:48.624 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:44.803+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:45.604+0000 7f7f42ffd640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/2981478510 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:45.604+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:47.039+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f7f3c06ac70 con 0x7f7f4c1080c0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:47.039+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 msgr2=0x7f7f3803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:47.039+0000 7f7f497fa640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.041+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f7f3c03c7c0 con 0x7f7f4c1080c0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.041+0000 7f7f497fa640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 0x7f7f3803ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.041+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7f3c036e50 con 0x7f7f3803c8f0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.041+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 0x7f7f3803ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.042+0000 7f7f42ffd640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 0x7f7f3803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f34003980 tx=0x7f7f340092b0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.042+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f7f3c036e50 con 0x7f7f3803c8f0 2024-09-12T03:09:48.625 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.055+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7f4c108520 con 0x7f7f3803c8f0 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.055+0000 7f7f497fa640 1 -- 172.21.15.19:0/3414078128 <== mgr.14118 v2:172.21.15.19:6800/1268895810 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f7f4c108520 con 0x7f7f3803c8f0 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 msgr2=0x7f7f3803ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 0x7f7f3803ece0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7f34003980 tx=0x7f7f340092b0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 msgr2=0x7f7f4c19b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c032ce0 tx=0x7f7f3c032d10 comp rx=0 tx=0).stop 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 shutdown_connections 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f7f3803c8f0 0x7f7f3803ece0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:6800/2981478510,v1:172.21.15.19:6801/2981478510] conn(0x7f7f38038fc0 0x7f7f3803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 --2- 172.21.15.19:0/3414078128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f4c1080c0 0x7f7f4c19b5c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 >> 172.21.15.19:0/3414078128 conn(0x7f7f4c101ca0 msgr2=0x7f7f4c1028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 shutdown_connections 2024-09-12T03:09:48.626 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:48.056+0000 7f7f51e21640 1 -- 172.21.15.19:0/3414078128 wait complete. 2024-09-12T03:09:48.627 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 5 is available 2024-09-12T03:09:48.627 INFO:teuthology.orchestra.run.smithi019.stdout:Setting orchestrator backend to cephadm... 2024-09-12T03:09:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-mon[30953]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T03:09:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-mon[30953]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:49.244+0000 7fd2e6321640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:09:49] ENGINE Error in HTTPServer.serve 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: Traceback (most recent call last): 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._connections.run(self.expiration_interval) 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._run(expiration_interval) 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:09:49.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: s = self.context.wrap_socket( 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: return self.sslsocket_class._create( 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self.do_handshake() 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._sslobj.do_handshake() 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:09:49.416 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:09:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.051+0000 7efc3f6b9640 1 Processor -- start 2024-09-12T03:09:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3f6b9640 1 -- start start 2024-09-12T03:09:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3f6b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.883 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3f6b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc38106ef0 con 0x7efc38106520 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3d455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3d455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56980/0 (socket says 172.21.15.19:56980) 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3d455640 1 -- 172.21.15.19:0/353753294 learned_addr learned my addr 172.21.15.19:0/353753294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.052+0000 7efc3d455640 1 -- 172.21.15.19:0/353753294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc38107720 con 0x7efc38106520 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.053+0000 7efc3d455640 1 --2- 172.21.15.19:0/353753294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efc34009820 tx=0x7efc3402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b6c6c63c8f16ce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc2ffff640 1 -- 172.21.15.19:0/353753294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc34037070 con 0x7efc38106520 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc2ffff640 1 -- 172.21.15.19:0/353753294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc34032900 con 0x7efc38106520 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc2ffff640 1 -- 172.21.15.19:0/353753294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc34030850 con 0x7efc38106520 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 -- 172.21.15.19:0/353753294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 msgr2=0x7efc38106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.884 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/353753294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efc34009820 tx=0x7efc3402a270 comp rx=0 tx=0).stop 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 -- 172.21.15.19:0/353753294 shutdown_connections 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/353753294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc38106920 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 -- 172.21.15.19:0/353753294 >> 172.21.15.19:0/353753294 conn(0x7efc38101db0 msgr2=0x7efc381041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 -- 172.21.15.19:0/353753294 shutdown_connections 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.054+0000 7efc3f6b9640 1 -- 172.21.15.19:0/353753294 wait complete. 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3f6b9640 1 Processor -- start 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3f6b9640 1 -- start start 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3f6b9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3f6b9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc3810aad0 con 0x7efc38106520 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3d455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.885 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3d455640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56982/0 (socket says 172.21.15.19:56982) 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3d455640 1 -- 172.21.15.19:0/1863060453 learned_addr learned my addr 172.21.15.19:0/1863060453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.055+0000 7efc3d455640 1 -- 172.21.15.19:0/1863060453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3819bb80 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc3d455640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc340097f0 tx=0x7efc34030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc34037040 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efc34040070 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3819be10 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc3403a390 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.056+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3819eb00 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.057+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3810ac50 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.057+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7efc3403b430 con 0x7efc38106520 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.058+0000 7efc2e7fc640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 0x7efc2403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:49.886 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.058+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efc3406c2f0 con 0x7efc38106520 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.058+0000 7efc3cc54640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 0x7efc2403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.058+0000 7efc3cc54640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 0x7efc2403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc280098c0 tx=0x7efc28006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.062+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc34032900 con 0x7efc38106520 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.096+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efc34040ca0 con 0x7efc38106520 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.261+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7efc3802da40 con 0x7efc24038e60 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.266+0000 7efc2e7fc640 1 -- 172.21.15.19:0/1863060453 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efc3802da40 con 0x7efc24038e60 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 msgr2=0x7efc2403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 0x7efc2403b320 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efc280098c0 tx=0x7efc28006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 msgr2=0x7efc3819b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efc340097f0 tx=0x7efc34030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 shutdown_connections 2024-09-12T03:09:49.887 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efc24038e60 0x7efc2403b320 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 --2- 172.21.15.19:0/1863060453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc38106520 0x7efc3819b640 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:49.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 >> 172.21.15.19:0/1863060453 conn(0x7efc38101db0 msgr2=0x7efc38102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:49.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 shutdown_connections 2024-09-12T03:09:49.888 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:49.268+0000 7efc3f6b9640 1 -- 172.21.15.19:0/1863060453 wait complete. 2024-09-12T03:09:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: mgrmap e8: x(active, since 2s) 2024-09-12T03:09:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: [12/Sep/2024:03:09:49] ENGINE Bus STARTING 2024-09-12T03:09:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: [12/Sep/2024:03:09:49] ENGINE Serving on https://172.21.15.19:7150 2024-09-12T03:09:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: [12/Sep/2024:03:09:49] ENGINE Error in HTTPServer.serve 2024-09-12T03:09:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout: Traceback (most recent call last): 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._run(expiration_interval) 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: s = self.context.wrap_socket( 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: return self.sslsocket_class._create( 2024-09-12T03:09:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout: self.do_handshake() 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._sslobj.do_handshake() 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: [12/Sep/2024:03:09:49] ENGINE Serving on http://172.21.15.19:8765 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: [12/Sep/2024:03:09:49] ENGINE Bus STARTED 2024-09-12T03:09:50.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:50 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:09:51.047 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.271+0000 7f85e86f7640 1 Processor -- start 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.271+0000 7f85e86f7640 1 -- start start 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e86f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e86f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85e007be50 con 0x7f85e0105f20 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e6493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e6493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56986/0 (socket says 172.21.15.19:56986) 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e6493640 1 -- 172.21.15.19:0/3171318294 learned_addr learned my addr 172.21.15.19:0/3171318294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.272+0000 7f85e6493640 1 -- 172.21.15.19:0/3171318294 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e0106860 con 0x7f85e0105f20 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.273+0000 7f85e6493640 1 --2- 172.21.15.19:0/3171318294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009820 tx=0x7f85dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952bac6b77dd3e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.273+0000 7f85e5491640 1 -- 172.21.15.19:0/3171318294 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85dc037070 con 0x7f85e0105f20 2024-09-12T03:09:51.048 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.273+0000 7f85e5491640 1 -- 172.21.15.19:0/3171318294 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85dc032900 con 0x7f85e0105f20 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.273+0000 7f85e5491640 1 -- 172.21.15.19:0/3171318294 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85dc030850 con 0x7f85e0105f20 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 -- 172.21.15.19:0/3171318294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 msgr2=0x7f85e0106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 --2- 172.21.15.19:0/3171318294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009820 tx=0x7f85dc02a270 comp rx=0 tx=0).stop 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 -- 172.21.15.19:0/3171318294 shutdown_connections 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 --2- 172.21.15.19:0/3171318294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e0106320 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 -- 172.21.15.19:0/3171318294 >> 172.21.15.19:0/3171318294 conn(0x7f85e0101d30 msgr2=0x7f85e0104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 -- 172.21.15.19:0/3171318294 shutdown_connections 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 -- 172.21.15.19:0/3171318294 wait complete. 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.274+0000 7f85e86f7640 1 Processor -- start 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e86f7640 1 -- start start 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e86f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e86f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85e0107960 con 0x7f85e0105f20 2024-09-12T03:09:51.049 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e6493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e6493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57000/0 (socket says 172.21.15.19:57000) 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e6493640 1 -- 172.21.15.19:0/1642438206 learned_addr learned my addr 172.21.15.19:0/1642438206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e6493640 1 -- 172.21.15.19:0/1642438206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e019dda0 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e6493640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85dc031000 tx=0x7f85dc0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85dc042070 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f85dc032be0 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e019e030 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85dc037040 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.275+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e01a0d20 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.276+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f85dc045070 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.276+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e0106320 con 0x7f85e0105f20 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.277+0000 7f85d77fe640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 0x7f85cc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:51.050 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.277+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f85dc06bff0 con 0x7f85e0105f20 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.277+0000 7f85e5c92640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 0x7f85cc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.277+0000 7f85e5c92640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 0x7f85cc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85d00098c0 tx=0x7f85d0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.282+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85dc032d90 con 0x7f85e0105f20 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.459+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f85e002da60 con 0x7f85cc038fc0 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.460+0000 7f85d77fe640 1 -- 172.21.15.19:0/1642438206 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f85e002da60 con 0x7f85cc038fc0 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 msgr2=0x7f85cc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 0x7f85cc03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f85d00098c0 tx=0x7f85d0006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 msgr2=0x7f85e019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f85dc031000 tx=0x7f85dc0326e0 comp rx=0 tx=0).stop 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 shutdown_connections 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f85cc038fc0 0x7f85cc03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 --2- 172.21.15.19:0/1642438206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85e0105f20 0x7f85e019d860 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:51.051 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 >> 172.21.15.19:0/1642438206 conn(0x7f85e0101d30 msgr2=0x7f85e0194520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:51.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 shutdown_connections 2024-09-12T03:09:51.052 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:50.462+0000 7f85e86f7640 1 -- 172.21.15.19:0/1642438206 wait complete. 2024-09-12T03:09:51.052 INFO:teuthology.orchestra.run.smithi019.stdout:Generating ssh key... 2024-09-12T03:09:51.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-mon[30953]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:09:51.914 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: Generating public/private rsa key pair. 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: Your identification has been saved in /tmp/tmp8xscoxcz/key 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: Your public key has been saved in /tmp/tmp8xscoxcz/key.pub 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: The key fingerprint is: 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: SHA256:YD86Iju+n3oXRJKUZQRt7fT9tnlt6O+/4Ayr1HF6cvM ceph-45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: The key's randomart image is: 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: +---[RSA 3072]----+ 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | .oB+. | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | +.+ o | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | +oo . . | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | ..o. . . | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | . S . o | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | .. .. + o | 2024-09-12T03:09:51.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | . . o. . = * +.| 2024-09-12T03:09:51.916 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | .o.o... O B +| 2024-09-12T03:09:51.916 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: | .=*o. ... +.E*| 2024-09-12T03:09:51.916 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:09:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: +----[SHA256]-----+ 2024-09-12T03:09:52.312 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.458+0000 7f9fd5fbc640 1 Processor -- start 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.458+0000 7f9fd5fbc640 1 -- start start 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.458+0000 7f9fd5fbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.458+0000 7f9fd5fbc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd007be70 con 0x7f9fd0108140 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.459+0000 7f9fcf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.459+0000 7f9fcf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45138/0 (socket says 172.21.15.19:45138) 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.459+0000 7f9fcf7fe640 1 -- 172.21.15.19:0/3279736228 learned_addr learned my addr 172.21.15.19:0/3279736228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.459+0000 7f9fcf7fe640 1 -- 172.21.15.19:0/3279736228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd010aa70 con 0x7f9fd0108140 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fcf7fe640 1 --2- 172.21.15.19:0/3279736228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009820 tx=0x7f9fc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a788f11182492c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fce7fc640 1 -- 172.21.15.19:0/3279736228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0037070 con 0x7f9fd0108140 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fce7fc640 1 -- 172.21.15.19:0/3279736228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fc0032900 con 0x7f9fd0108140 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fce7fc640 1 -- 172.21.15.19:0/3279736228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0030850 con 0x7f9fd0108140 2024-09-12T03:09:52.313 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/3279736228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 msgr2=0x7f9fd010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/3279736228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009820 tx=0x7f9fc002a270 comp rx=0 tx=0).stop 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/3279736228 shutdown_connections 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.460+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/3279736228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd010a530 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.461+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/3279736228 >> 172.21.15.19:0/3279736228 conn(0x7f9fd0101de0 msgr2=0x7f9fd0104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.461+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/3279736228 shutdown_connections 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.461+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/3279736228 wait complete. 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.461+0000 7f9fd5fbc640 1 Processor -- start 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.461+0000 7f9fd5fbc640 1 -- start start 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fd5fbc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fd5fbc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd010ba40 con 0x7f9fd0108140 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fcf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:52.314 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fcf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45144/0 (socket says 172.21.15.19:45144) 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fcf7fe640 1 -- 172.21.15.19:0/441091198 learned_addr learned my addr 172.21.15.19:0/441091198 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fcf7fe640 1 -- 172.21.15.19:0/441091198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd019bb20 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fcf7fe640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097f0 tx=0x7f9fc0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc0037040 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd019bdb0 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9fc0040070 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fc003a390 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.462+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd019eaa0 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.463+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9fc003a5e0 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.463+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd01085a0 con 0x7f9fd0108140 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.463+0000 7f9fccff9640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:52.315 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.463+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9fc006c450 con 0x7f9fd0108140 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.464+0000 7f9fceffd640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.464+0000 7f9fceffd640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 0x7f9fb803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098c0 tx=0x7f9fc4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.468+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fc003a9d0 con 0x7f9fd0108140 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.644+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f9fd0103000 con 0x7f9fb8038fc0 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.720+0000 7f9fccff9640 1 -- 172.21.15.19:0/441091198 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9fd0103000 con 0x7f9fb8038fc0 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 msgr2=0x7f9fb803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 0x7f9fb803b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098c0 tx=0x7f9fc4006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 msgr2=0x7f9fd019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097f0 tx=0x7f9fc0030d80 comp rx=0 tx=0).stop 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 shutdown_connections 2024-09-12T03:09:52.316 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f9fb8038fc0 0x7f9fb803b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:52.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 --2- 172.21.15.19:0/441091198 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9fd0108140 0x7f9fd019b5e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:52.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 >> 172.21.15.19:0/441091198 conn(0x7f9fd0101de0 msgr2=0x7f9fd01028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:52.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 shutdown_connections 2024-09-12T03:09:52.317 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:51.722+0000 7f9fd5fbc640 1 -- 172.21.15.19:0/441091198 wait complete. 2024-09-12T03:09:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:52 smithi019 ceph-mon[30953]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:09:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:52 smithi019 ceph-mon[30953]: Generating ssh key... 2024-09-12T03:09:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:52 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:52.899 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:52 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:53.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:09:53.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa83d6b640 1 Processor -- start 2024-09-12T03:09:53.443 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa83d6b640 1 -- start start 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa83d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa83d6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c07c860 con 0x7ffa7c07de20 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa81b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa81b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45152/0 (socket says 172.21.15.19:45152) 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.710+0000 7ffa81b07640 1 -- 172.21.15.19:0/696909158 learned_addr learned my addr 172.21.15.19:0/696909158 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.711+0000 7ffa81b07640 1 -- 172.21.15.19:0/696909158 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c07c9e0 con 0x7ffa7c07de20 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.711+0000 7ffa81b07640 1 --2- 172.21.15.19:0/696909158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009820 tx=0x7ffa7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d91005c3f4cd85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.711+0000 7ffa80b05640 1 -- 172.21.15.19:0/696909158 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa78037070 con 0x7ffa7c07de20 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.711+0000 7ffa80b05640 1 -- 172.21.15.19:0/696909158 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa78032900 con 0x7ffa7c07de20 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa80b05640 1 -- 172.21.15.19:0/696909158 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa78030850 con 0x7ffa7c07de20 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 -- 172.21.15.19:0/696909158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 msgr2=0x7ffa7c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.444 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/696909158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ffa78009820 tx=0x7ffa7802a270 comp rx=0 tx=0).stop 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 -- 172.21.15.19:0/696909158 shutdown_connections 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/696909158 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c07c320 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 -- 172.21.15.19:0/696909158 >> 172.21.15.19:0/696909158 conn(0x7ffa7c102eb0 msgr2=0x7ffa7c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 -- 172.21.15.19:0/696909158 shutdown_connections 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.712+0000 7ffa83d6b640 1 -- 172.21.15.19:0/696909158 wait complete. 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa83d6b640 1 Processor -- start 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa83d6b640 1 -- start start 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa83d6b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa83d6b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffa7c10dcd0 con 0x7ffa7c07de20 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa81b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa81b07640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45166/0 (socket says 172.21.15.19:45166) 2024-09-12T03:09:53.445 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.713+0000 7ffa81b07640 1 -- 172.21.15.19:0/1201733522 learned_addr learned my addr 172.21.15.19:0/1201733522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa81b07640 1 -- 172.21.15.19:0/1201733522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa7c19dde0 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa81b07640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097f0 tx=0x7ffa78030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa78042070 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa78032ca0 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa78037040 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa7c19e070 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.714+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa7c1a0d60 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.715+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ffa78045070 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.715+0000 7ffa72ffd640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 0x7ffa6803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.715+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa7c07c3a0 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.715+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa7806bf80 con 0x7ffa7c07de20 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.715+0000 7ffa81306640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 0x7ffa6803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:53.446 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.716+0000 7ffa81306640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 0x7ffa6803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0098c0 tx=0x7ffa6c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.720+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa78045360 con 0x7ffa7c07de20 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.899+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7ffa7c1041e0 con 0x7ffa68038f70 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.899+0000 7ffa72ffd640 1 -- 172.21.15.19:0/1201733522 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7ffa7c1041e0 con 0x7ffa68038f70 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 msgr2=0x7ffa6803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 0x7ffa6803b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0098c0 tx=0x7ffa6c006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 msgr2=0x7ffa7c19d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ffa780097f0 tx=0x7ffa78030ec0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 shutdown_connections 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ffa68038f70 0x7ffa6803b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 --2- 172.21.15.19:0/1201733522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffa7c07de20 0x7ffa7c19d8a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.901+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 >> 172.21.15.19:0/1201733522 conn(0x7ffa7c102eb0 msgr2=0x7ffa7c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.902+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 shutdown_connections 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:52.902+0000 7ffa83d6b640 1 -- 172.21.15.19:0/1201733522 wait complete. 2024-09-12T03:09:53.447 INFO:teuthology.orchestra.run.smithi019.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:09:53.448 INFO:teuthology.orchestra.run.smithi019.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T03:09:53.448 INFO:teuthology.orchestra.run.smithi019.stdout:Adding host smithi019... 2024-09-12T03:09:54.060 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:53 smithi019 ceph-mon[30953]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:09:55.103 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:54 smithi019 ceph-mon[30953]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:09:56.040 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:55 smithi019 ceph-mon[30953]: Deploying cephadm binary to smithi019 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Added host 'smithi019' with addr '172.21.15.19' 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.867+0000 7fe558aa3640 1 Processor -- start 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe558aa3640 1 -- start start 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe558aa3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe558aa3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe55407c860 con 0x7fe55407de20 2024-09-12T03:09:58.576 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe55259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe55259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45172/0 (socket says 172.21.15.19:45172) 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe55259c640 1 -- 172.21.15.19:0/2323560055 learned_addr learned my addr 172.21.15.19:0/2323560055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.868+0000 7fe55259c640 1 -- 172.21.15.19:0/2323560055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55407c9e0 con 0x7fe55407de20 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.869+0000 7fe55259c640 1 --2- 172.21.15.19:0/2323560055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe544009820 tx=0x7fe54402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a687fe77bb3810 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.869+0000 7fe55159a640 1 -- 172.21.15.19:0/2323560055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe544037070 con 0x7fe55407de20 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.869+0000 7fe55159a640 1 -- 172.21.15.19:0/2323560055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe544032900 con 0x7fe55407de20 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.869+0000 7fe55159a640 1 -- 172.21.15.19:0/2323560055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe544030850 con 0x7fe55407de20 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 -- 172.21.15.19:0/2323560055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 msgr2=0x7fe55407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 --2- 172.21.15.19:0/2323560055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe544009820 tx=0x7fe54402a270 comp rx=0 tx=0).stop 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 -- 172.21.15.19:0/2323560055 shutdown_connections 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 --2- 172.21.15.19:0/2323560055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55407c320 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.577 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 -- 172.21.15.19:0/2323560055 >> 172.21.15.19:0/2323560055 conn(0x7fe554102eb0 msgr2=0x7fe554105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 -- 172.21.15.19:0/2323560055 shutdown_connections 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.870+0000 7fe558aa3640 1 -- 172.21.15.19:0/2323560055 wait complete. 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe558aa3640 1 Processor -- start 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe558aa3640 1 -- start start 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe558aa3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe558aa3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe55410dcd0 con 0x7fe55407de20 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe55259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe55259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45178/0 (socket says 172.21.15.19:45178) 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe55259c640 1 -- 172.21.15.19:0/3620927121 learned_addr learned my addr 172.21.15.19:0/3620927121 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:09:58.578 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.871+0000 7fe55259c640 1 -- 172.21.15.19:0/3620927121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55419dde0 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe55259c640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe544031000 tx=0x7fe5440326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe544042070 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55419e070 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fe544032be0 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe544037040 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.872+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5541a0d60 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.873+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe544045070 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.873+0000 7fe5437fe640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 0x7fe53c03b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.873+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe544032d90 con 0x7fe55407de20 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.873+0000 7fe551d9b640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 0x7fe53c03b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:09:58.579 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.873+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55407c3a0 con 0x7fe55407de20 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.874+0000 7fe551d9b640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 0x7fe53c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5480098c0 tx=0x7fe548006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:53.878+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe544042210 con 0x7fe55407de20 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:54.059+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi019", "addr": "172.21.15.19", "target": ["mon-mgr", ""]}) -- 0x7fe5541041e0 con 0x7fe53c038f70 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.992+0000 7fe5437fe640 1 -- 172.21.15.19:0/3620927121 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe5541041e0 con 0x7fe53c038f70 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 msgr2=0x7fe53c03b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 0x7fe53c03b430 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5480098c0 tx=0x7fe548006db0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 msgr2=0x7fe55419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe544031000 tx=0x7fe5440326e0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 shutdown_connections 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fe53c038f70 0x7fe53c03b430 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 --2- 172.21.15.19:0/3620927121 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe55407de20 0x7fe55419d8a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 >> 172.21.15.19:0/3620927121 conn(0x7fe554102eb0 msgr2=0x7fe554103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:09:58.580 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 shutdown_connections 2024-09-12T03:09:58.581 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:57.995+0000 7fe558aa3640 1 -- 172.21.15.19:0/3620927121 wait complete. 2024-09-12T03:09:58.581 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mon service... 2024-09-12T03:09:59.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:58 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:09:59.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:58 smithi019 ceph-mon[30953]: Added host smithi019 2024-09-12T03:09:59.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:09:58 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:00.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T03:10:00.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:58.999+0000 7f66a91a5640 1 Processor -- start 2024-09-12T03:10:00.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a91a5640 1 -- start start 2024-09-12T03:10:00.001 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a91a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a91a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66a407beb0 con 0x7f66a4108090 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45184/0 (socket says 172.21.15.19:45184) 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a2d9d640 1 -- 172.21.15.19:0/1971922213 learned_addr learned my addr 172.21.15.19:0/1971922213 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.000+0000 7f66a2d9d640 1 -- 172.21.15.19:0/1971922213 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a410a9c0 con 0x7f66a4108090 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.001+0000 7f66a2d9d640 1 --2- 172.21.15.19:0/1971922213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6694009820 tx=0x7f669402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50232d6daa6b82f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.001+0000 7f66a259c640 1 -- 172.21.15.19:0/1971922213 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6694037070 con 0x7f66a4108090 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.001+0000 7f66a259c640 1 -- 172.21.15.19:0/1971922213 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6694032900 con 0x7f66a4108090 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.001+0000 7f66a259c640 1 -- 172.21.15.19:0/1971922213 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6694030850 con 0x7f66a4108090 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 -- 172.21.15.19:0/1971922213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 msgr2=0x7f66a410a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1971922213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6694009820 tx=0x7f669402a270 comp rx=0 tx=0).stop 2024-09-12T03:10:00.002 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 -- 172.21.15.19:0/1971922213 shutdown_connections 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1971922213 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a410a480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 -- 172.21.15.19:0/1971922213 >> 172.21.15.19:0/1971922213 conn(0x7f66a4101db0 msgr2=0x7f66a41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 -- 172.21.15.19:0/1971922213 shutdown_connections 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.002+0000 7f66a91a5640 1 -- 172.21.15.19:0/1971922213 wait complete. 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a91a5640 1 Processor -- start 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a91a5640 1 -- start start 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a91a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a91a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66a410ba60 con 0x7f66a4108090 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a2d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45200/0 (socket says 172.21.15.19:45200) 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a2d9d640 1 -- 172.21.15.19:0/1162550404 learned_addr learned my addr 172.21.15.19:0/1162550404 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:00.003 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.003+0000 7f66a2d9d640 1 -- 172.21.15.19:0/1162550404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a419baf0 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a2d9d640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66940097f0 tx=0x7f6694030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6694037040 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a419bd80 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6694040070 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f669403a390 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.004+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a419ea70 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.005+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f669403b430 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.005+0000 7f66a0d99640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 0x7f669003b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.005+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a41084f0 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.005+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f669406c490 con 0x7f66a4108090 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.005+0000 7f669bfff640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 0x7f669003b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.006+0000 7f669bfff640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 0x7f669003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f668c0098c0 tx=0x7f668c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:00.004 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.010+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f669403a5e0 con 0x7f66a4108090 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.228+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f66a4102fb0 con 0x7f6690038f70 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.233+0000 7f66a0d99640 1 -- 172.21.15.19:0/1162550404 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f66a4102fb0 con 0x7f6690038f70 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 msgr2=0x7f669003b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 0x7f669003b430 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f668c0098c0 tx=0x7f668c006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 msgr2=0x7f66a419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f66940097f0 tx=0x7f6694030d80 comp rx=0 tx=0).stop 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 shutdown_connections 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f6690038f70 0x7f669003b430 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 --2- 172.21.15.19:0/1162550404 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66a4108090 0x7f66a419b5b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 >> 172.21.15.19:0/1162550404 conn(0x7f66a4101db0 msgr2=0x7f66a41028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 shutdown_connections 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:09:59.235+0000 7f66a91a5640 1 -- 172.21.15.19:0/1162550404 wait complete. 2024-09-12T03:10:00.005 INFO:teuthology.orchestra.run.smithi019.stdout:Deploying unmanaged mgr service... 2024-09-12T03:10:00.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:00 smithi019 ceph-mon[30953]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:00.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:00 smithi019 ceph-mon[30953]: Saving service mon spec with placement count:5 2024-09-12T03:10:00.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:00 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:01.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:01 smithi019 ceph-mon[30953]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:01.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:01 smithi019 ceph-mon[30953]: Saving service mgr spec with placement count:2 2024-09-12T03:10:01.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:01 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:01.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:01 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T03:10:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b83d5640 1 Processor -- start 2024-09-12T03:10:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b83d5640 1 -- start start 2024-09-12T03:10:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b83d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.939 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b83d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11b0108110 con 0x7f11b0107740 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b6171640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b6171640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45206/0 (socket says 172.21.15.19:45206) 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.408+0000 7f11b6171640 1 -- 172.21.15.19:0/1881352507 learned_addr learned my addr 172.21.15.19:0/1881352507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.409+0000 7f11b6171640 1 -- 172.21.15.19:0/1881352507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b01088a0 con 0x7f11b0107740 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.409+0000 7f11b6171640 1 --2- 172.21.15.19:0/1881352507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009820 tx=0x7f11ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f335483894ea5b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b516f640 1 -- 172.21.15.19:0/1881352507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac037070 con 0x7f11b0107740 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b516f640 1 -- 172.21.15.19:0/1881352507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11ac032900 con 0x7f11b0107740 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b516f640 1 -- 172.21.15.19:0/1881352507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac030850 con 0x7f11b0107740 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 -- 172.21.15.19:0/1881352507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 msgr2=0x7f11b0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.940 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 --2- 172.21.15.19:0/1881352507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009820 tx=0x7f11ac02a270 comp rx=0 tx=0).stop 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 -- 172.21.15.19:0/1881352507 shutdown_connections 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 --2- 172.21.15.19:0/1881352507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b0107b40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 -- 172.21.15.19:0/1881352507 >> 172.21.15.19:0/1881352507 conn(0x7f11b0102eb0 msgr2=0x7f11b0105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 -- 172.21.15.19:0/1881352507 shutdown_connections 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.410+0000 7f11b83d5640 1 -- 172.21.15.19:0/1881352507 wait complete. 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b83d5640 1 Processor -- start 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b83d5640 1 -- start start 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b83d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b83d5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f11b010bc50 con 0x7f11b0107740 2024-09-12T03:10:01.941 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b6171640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b6171640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45218/0 (socket says 172.21.15.19:45218) 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b6171640 1 -- 172.21.15.19:0/4181907734 learned_addr learned my addr 172.21.15.19:0/4181907734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.411+0000 7f11b6171640 1 -- 172.21.15.19:0/4181907734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b019bbc0 con 0x7f11b0107740 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11b6171640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009940 tx=0x7f11ac0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac042070 con 0x7f11b0107740 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f11ac032be0 con 0x7f11b0107740 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b019be50 con 0x7f11b0107740 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f11ac037040 con 0x7f11b0107740 2024-09-12T03:10:01.942 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.412+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b019eb40 con 0x7f11b0107740 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.413+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f11ac045070 con 0x7f11b0107740 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.413+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11b0107bc0 con 0x7f11b0107740 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.413+0000 7f11a77fe640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 0x7f119c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.413+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f11ac06bf20 con 0x7f11b0107740 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.413+0000 7f11b5970640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 0x7f119c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.414+0000 7f11b5970640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 0x7f119c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098c0 tx=0x7f11a0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.418+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f11ac045360 con 0x7f11b0107740 2024-09-12T03:10:01.943 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.621+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f11b01041c0 con 0x7f119c038fc0 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.626+0000 7f11a77fe640 1 -- 172.21.15.19:0/4181907734 <== mgr.14118 v2:172.21.15.19:6800/1268895810 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f11b01041c0 con 0x7f119c038fc0 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 msgr2=0x7f119c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 0x7f119c03b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f11a00098c0 tx=0x7f11a0006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 msgr2=0x7f11b019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009940 tx=0x7f11ac0326e0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.944 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 shutdown_connections 2024-09-12T03:10:01.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7f119c038fc0 0x7f119c03b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 --2- 172.21.15.19:0/4181907734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f11b0107740 0x7f11b019b680 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:01.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 >> 172.21.15.19:0/4181907734 conn(0x7f11b0102eb0 msgr2=0x7f11b0103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:01.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 shutdown_connections 2024-09-12T03:10:01.945 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:00.628+0000 7f11b83d5640 1 -- 172.21.15.19:0/4181907734 wait complete. 2024-09-12T03:10:03.330 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:03 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:03.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3398785165' entity='client.admin' 2024-09-12T03:10:03.338 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff10169b640 1 Processor -- start 2024-09-12T03:10:03.338 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff10169b640 1 -- start start 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff10169b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff10169b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc106ef0 con 0x7ff0fc106520 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff0faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff0faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45336/0 (socket says 172.21.15.19:45336) 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff0faffd640 1 -- 172.21.15.19:0/3671625997 learned_addr learned my addr 172.21.15.19:0/3671625997 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.384+0000 7ff0faffd640 1 -- 172.21.15.19:0/3671625997 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc107720 con 0x7ff0fc106520 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.385+0000 7ff0faffd640 1 --2- 172.21.15.19:0/3671625997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009820 tx=0x7ff0ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da5835ba4112131 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.385+0000 7ff0fa7fc640 1 -- 172.21.15.19:0/3671625997 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec037070 con 0x7ff0fc106520 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.385+0000 7ff0fa7fc640 1 -- 172.21.15.19:0/3671625997 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ec032920 con 0x7ff0fc106520 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.385+0000 7ff0fa7fc640 1 -- 172.21.15.19:0/3671625997 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec030810 con 0x7ff0fc106520 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 -- 172.21.15.19:0/3671625997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 msgr2=0x7ff0fc106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.339 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 --2- 172.21.15.19:0/3671625997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009820 tx=0x7ff0ec02a270 comp rx=0 tx=0).stop 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 -- 172.21.15.19:0/3671625997 shutdown_connections 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 --2- 172.21.15.19:0/3671625997 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc106920 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 -- 172.21.15.19:0/3671625997 >> 172.21.15.19:0/3671625997 conn(0x7ff0fc101db0 msgr2=0x7ff0fc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 -- 172.21.15.19:0/3671625997 shutdown_connections 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.386+0000 7ff10169b640 1 -- 172.21.15.19:0/3671625997 wait complete. 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff10169b640 1 Processor -- start 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff10169b640 1 -- start start 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff10169b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff10169b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc10aad0 con 0x7ff0fc106520 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff0faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff0faffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45352/0 (socket says 172.21.15.19:45352) 2024-09-12T03:10:03.340 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff0faffd640 1 -- 172.21.15.19:0/3398785165 learned_addr learned my addr 172.21.15.19:0/3398785165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff0faffd640 1 -- 172.21.15.19:0/3398785165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc19bbb0 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.387+0000 7ff0faffd640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec030560 tx=0x7ff0ec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.388+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec037040 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.388+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc19be40 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.388+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0ec040070 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.388+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0ec03a740 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.388+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc19eb30 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff0ec03a9b0 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff0f8ff9640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 0x7ff0e803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0fc10ac50 con 0x7ff0fc106520 2024-09-12T03:10:03.341 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff0ec06c720 con 0x7ff0fc106520 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff0f3fff640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 0x7ff0e803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.389+0000 7ff0f3fff640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 0x7ff0e803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40098c0 tx=0x7ff0e4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.394+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0ec02ee60 con 0x7ff0fc106520 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.575+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7ff0fc02d9b0 con 0x7ff0fc106520 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.577+0000 7ff0f8ff9640 1 -- 172.21.15.19:0/3398785165 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7ff0ec0328f0 con 0x7ff0fc106520 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 msgr2=0x7ff0e803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 0x7ff0e803b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff0e40098c0 tx=0x7ff0e4006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 msgr2=0x7ff0fc19b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec030560 tx=0x7ff0ec032670 comp rx=0 tx=0).stop 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 shutdown_connections 2024-09-12T03:10:03.342 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7ff0e8038f70 0x7ff0e803b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.343 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 --2- 172.21.15.19:0/3398785165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0fc106520 0x7ff0fc19b670 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:03.343 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 >> 172.21.15.19:0/3398785165 conn(0x7ff0fc101db0 msgr2=0x7ff0fc102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:03.343 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 shutdown_connections 2024-09-12T03:10:03.343 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:02.580+0000 7ff10169b640 1 -- 172.21.15.19:0/3398785165 wait complete. 2024-09-12T03:10:04.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.735+0000 7efe00ca0640 1 Processor -- start 2024-09-12T03:10:04.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.735+0000 7efe00ca0640 1 -- start start 2024-09-12T03:10:04.587 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efe00ca0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efe00ca0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc07be30 con 0x7efdfc108180 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45360/0 (socket says 172.21.15.19:45360) 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efdfa59c640 1 -- 172.21.15.19:0/3476888288 learned_addr learned my addr 172.21.15.19:0/3476888288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.736+0000 7efdfa59c640 1 -- 172.21.15.19:0/3476888288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc10aab0 con 0x7efdfc108180 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.737+0000 7efdfa59c640 1 --2- 172.21.15.19:0/3476888288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efde4009820 tx=0x7efde402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89f3582366abe53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.737+0000 7efdf959a640 1 -- 172.21.15.19:0/3476888288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4037070 con 0x7efdfc108180 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.737+0000 7efdf959a640 1 -- 172.21.15.19:0/3476888288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde4032920 con 0x7efdfc108180 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.737+0000 7efdf959a640 1 -- 172.21.15.19:0/3476888288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4030810 con 0x7efdfc108180 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 -- 172.21.15.19:0/3476888288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 msgr2=0x7efdfc10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 --2- 172.21.15.19:0/3476888288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efde4009820 tx=0x7efde402a270 comp rx=0 tx=0).stop 2024-09-12T03:10:04.588 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 -- 172.21.15.19:0/3476888288 shutdown_connections 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 --2- 172.21.15.19:0/3476888288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc10a570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 -- 172.21.15.19:0/3476888288 >> 172.21.15.19:0/3476888288 conn(0x7efdfc101de0 msgr2=0x7efdfc104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 -- 172.21.15.19:0/3476888288 shutdown_connections 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 -- 172.21.15.19:0/3476888288 wait complete. 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.738+0000 7efe00ca0640 1 Processor -- start 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efe00ca0640 1 -- start start 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efe00ca0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efe00ca0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efdfc10ba60 con 0x7efdfc108180 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdfa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45362/0 (socket says 172.21.15.19:45362) 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdfa59c640 1 -- 172.21.15.19:0/2245269116 learned_addr learned my addr 172.21.15.19:0/2245269116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:04.589 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdfa59c640 1 -- 172.21.15.19:0/2245269116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdfc107f40 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdfa59c640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efde4030380 tx=0x7efde4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde4037040 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efde4040070 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.739+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efde403a7b0 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.740+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdfc106130 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.740+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdfc106670 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.740+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efde403a9e0 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.741+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdfc1085e0 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.741+0000 7efdf37fe640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 0x7efde803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.741+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efde406c710 con 0x7efdfc108180 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.741+0000 7efdf9d9b640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 0x7efde803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.741+0000 7efdf9d9b640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 0x7efde803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdec0098c0 tx=0x7efdec006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:04.590 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.745+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efde4044c70 con 0x7efdfc108180 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.921+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7efdfc106fc0 con 0x7efdfc108180 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.923+0000 7efdf37fe640 1 -- 172.21.15.19:0/2245269116 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7efdfc106fc0 con 0x7efdfc108180 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 msgr2=0x7efde803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 0x7efde803b480 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efdec0098c0 tx=0x7efdec006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 msgr2=0x7efdfc107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efde4030380 tx=0x7efde4032670 comp rx=0 tx=0).stop 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 shutdown_connections 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7efde8038fc0 0x7efde803b480 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 --2- 172.21.15.19:0/2245269116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efdfc108180 0x7efdfc107a00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 >> 172.21.15.19:0/2245269116 conn(0x7efdfc101de0 msgr2=0x7efdfc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 shutdown_connections 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:03.926+0000 7efe00ca0640 1 -- 172.21.15.19:0/2245269116 wait complete. 2024-09-12T03:10:04.591 INFO:teuthology.orchestra.run.smithi019.stdout:Enabling the dashboard module... 2024-09-12T03:10:04.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:04 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2245269116' entity='client.admin' 2024-09-12T03:10:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/929846416' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T03:10:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-mon[30953]: from='mgr.14118 172.21.15.19:0/2274757669' entity='mgr.x' 2024-09-12T03:10:06.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ignoring --setuser ceph since I am not root 2024-09-12T03:10:06.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ignoring --setgroup ceph since I am not root 2024-09-12T03:10:06.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.240+0000 7faa774a9040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:10:06.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.303+0000 7faa774a9040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:10:06.829 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.457+0000 7faa774a9040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:10:06.830 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.526+0000 7faa774a9040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:10:06.830 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.682+0000 7faa774a9040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:10:06.830 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:06.829+0000 7faa774a9040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:10:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:07 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/929846416' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T03:10:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:07 smithi019 ceph-mon[30953]: mgrmap e9: x(active, since 19s) 2024-09-12T03:10:07.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:07 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:07.156+0000 7faa774a9040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:10:07.415 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:07 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:07.220+0000 7faa774a9040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:10:07.684 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:07 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:07.414+0000 7faa774a9040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad84552640 1 Processor -- start 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad84552640 1 -- start start 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad84552640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad84552640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c07c860 con 0x7fad7c07de20 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad822ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad822ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45372/0 (socket says 172.21.15.19:45372) 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.992+0000 7fad822ee640 1 -- 172.21.15.19:0/2642243560 learned_addr learned my addr 172.21.15.19:0/2642243560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.993+0000 7fad822ee640 1 -- 172.21.15.19:0/2642243560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c07c9e0 con 0x7fad7c07de20 2024-09-12T03:10:07.692 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.993+0000 7fad822ee640 1 --2- 172.21.15.19:0/2642243560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad78009820 tx=0x7fad7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd39a31851221195 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad812ec640 1 -- 172.21.15.19:0/2642243560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad78037070 con 0x7fad7c07de20 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad812ec640 1 -- 172.21.15.19:0/2642243560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78032920 con 0x7fad7c07de20 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad812ec640 1 -- 172.21.15.19:0/2642243560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad78030810 con 0x7fad7c07de20 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 -- 172.21.15.19:0/2642243560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 msgr2=0x7fad7c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 --2- 172.21.15.19:0/2642243560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fad78009820 tx=0x7fad7802a270 comp rx=0 tx=0).stop 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 -- 172.21.15.19:0/2642243560 shutdown_connections 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 --2- 172.21.15.19:0/2642243560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c07c320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 -- 172.21.15.19:0/2642243560 >> 172.21.15.19:0/2642243560 conn(0x7fad7c102eb0 msgr2=0x7fad7c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 -- 172.21.15.19:0/2642243560 shutdown_connections 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.994+0000 7fad84552640 1 -- 172.21.15.19:0/2642243560 wait complete. 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad84552640 1 Processor -- start 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad84552640 1 -- start start 2024-09-12T03:10:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad84552640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad84552640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c10dcc0 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad822ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad822ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45374/0 (socket says 172.21.15.19:45374) 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.995+0000 7fad822ee640 1 -- 172.21.15.19:0/929846416 learned_addr learned my addr 172.21.15.19:0/929846416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad822ee640 1 -- 172.21.15.19:0/929846416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c19de00 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad822ee640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad780097f0 tx=0x7fad780327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad78037040 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad78030c40 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c19e090 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad7803a5d0 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.996+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c1a0d80 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.997+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fad7803b3f0 con 0x7fad7c07de20 2024-09-12T03:10:07.694 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.997+0000 7fad737fe640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 0x7fad6803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.997+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad7806c450 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.997+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7c07c3a0 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.997+0000 7fad81aed640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 0x7fad6803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:04.998+0000 7fad81aed640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 0x7fad6803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098c0 tx=0x7fad6c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:05.002+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad780313a0 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:05.232+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fad7c1a1210 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.084+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fad78038250 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.086+0000 7fad737fe640 1 -- 172.21.15.19:0/929846416 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fad78041d10 con 0x7fad7c07de20 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 msgr2=0x7fad6803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 0x7fad6803b480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fad6c0098c0 tx=0x7fad6c006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 msgr2=0x7fad7c19d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad780097f0 tx=0x7fad780327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.695 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 shutdown_connections 2024-09-12T03:10:07.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fad68038fc0 0x7fad6803b480 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 --2- 172.21.15.19:0/929846416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad7c07de20 0x7fad7c19d8c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:07.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.087+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 >> 172.21.15.19:0/929846416 conn(0x7fad7c102eb0 msgr2=0x7fad7c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:07.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.088+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 shutdown_connections 2024-09-12T03:10:07.696 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:06.088+0000 7fad84552640 1 -- 172.21.15.19:0/929846416 wait complete. 2024-09-12T03:10:08.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:08 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3240260037' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T03:10:08.531 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:08 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:08.367+0000 7faa774a9040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:10:08.532 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:08 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:08.438+0000 7faa774a9040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:10:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:10:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T03:10:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T03:10:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T03:10:08.908 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.042+0000 7fa9a65b2640 1 Processor -- start 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.042+0000 7fa9a65b2640 1 -- start start 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.042+0000 7fa9a65b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.042+0000 7fa9a65b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a007beb0 con 0x7fa9a0108090 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.043+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.043+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45396/0 (socket says 172.21.15.19:45396) 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.043+0000 7fa99ffff640 1 -- 172.21.15.19:0/1222792353 learned_addr learned my addr 172.21.15.19:0/1222792353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.043+0000 7fa99ffff640 1 -- 172.21.15.19:0/1222792353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a010a9c0 con 0x7fa9a0108090 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.044+0000 7fa99ffff640 1 --2- 172.21.15.19:0/1222792353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa990009820 tx=0x7fa99002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4888d6c97d7883bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:08.909 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.044+0000 7fa99f7fe640 1 -- 172.21.15.19:0/1222792353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa990037070 con 0x7fa9a0108090 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.044+0000 7fa99f7fe640 1 -- 172.21.15.19:0/1222792353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa990032920 con 0x7fa9a0108090 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.044+0000 7fa99f7fe640 1 -- 172.21.15.19:0/1222792353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa990030810 con 0x7fa9a0108090 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- 172.21.15.19:0/1222792353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 msgr2=0x7fa9a010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/1222792353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa990009820 tx=0x7fa99002a270 comp rx=0 tx=0).stop 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- 172.21.15.19:0/1222792353 shutdown_connections 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/1222792353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a010a480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:08.910 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- 172.21.15.19:0/1222792353 >> 172.21.15.19:0/1222792353 conn(0x7fa9a0101db0 msgr2=0x7fa9a01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- 172.21.15.19:0/1222792353 shutdown_connections 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- 172.21.15.19:0/1222792353 wait complete. 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 Processor -- start 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.045+0000 7fa9a65b2640 1 -- start start 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa9a65b2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa9a65b2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a010baa0 con 0x7fa9a0108090 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45412/0 (socket says 172.21.15.19:45412) 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99ffff640 1 -- 172.21.15.19:0/3240260037 learned_addr learned my addr 172.21.15.19:0/3240260037 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99ffff640 1 -- 172.21.15.19:0/3240260037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a019bb70 con 0x7fa9a0108090 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99ffff640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9900097f0 tx=0x7fa9900327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:08.911 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa990037040 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a019be00 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa990030cc0 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa99003a6b0 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.046+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a019eaf0 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.047+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a01084f0 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.048+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa99003baf0 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.048+0000 7fa99dffb640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 0x7fa98c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.048+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa99006c550 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.048+0000 7fa997fff640 1 -- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 msgr2=0x7fa98c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.048+0000 7fa997fff640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 0x7fa98c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.052+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa990042e10 con 0x7fa9a0108090 2024-09-12T03:10:08.912 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.249+0000 7fa997fff640 1 -- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 msgr2=0x7fa98c03b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.249+0000 7fa997fff640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 0x7fa98c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.279+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa9a019ecc0 con 0x7fa9a0108090 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.280+0000 7fa99dffb640 1 -- 172.21.15.19:0/3240260037 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fa9a019ecc0 con 0x7fa9a0108090 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 msgr2=0x7fa98c03b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 0x7fa98c03b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 msgr2=0x7fa9a019b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa9900097f0 tx=0x7fa9900327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 shutdown_connections 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fa98c038fc0 0x7fa98c03b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 --2- 172.21.15.19:0/3240260037 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa9a0108090 0x7fa9a019b630 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 >> 172.21.15.19:0/3240260037 conn(0x7fa9a0101db0 msgr2=0x7fa9a01028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 shutdown_connections 2024-09-12T03:10:08.913 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:08.282+0000 7fa9a65b2640 1 -- 172.21.15.19:0/3240260037 wait complete. 2024-09-12T03:10:08.914 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for the mgr to restart... 2024-09-12T03:10:08.914 INFO:teuthology.orchestra.run.smithi019.stdout:Waiting for mgr epoch 9... 2024-09-12T03:10:09.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.206+0000 7faa774a9040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:10:09.414 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.333+0000 7faa774a9040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:10:09.776 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.483+0000 7faa774a9040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:10:09.776 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.548+0000 7faa774a9040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:10:09.776 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.705+0000 7faa774a9040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:10:09.777 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.776+0000 7faa774a9040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:10:10.072 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:09 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:09.839+0000 7faa774a9040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:10:10.072 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.004+0000 7faa774a9040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:10:10.072 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.072+0000 7faa774a9040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:10:10.351 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.141+0000 7faa774a9040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:10:10.351 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.227+0000 7faa774a9040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:10:10.352 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.289+0000 7faa774a9040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:10:10.352 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:10 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:10.351+0000 7faa774a9040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:11.177+0000 7faa774a9040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: Active manager daemon x restarted 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: Activating manager daemon x 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: mgrmap e10: x(active, starting, since 0.00337223s) 2024-09-12T03:10:11.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: Manager daemon x is now available 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T03:10:11.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:12.772+0000 7fa9e27bd640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:03:10:12] ENGINE Error in HTTPServer.serve 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: Traceback (most recent call last): 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._connections.run(self.expiration_interval) 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._run(expiration_interval) 2024-09-12T03:10:12.868 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: s = self.context.wrap_socket( 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: return self.sslsocket_class._create( 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self.do_handshake() 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: self._sslobj.do_handshake() 2024-09-12T03:10:12.869 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:10:12.870 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:10:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:10:13.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout { 2024-09-12T03:10:13.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T03:10:13.143 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout } 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd314ba1640 1 Processor -- start 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd314ba1640 1 -- start start 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd314ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd314ba1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd31007be60 con 0x7fd310105e70 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd30e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.310+0000 7fd30e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45416/0 (socket says 172.21.15.19:45416) 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.311+0000 7fd30e59c640 1 -- 172.21.15.19:0/2716875770 learned_addr learned my addr 172.21.15.19:0/2716875770 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.311+0000 7fd30e59c640 1 -- 172.21.15.19:0/2716875770 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3101067b0 con 0x7fd310105e70 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd30e59c640 1 --2- 172.21.15.19:0/2716875770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd300009820 tx=0x7fd30002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c0b09c1013997a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd30d59a640 1 -- 172.21.15.19:0/2716875770 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd300037070 con 0x7fd310105e70 2024-09-12T03:10:13.144 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd30d59a640 1 -- 172.21.15.19:0/2716875770 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd300032920 con 0x7fd310105e70 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd30d59a640 1 -- 172.21.15.19:0/2716875770 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd300030810 con 0x7fd310105e70 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd314ba1640 1 -- 172.21.15.19:0/2716875770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 msgr2=0x7fd310106270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.312+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2716875770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd300009820 tx=0x7fd30002a270 comp rx=0 tx=0).stop 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.313+0000 7fd314ba1640 1 -- 172.21.15.19:0/2716875770 shutdown_connections 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.313+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2716875770 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd310106270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.313+0000 7fd314ba1640 1 -- 172.21.15.19:0/2716875770 >> 172.21.15.19:0/2716875770 conn(0x7fd310101ca0 msgr2=0x7fd3101040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.313+0000 7fd314ba1640 1 -- 172.21.15.19:0/2716875770 shutdown_connections 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.313+0000 7fd314ba1640 1 -- 172.21.15.19:0/2716875770 wait complete. 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd314ba1640 1 Processor -- start 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd314ba1640 1 -- start start 2024-09-12T03:10:13.145 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd314ba1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd314ba1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3101078f0 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd30e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd30e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:45418/0 (socket says 172.21.15.19:45418) 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd30e59c640 1 -- 172.21.15.19:0/2970939470 learned_addr learned my addr 172.21.15.19:0/2970939470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd30e59c640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31019dd00 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.314+0000 7fd30e59c640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3000097f0 tx=0x7fd3000327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.315+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd300037040 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.315+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31019df90 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.315+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd300030cc0 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.315+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd30003a6b0 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.315+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3101a0c80 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd30003a8a0 con 0x7fd310105e70 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd2ff7fe640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.146 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd2f803bb90 con 0x7fd2f8038fc0 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd30006c540 con 0x7fd310105e70 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd30dd9b640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 msgr2=0x7fd2f803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.316+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.517+0000 7fd30dd9b640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 msgr2=0x7fd2f803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.517+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.918+0000 7fd30dd9b640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 msgr2=0x7fd2f803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:09.918+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:10.719+0000 7fd30dd9b640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 msgr2=0x7fd2f803b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.19:6800/1268895810 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:10.719+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:11.181+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fd30002fd60 con 0x7fd310105e70 2024-09-12T03:10:13.147 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:11.181+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 msgr2=0x7fd2f803b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:11.181+0000 7fd2ff7fe640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.183+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fd300040d60 con 0x7fd310105e70 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.183+0000 7fd2ff7fe640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 0x7fd2f803ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.183+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd300001d30 con 0x7fd2f803ca80 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.184+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 0x7fd2f803ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.184+0000 7fd30dd9b640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 0x7fd2f803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd304003980 tx=0x7fd3040092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.185+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd300001d30 con 0x7fd2f803ca80 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd310106270 con 0x7fd2f803ca80 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd2ff7fe640 1 -- 172.21.15.19:0/2970939470 <== mgr.14150 v2:172.21.15.19:6800/2264764467 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fd310106270 con 0x7fd2f803ca80 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 msgr2=0x7fd2f803ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 0x7fd2f803ee70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd304003980 tx=0x7fd3040092b0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.148 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 msgr2=0x7fd31019d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd3000097f0 tx=0x7fd3000327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 shutdown_connections 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/1268895810,v1:172.21.15.19:6801/1268895810] conn(0x7fd2f8038fc0 0x7fd2f803b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd2f803ca80 0x7fd2f803ee70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 --2- 172.21.15.19:0/2970939470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd310105e70 0x7fd31019d7c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 >> 172.21.15.19:0/2970939470 conn(0x7fd310101ca0 msgr2=0x7fd310194460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 shutdown_connections 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:12.192+0000 7fd314ba1640 1 -- 172.21.15.19:0/2970939470 wait complete. 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:mgr epoch 9 is available 2024-09-12T03:10:13.149 INFO:teuthology.orchestra.run.smithi019.stdout:Generating a dashboard self-signed certificate... 2024-09-12T03:10:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: mgrmap e11: x(active, since 1.00596s) 2024-09-12T03:10:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: [12/Sep/2024:03:10:12] ENGINE Bus STARTING 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: [12/Sep/2024:03:10:12] ENGINE Serving on https://172.21.15.19:7150 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: [12/Sep/2024:03:10:12] ENGINE Error in HTTPServer.serve 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: Traceback (most recent call last): 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._connections.run(self.expiration_interval) 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._run(expiration_interval) 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T03:10:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: s = self.context.wrap_socket( 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: return self.sslsocket_class._create( 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: self.do_handshake() 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: self._sslobj.do_handshake() 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: [12/Sep/2024:03:10:12] ENGINE Serving on http://172.21.15.19:8765 2024-09-12T03:10:13.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:13 smithi019 ceph-mon[30953]: [12/Sep/2024:03:10:12] ENGINE Bus STARTED 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.548+0000 7f9f618ba640 1 Processor -- start 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.548+0000 7f9f618ba640 1 -- start start 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f618ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f618ba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c07bdd0 con 0x7f9f5c108170 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:14.423 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46698/0 (socket says 172.21.15.19:46698) 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f5affd640 1 -- 172.21.15.19:0/1991398229 learned_addr learned my addr 172.21.15.19:0/1991398229 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.549+0000 7f9f5affd640 1 -- 172.21.15.19:0/1991398229 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c10aaa0 con 0x7f9f5c108170 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.550+0000 7f9f5affd640 1 --2- 172.21.15.19:0/1991398229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009820 tx=0x7f9f4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30c8849ab60cd30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.550+0000 7f9f5a7fc640 1 -- 172.21.15.19:0/1991398229 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c037070 con 0x7f9f5c108170 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.550+0000 7f9f5a7fc640 1 -- 172.21.15.19:0/1991398229 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c032920 con 0x7f9f5c108170 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.550+0000 7f9f5a7fc640 1 -- 172.21.15.19:0/1991398229 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c030810 con 0x7f9f5c108170 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 -- 172.21.15.19:0/1991398229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 msgr2=0x7f9f5c10a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 --2- 172.21.15.19:0/1991398229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009820 tx=0x7f9f4c02a270 comp rx=0 tx=0).stop 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 -- 172.21.15.19:0/1991398229 shutdown_connections 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 --2- 172.21.15.19:0/1991398229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c10a560 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 -- 172.21.15.19:0/1991398229 >> 172.21.15.19:0/1991398229 conn(0x7f9f5c101db0 msgr2=0x7f9f5c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 -- 172.21.15.19:0/1991398229 shutdown_connections 2024-09-12T03:10:14.424 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 -- 172.21.15.19:0/1991398229 wait complete. 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.551+0000 7f9f618ba640 1 Processor -- start 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f618ba640 1 -- start start 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f618ba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f618ba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5c10ba60 con 0x7f9f5c108170 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46706/0 (socket says 172.21.15.19:46706) 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f5affd640 1 -- 172.21.15.19:0/3984039386 learned_addr learned my addr 172.21.15.19:0/3984039386 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f5affd640 1 -- 172.21.15.19:0/3984039386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5c19bb30 con 0x7f9f5c108170 2024-09-12T03:10:14.425 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f5affd640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097f0 tx=0x7f9f4c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.552+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c037040 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.553+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f4c030cc0 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.553+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f4c03a6b0 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.553+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5c19bdc0 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.553+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f5c19eab0 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.553+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f9f4c03a850 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.554+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f5c1085d0 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.554+0000 7f9f608b8640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 0x7f9f4803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.554+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9f4c06c640 con 0x7f9f5c108170 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.554+0000 7f9f53fff640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 0x7f9f4803b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.554+0000 7f9f53fff640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 0x7f9f4803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f440098c0 tx=0x7f9f44006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:14.426 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.559+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f4c0371e0 con 0x7f9f5c108170 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.740+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f9f5c102ff0 con 0x7f9f48039020 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.858+0000 7f9f608b8640 1 -- 172.21.15.19:0/3984039386 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f9f5c102ff0 con 0x7f9f48039020 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 msgr2=0x7f9f4803b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 0x7f9f4803b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9f440098c0 tx=0x7f9f44006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 msgr2=0x7f9f5c19b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0097f0 tx=0x7f9f4c0327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 shutdown_connections 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f48039020 0x7f9f4803b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 --2- 172.21.15.19:0/3984039386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f5c108170 0x7f9f5c19b5f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 >> 172.21.15.19:0/3984039386 conn(0x7f9f5c101db0 msgr2=0x7f9f5c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 shutdown_connections 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:13.861+0000 7f9f618ba640 1 -- 172.21.15.19:0/3984039386 wait complete. 2024-09-12T03:10:14.427 INFO:teuthology.orchestra.run.smithi019.stdout:Creating initial admin user... 2024-09-12T03:10:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:14 smithi019 ceph-mon[30953]: mgrmap e12: x(active, since 2s) 2024-09-12T03:10:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:14 smithi019 ceph-mon[30953]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:14 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:14 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:14.735 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:14 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:15.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:15 smithi019 ceph-mon[30953]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:15.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:15.890 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$AmMvt97tAux5IvWRae2I8eY.Zqhx.AGMMEohfiBKs1nPHfV0Nr6yK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726110615, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T03:10:15.890 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.818+0000 7f93c7d79640 1 Processor -- start 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c7d79640 1 -- start start 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c7d79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c7d79640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93c0108050 con 0x7f93c0107680 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c5b15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c5b15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46714/0 (socket says 172.21.15.19:46714) 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c5b15640 1 -- 172.21.15.19:0/2631859778 learned_addr learned my addr 172.21.15.19:0/2631859778 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.819+0000 7f93c5b15640 1 -- 172.21.15.19:0/2631859778 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c0108830 con 0x7f93c0107680 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.820+0000 7f93c5b15640 1 --2- 172.21.15.19:0/2631859778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009820 tx=0x7f93bc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149c5b5277edf424 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.820+0000 7f93c4b13640 1 -- 172.21.15.19:0/2631859778 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93bc037070 con 0x7f93c0107680 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.820+0000 7f93c4b13640 1 -- 172.21.15.19:0/2631859778 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc032920 con 0x7f93c0107680 2024-09-12T03:10:15.891 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.820+0000 7f93c4b13640 1 -- 172.21.15.19:0/2631859778 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93bc030810 con 0x7f93c0107680 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 -- 172.21.15.19:0/2631859778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 msgr2=0x7f93c0107a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2631859778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f93bc009820 tx=0x7f93bc02a270 comp rx=0 tx=0).stop 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 -- 172.21.15.19:0/2631859778 shutdown_connections 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2631859778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c0107a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 -- 172.21.15.19:0/2631859778 >> 172.21.15.19:0/2631859778 conn(0x7f93c0102e50 msgr2=0x7f93c0105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 -- 172.21.15.19:0/2631859778 shutdown_connections 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.821+0000 7f93c7d79640 1 -- 172.21.15.19:0/2631859778 wait complete. 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c7d79640 1 Processor -- start 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c7d79640 1 -- start start 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c7d79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c7d79640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f93c010bbe0 con 0x7f93c0107680 2024-09-12T03:10:15.892 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c5b15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c5b15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46720/0 (socket says 172.21.15.19:46720) 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c5b15640 1 -- 172.21.15.19:0/2135042613 learned_addr learned my addr 172.21.15.19:0/2135042613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.822+0000 7f93c5b15640 1 -- 172.21.15.19:0/2135042613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c007e0b0 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93c5b15640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097f0 tx=0x7f93bc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93bc037040 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c007c2a0 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f93bc030c40 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c007c7e0 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.823+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f93bc03a5b0 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.824+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f93bc03b3f0 con 0x7f93c0107680 2024-09-12T03:10:15.893 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.824+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c010bd60 con 0x7f93c0107680 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.824+0000 7f93b6ffd640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 0x7f93ac03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.824+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f93bc06c630 con 0x7f93c0107680 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.825+0000 7f93c5314640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 0x7f93ac03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.825+0000 7f93c5314640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 0x7f93ac03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098c0 tx=0x7f93b0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:14.829+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f93bc0313a0 con 0x7f93c0107680 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.023+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f93c0107b00 con 0x7f93ac038fc0 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.272+0000 7f93b6ffd640 1 -- 172.21.15.19:0/2135042613 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f93c0107b00 con 0x7f93ac038fc0 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 msgr2=0x7f93ac03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 0x7f93ac03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f93b00098c0 tx=0x7f93b0006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 msgr2=0x7f93c007db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:15.894 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f93bc0097f0 tx=0x7f93bc0327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 shutdown_connections 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f93ac038fc0 0x7f93ac03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 --2- 172.21.15.19:0/2135042613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f93c0107680 0x7f93c007db70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 >> 172.21.15.19:0/2135042613 conn(0x7f93c0102e50 msgr2=0x7f93c0103a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 shutdown_connections 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:15.274+0000 7f93c7d79640 1 -- 172.21.15.19:0/2135042613 wait complete. 2024-09-12T03:10:15.895 INFO:teuthology.orchestra.run.smithi019.stdout:Fetching dashboard port number... 2024-09-12T03:10:16.730 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:16 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2605767242' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T03:10:17.375 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.280+0000 7fe56e621640 1 Processor -- start 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.280+0000 7fe56e621640 1 -- start start 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe56e621640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe56e621640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe56807be70 con 0x7fe5681080d0 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe567fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe567fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46722/0 (socket says 172.21.15.19:46722) 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe567fff640 1 -- 172.21.15.19:0/2274202010 learned_addr learned my addr 172.21.15.19:0/2274202010 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.281+0000 7fe567fff640 1 -- 172.21.15.19:0/2274202010 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56810aa00 con 0x7fe5681080d0 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.282+0000 7fe567fff640 1 --2- 172.21.15.19:0/2274202010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe558009820 tx=0x7fe55802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767501439d79a9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.282+0000 7fe566ffd640 1 -- 172.21.15.19:0/2274202010 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558037070 con 0x7fe5681080d0 2024-09-12T03:10:17.376 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.282+0000 7fe566ffd640 1 -- 172.21.15.19:0/2274202010 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe558032920 con 0x7fe5681080d0 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.282+0000 7fe566ffd640 1 -- 172.21.15.19:0/2274202010 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558030810 con 0x7fe5681080d0 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- 172.21.15.19:0/2274202010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 msgr2=0x7fe56810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 --2- 172.21.15.19:0/2274202010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe558009820 tx=0x7fe55802a270 comp rx=0 tx=0).stop 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- 172.21.15.19:0/2274202010 shutdown_connections 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 --2- 172.21.15.19:0/2274202010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56810a4c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- 172.21.15.19:0/2274202010 >> 172.21.15.19:0/2274202010 conn(0x7fe568101db0 msgr2=0x7fe5681041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- 172.21.15.19:0/2274202010 shutdown_connections 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- 172.21.15.19:0/2274202010 wait complete. 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 Processor -- start 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.283+0000 7fe56e621640 1 -- start start 2024-09-12T03:10:17.377 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe56e621640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe56e621640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810ba80 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe567fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe567fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46734/0 (socket says 172.21.15.19:46734) 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe567fff640 1 -- 172.21.15.19:0/2605767242 learned_addr learned my addr 172.21.15.19:0/2605767242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe567fff640 1 -- 172.21.15.19:0/2605767242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56819dd70 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.284+0000 7fe567fff640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe558030380 tx=0x7fe558032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.285+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558037040 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.285+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56819e000 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.285+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe558040070 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.285+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe55803a7b0 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.285+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681a0cf0 con 0x7fe5681080d0 2024-09-12T03:10:17.378 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.286+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fe55803aa30 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.286+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe568108530 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.286+0000 7fe5657fa640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 0x7fe55003b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.286+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe55806c810 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.286+0000 7fe5677fe640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 0x7fe55003b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.287+0000 7fe5677fe640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 0x7fe55003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098c0 tx=0x7fe55c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.295+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe558041270 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.474+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fe5681a0ec0 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.474+0000 7fe5657fa640 1 -- 172.21.15.19:0/2605767242 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fe55803ae70 con 0x7fe5681080d0 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 msgr2=0x7fe55003b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 0x7fe55003b4e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0098c0 tx=0x7fe55c006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 msgr2=0x7fe56819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:17.379 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe558030380 tx=0x7fe558032670 comp rx=0 tx=0).stop 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 shutdown_connections 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe550039020 0x7fe55003b4e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 --2- 172.21.15.19:0/2605767242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5681080d0 0x7fe56819d830 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 >> 172.21.15.19:0/2605767242 conn(0x7fe568101db0 msgr2=0x7fe568102890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 shutdown_connections 2024-09-12T03:10:17.380 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:16.477+0000 7fe56e621640 1 -- 172.21.15.19:0/2605767242 wait complete. 2024-09-12T03:10:17.386 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T03:10:17.386 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout disabled 2024-09-12T03:10:17.395 INFO:teuthology.orchestra.run.smithi019.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T03:10:17.395 INFO:teuthology.orchestra.run.smithi019.stdout:systemctl: stdout inactive 2024-09-12T03:10:17.395 INFO:teuthology.orchestra.run.smithi019.stdout:firewalld.service is not enabled 2024-09-12T03:10:17.395 INFO:teuthology.orchestra.run.smithi019.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T03:10:17.397 INFO:teuthology.orchestra.run.smithi019.stdout:Ceph Dashboard is now available at: 2024-09-12T03:10:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: URL: https://smithi019.front.sepia.ceph.com:8443/ 2024-09-12T03:10:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: User: admin 2024-09-12T03:10:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: Password: u2qjjykzjy 2024-09-12T03:10:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:17.398 INFO:teuthology.orchestra.run.smithi019.stdout:Saving cluster configuration to /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config directory 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.812+0000 7ff0ef0de640 1 Processor -- start 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.812+0000 7ff0ef0de640 1 -- start start 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.812+0000 7ff0ef0de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.813+0000 7ff0ef0de640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e807be70 con 0x7ff0e8108140 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.813+0000 7ff0ece7a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.813+0000 7ff0ece7a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46748/0 (socket says 172.21.15.19:46748) 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.813+0000 7ff0ece7a640 1 -- 172.21.15.19:0/226979016 learned_addr learned my addr 172.21.15.19:0/226979016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.813+0000 7ff0ece7a640 1 -- 172.21.15.19:0/226979016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e810aa70 con 0x7ff0e8108140 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.814+0000 7ff0ece7a640 1 --2- 172.21.15.19:0/226979016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009820 tx=0x7ff0d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de07411be488c91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:18.631 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.814+0000 7ff0e77fe640 1 -- 172.21.15.19:0/226979016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0d8037070 con 0x7ff0e8108140 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.814+0000 7ff0e77fe640 1 -- 172.21.15.19:0/226979016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d8032920 con 0x7ff0e8108140 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.814+0000 7ff0e77fe640 1 -- 172.21.15.19:0/226979016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0d8030810 con 0x7ff0e8108140 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 -- 172.21.15.19:0/226979016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 msgr2=0x7ff0e810a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/226979016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009820 tx=0x7ff0d802a270 comp rx=0 tx=0).stop 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 -- 172.21.15.19:0/226979016 shutdown_connections 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/226979016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e810a530 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 -- 172.21.15.19:0/226979016 >> 172.21.15.19:0/226979016 conn(0x7ff0e8101de0 msgr2=0x7ff0e8104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 -- 172.21.15.19:0/226979016 shutdown_connections 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.815+0000 7ff0ef0de640 1 -- 172.21.15.19:0/226979016 wait complete. 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ef0de640 1 Processor -- start 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ef0de640 1 -- start start 2024-09-12T03:10:18.632 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ef0de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ef0de640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810ba80 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ece7a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ece7a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46750/0 (socket says 172.21.15.19:46750) 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ece7a640 1 -- 172.21.15.19:0/4161215768 learned_addr learned my addr 172.21.15.19:0/4161215768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ece7a640 1 -- 172.21.15.19:0/4161215768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e819dd70 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.816+0000 7ff0ece7a640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80097f0 tx=0x7ff0d80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.817+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0d8037040 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.817+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d8030cc0 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.817+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e819e000 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.817+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0d803a6b0 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.817+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a0cf0 con 0x7ff0e8108140 2024-09-12T03:10:18.633 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.818+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff0d803b3f0 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.818+0000 7ff0e5ffb640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 0x7ff0d003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.818+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e81085a0 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.818+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff0d806c680 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.818+0000 7ff0e7fff640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 0x7ff0d003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.819+0000 7ff0e7fff640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 0x7ff0d003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098c0 tx=0x7ff0dc006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:17.823+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0d803a850 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.101+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7ff0e81a0ec0 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.103+0000 7ff0e5ffb640 1 -- 172.21.15.19:0/4161215768 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7ff0d8044c50 con 0x7ff0e8108140 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 msgr2=0x7ff0d003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 0x7ff0d003b580 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0098c0 tx=0x7ff0dc006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:18.634 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 msgr2=0x7ff0e819d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80097f0 tx=0x7ff0d80327b0 comp rx=0 tx=0).stop 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 shutdown_connections 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0d00390c0 0x7ff0d003b580 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.105+0000 7ff0ef0de640 1 --2- 172.21.15.19:0/4161215768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0e8108140 0x7ff0e819d830 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.106+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 >> 172.21.15.19:0/4161215768 conn(0x7ff0e8101de0 msgr2=0x7ff0e81028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.106+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 shutdown_connections 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:/usr/bin/ceph: stderr 2024-09-12T03:10:18.106+0000 7ff0ef0de640 1 -- 172.21.15.19:0/4161215768 wait complete. 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.635 INFO:teuthology.orchestra.run.smithi019.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: ceph telemetry on 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout:For more information see: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:18.636 INFO:teuthology.orchestra.run.smithi019.stdout:Bootstrap complete. 2024-09-12T03:10:18.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:18 smithi019 ceph-mon[30953]: mgrmap e13: x(active, since 6s) 2024-09-12T03:10:18.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:18 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4161215768' entity='client.admin' 2024-09-12T03:10:18.671 INFO:tasks.cephadm:Fetching config... 2024-09-12T03:10:18.671 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:10:18.671 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T03:10:18.688 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T03:10:18.688 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:10:18.688 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T03:10:18.744 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T03:10:18.744 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:10:18.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T03:10:18.810 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T03:10:18.810 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:10:18.810 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T03:10:18.866 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T03:10:18.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:10:18.942 INFO:teuthology.orchestra.run.smithi019.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:18.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:10:18.996 INFO:teuthology.orchestra.run.smithi121.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:19.009 DEBUG:teuthology.orchestra.run.smithi135:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T03:10:19.050 INFO:teuthology.orchestra.run.smithi135.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQClFgChbJcVQErVR8hQlNxnQVrpQKQcRUt1S+eUmgtZD8UClq3hUOA7I4RMfQgrkJBYGufB24ONLCidKpyOKu/c27NeIQYHhPTM+q4ddID00HyGUYPbpTLCf+uOD8IGgG3QAmRaDI6wvHpBDBoHioDIw63+JuHy3ksrwG0PVdiTnMf30Hl5nL7EIA0hPSjHvo9dau60Yki/6jKcrC1pBE07ww8JPOQzP/LXp2uk7owVrbiIBJjSZnvVW65SWkryw2ls20aSCie38adNe/KTg67af5y1xnuvkk4ilV6Pmk1ZPSbTVjgCPMBujVuM7puFJOyuqNC//DxRchGYc+d1R1VTiHvYOUfQXFA2TYknsqj9vKPxwgRaRK5CDhqtmCmVTZCjMMuIiK1SXqvWXnKga84/8WGqn33D9UxHEdF71GSrhDu72hiIvl/iYw/QB5miSMpB3wRpvbhC4510UumdF/Z+L9iqiiAW/K/nkP3kgnaxH0NnJcSNqkWRiajQaS3wQg0= ceph-45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:19.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T03:10:19.317 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.740+0000 7f4e8381a640 1 -- 172.21.15.19:0/683592763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 msgr2=0x7f4e7c1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.740+0000 7f4e8381a640 1 --2- 172.21.15.19:0/683592763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c1008e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e78009880 tx=0x7f4e7802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.740+0000 7f4e8381a640 1 -- 172.21.15.19:0/683592763 shutdown_connections 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.740+0000 7f4e8381a640 1 --2- 172.21.15.19:0/683592763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c1008e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.740+0000 7f4e8381a640 1 -- 172.21.15.19:0/683592763 >> 172.21.15.19:0/683592763 conn(0x7f4e7c0fbc90 msgr2=0x7f4e7c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.741+0000 7f4e8381a640 1 -- 172.21.15.19:0/683592763 shutdown_connections 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.741+0000 7f4e8381a640 1 -- 172.21.15.19:0/683592763 wait complete. 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.741+0000 7f4e8381a640 1 Processor -- start 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.741+0000 7f4e8381a640 1 -- start start 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e8381a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e8381a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4e7c1065f0 con 0x7f4e7c1004e0 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e815b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e815b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46766/0 (socket says 172.21.15.19:46766) 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e815b6640 1 -- 172.21.15.19:0/2488464062 learned_addr learned my addr 172.21.15.19:0/2488464062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:19.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e815b6640 1 -- 172.21.15.19:0/2488464062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e7c0fe040 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e815b6640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e78004f60 tx=0x7f4e780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e7803f070 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.742+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4e78004d40 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.743+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e7c0fc230 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.743+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e78035030 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.743+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e7c0fc770 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.743+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4e78042070 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.744+0000 7f4e727fc640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 0x7f4e6803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.744+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e7c106d70 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.744+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4e7806d030 con 0x7f4e7c1004e0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.744+0000 7f4e80db5640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 0x7f4e6803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.744+0000 7f4e80db5640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 0x7f4e6803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098c0 tx=0x7f4e6c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:19.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.749+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e78042360 con 0x7f4e7c1004e0 2024-09-12T03:10:19.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.926+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4e7c1902e0 con 0x7f4e7c1004e0 2024-09-12T03:10:19.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.929+0000 7f4e727fc640 1 -- 172.21.15.19:0/2488464062 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4e7802f300 con 0x7f4e7c1004e0 2024-09-12T03:10:19.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.931+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 msgr2=0x7f4e6803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.931+0000 7f4e8381a640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 0x7f4e6803b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0098c0 tx=0x7f4e6c006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.931+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 msgr2=0x7f4e7c0fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.931+0000 7f4e8381a640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e78004f60 tx=0x7f4e780048f0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 shutdown_connections 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4e680390c0 0x7f4e6803b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 --2- 172.21.15.19:0/2488464062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4e7c1004e0 0x7f4e7c0fdb00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 >> 172.21.15.19:0/2488464062 conn(0x7f4e7c0fbc90 msgr2=0x7f4e7c06a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 shutdown_connections 2024-09-12T03:10:19.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:19.932+0000 7f4e8381a640 1 -- 172.21.15.19:0/2488464062 wait complete. 2024-09-12T03:10:20.977 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T03:10:20.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2488464062' entity='client.admin' 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi019", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:21.087 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:21.430 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.933+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/2066214774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 msgr2=0x7f8bac100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.933+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/2066214774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac100940 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009880 tx=0x7f8ba402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.933+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/2066214774 shutdown_connections 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.933+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/2066214774 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac100940 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.933+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/2066214774 >> 172.21.15.19:0/2066214774 conn(0x7f8bac0fbcd0 msgr2=0x7f8bac0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/2066214774 shutdown_connections 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/2066214774 wait complete. 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 Processor -- start 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 -- start start 2024-09-12T03:10:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb3cf0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac1065f0 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb1a8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb1a8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36016/0 (socket says 172.21.15.19:36016) 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.934+0000 7f8bb1a8c640 1 -- 172.21.15.19:0/3170447247 learned_addr learned my addr 172.21.15.19:0/3170447247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8bb1a8c640 1 -- 172.21.15.19:0/3170447247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac1972f0 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8bb1a8c640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4004f60 tx=0x7f8ba40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba403f070 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac197580 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ba4004d40 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ba4035030 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.935+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac19a230 con 0x7f8bac100540 2024-09-12T03:10:21.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.936+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8ba4042070 con 0x7f8bac100540 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.936+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac106d70 con 0x7f8bac100540 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.936+0000 7f8baaffd640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 0x7f8b9403b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.936+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ba406cd10 con 0x7f8bac100540 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.937+0000 7f8bb128b640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 0x7f8b9403b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.937+0000 7f8bb128b640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 0x7f8b9403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00098c0 tx=0x7f8ba0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:21.941+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ba4039350 con 0x7f8bac100540 2024-09-12T03:10:22.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.122+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f8bac02d940 con 0x7f8b940390c0 2024-09-12T03:10:22.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.125+0000 7f8baaffd640 1 -- 172.21.15.19:0/3170447247 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8bac02d940 con 0x7f8b940390c0 2024-09-12T03:10:22.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 msgr2=0x7f8b9403b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:22.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 0x7f8b9403b580 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ba00098c0 tx=0x7f8ba0006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:22.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 msgr2=0x7f8bac196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:22.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4004f60 tx=0x7f8ba40048f0 comp rx=0 tx=0).stop 2024-09-12T03:10:22.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 shutdown_connections 2024-09-12T03:10:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8b940390c0 0x7f8b9403b580 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 --2- 172.21.15.19:0/3170447247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bac100540 0x7f8bac196db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 >> 172.21.15.19:0/3170447247 conn(0x7f8bac0fbcd0 msgr2=0x7f8bac069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 shutdown_connections 2024-09-12T03:10:22.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:22.128+0000 7f8bb3cf0640 1 -- 172.21.15.19:0/3170447247 wait complete. 2024-09-12T03:10:23.348 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi121 2024-09-12T03:10:23.348 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:10:23.348 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:10:23.368 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:10:23.368 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:23.426 INFO:tasks.cephadm:Adding host smithi121 to orchestrator... 2024-09-12T03:10:23.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch host add smithi121 2024-09-12T03:10:23.678 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:24.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 -- 172.21.15.19:0/2095834133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 msgr2=0x7f9aa8107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:24.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 --2- 172.21.15.19:0/2095834133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa8107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009880 tx=0x7f9aa402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:24.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 -- 172.21.15.19:0/2095834133 shutdown_connections 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 --2- 172.21.15.19:0/2095834133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa8107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 -- 172.21.15.19:0/2095834133 >> 172.21.15.19:0/2095834133 conn(0x7f9aa80fd290 msgr2=0x7f9aa80ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 -- 172.21.15.19:0/2095834133 shutdown_connections 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.074+0000 7f9ab0478640 1 -- 172.21.15.19:0/2095834133 wait complete. 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.075+0000 7f9ab0478640 1 Processor -- start 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.075+0000 7f9ab0478640 1 -- start start 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.075+0000 7f9ab0478640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9ab0478640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9aa810bc90 con 0x7f9aa8107660 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9aae214640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9aae214640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36038/0 (socket says 172.21.15.19:36038) 2024-09-12T03:10:24.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9aae214640 1 -- 172.21.15.19:0/3815562842 learned_addr learned my addr 172.21.15.19:0/3815562842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9aae214640 1 -- 172.21.15.19:0/3815562842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aa8197600 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9aae214640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009230 tx=0x7f9aa4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa403f070 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aa8197890 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9aa403bc80 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa4035030 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.076+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aa819b990 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.077+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f9aa4042070 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.077+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aa8102cd0 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.077+0000 7f9a9f7fe640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 0x7f9a9403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.077+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9aa406ccc0 con 0x7f9aa8107660 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.077+0000 7f9aada13640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 0x7f9a9403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:24.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.078+0000 7f9aada13640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 0x7f9a9403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a980098c0 tx=0x7f9a98006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:24.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.083+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9aa404c350 con 0x7f9aa8107660 2024-09-12T03:10:24.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:24.264+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}) -- 0x7f9aa8198420 con 0x7f9a94039060 2024-09-12T03:10:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:24 smithi019 ceph-mon[30953]: Updating smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:10:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:24 smithi019 ceph-mon[30953]: Updating smithi019:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:24 smithi019 ceph-mon[30953]: Updating smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:10:25.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:25 smithi019 ceph-mon[30953]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi121", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:26.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:26 smithi019 ceph-mon[30953]: Deploying cephadm binary to smithi121 2024-09-12T03:10:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.219+0000 7f9a9f7fe640 1 -- 172.21.15.19:0/3815562842 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f9aa8198420 con 0x7f9a94039060 2024-09-12T03:10:28.220 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi121' with addr '172.21.15.121' 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 msgr2=0x7f9a9403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 0x7f9a9403b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a980098c0 tx=0x7f9a98006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 msgr2=0x7f9aa81970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009230 tx=0x7f9aa4003d40 comp rx=0 tx=0).stop 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 shutdown_connections 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9a94039060 0x7f9a9403b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 --2- 172.21.15.19:0/3815562842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9aa8107660 0x7f9aa81970c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.221+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 >> 172.21.15.19:0/3815562842 conn(0x7f9aa80fd290 msgr2=0x7f9aa80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.222+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 shutdown_connections 2024-09-12T03:10:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:28.222+0000 7f9ab0478640 1 -- 172.21.15.19:0/3815562842 wait complete. 2024-09-12T03:10:28.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:10:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:29.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:29 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:29.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:29 smithi019 ceph-mon[30953]: Added host smithi121 2024-09-12T03:10:29.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:29 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:29.522 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:29 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:29.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:29 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- 172.21.15.19:0/4199144537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 msgr2=0x7f4cc4102bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/4199144537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4102bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009c20 tx=0x7f4cb402ab20 comp rx=0 tx=0).stop 2024-09-12T03:10:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- 172.21.15.19:0/4199144537 shutdown_connections 2024-09-12T03:10:29.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/4199144537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4102bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- 172.21.15.19:0/4199144537 >> 172.21.15.19:0/4199144537 conn(0x7f4cc40fc190 msgr2=0x7f4cc40fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- 172.21.15.19:0/4199144537 shutdown_connections 2024-09-12T03:10:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- 172.21.15.19:0/4199144537 wait complete. 2024-09-12T03:10:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 Processor -- start 2024-09-12T03:10:29.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.529+0000 7f4ccbceb640 1 -- start start 2024-09-12T03:10:29.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4ccbceb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:29.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4ccbceb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc410bcd0 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc9a87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc9a87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36054/0 (socket says 172.21.15.19:36054) 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc9a87640 1 -- 172.21.15.19:0/3819420034 learned_addr learned my addr 172.21.15.19:0/3819420034 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc9a87640 1 -- 172.21.15.19:0/3819420034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc41976d0 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc9a87640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4030040 tx=0x7f4cb402eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cb4047070 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb40368e0 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc4197960 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cb4042420 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.530+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc4197dc0 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.531+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4cb404a070 con 0x7f4cc41027f0 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.531+0000 7f4cc2ffd640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 0x7f4cac03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:29.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.531+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4cb403d030 con 0x7f4cc41027f0 2024-09-12T03:10:29.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.532+0000 7f4cc9286640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 0x7f4cac03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:29.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.532+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc406a730 con 0x7f4cc41027f0 2024-09-12T03:10:29.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.532+0000 7f4cc9286640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 0x7f4cac03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80098c0 tx=0x7f4cb8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:29.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.536+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cb4051020 con 0x7f4cc41027f0 2024-09-12T03:10:29.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.716+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cc4198580 con 0x7f4cac039060 2024-09-12T03:10:29.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.717+0000 7f4cc2ffd640 1 -- 172.21.15.19:0/3819420034 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f4cc4198580 con 0x7f4cac039060 2024-09-12T03:10:29.717 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:29.718 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}] 2024-09-12T03:10:29.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 msgr2=0x7f4cac03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 0x7f4cac03b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80098c0 tx=0x7f4cb8006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 msgr2=0x7f4cc4197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4030040 tx=0x7f4cb402eb50 comp rx=0 tx=0).stop 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 shutdown_connections 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cac039060 0x7f4cac03b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 --2- 172.21.15.19:0/3819420034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cc41027f0 0x7f4cc4197190 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 >> 172.21.15.19:0/3819420034 conn(0x7f4cc40fc190 msgr2=0x7f4cc41055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 shutdown_connections 2024-09-12T03:10:29.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:29.719+0000 7f4ccbceb640 1 -- 172.21.15.19:0/3819420034 wait complete. 2024-09-12T03:10:30.212 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi135 2024-09-12T03:10:30.212 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:10:30.212 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.conf 2024-09-12T03:10:30.232 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:10:30.232 DEBUG:teuthology.orchestra.run.smithi135:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:30.290 INFO:tasks.cephadm:Adding host smithi135 to orchestrator... 2024-09-12T03:10:30.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch host add smithi135 2024-09-12T03:10:30.539 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:30.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:30.650 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:30 smithi019 ceph-mon[30953]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:10:30.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.943+0000 7ff9dab49640 1 -- 172.21.15.19:0/2876739169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 msgr2=0x7ff9d4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.943+0000 7ff9dab49640 1 --2- 172.21.15.19:0/2876739169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009810 tx=0x7ff9c402a540 comp rx=0 tx=0).stop 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- 172.21.15.19:0/2876739169 shutdown_connections 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 --2- 172.21.15.19:0/2876739169 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- 172.21.15.19:0/2876739169 >> 172.21.15.19:0/2876739169 conn(0x7ff9d4077310 msgr2=0x7ff9d40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- 172.21.15.19:0/2876739169 shutdown_connections 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- 172.21.15.19:0/2876739169 wait complete. 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 Processor -- start 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- start start 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:30.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9dab49640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d410ab70 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9d88e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.944+0000 7ff9d88e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57554/0 (socket says 172.21.15.19:57554) 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d88e5640 1 -- 172.21.15.19:0/230347528 learned_addr learned my addr 172.21.15.19:0/230347528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d88e5640 1 -- 172.21.15.19:0/230347528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d4197650 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d88e5640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40045e0 tx=0x7ff9c4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c4030e70 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9c40318e0 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9c40049c0 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d41978e0 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.945+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d419b9e0 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.946+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff9c4039040 con 0x7ff9d4106540 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.946+0000 7ff9d1ffb640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 0x7ff9bc03b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:30.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.946+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d4101bb0 con 0x7ff9d4106540 2024-09-12T03:10:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.946+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff9c406d260 con 0x7ff9d4106540 2024-09-12T03:10:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.946+0000 7ff9d3fff640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 0x7ff9bc03b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.947+0000 7ff9d3fff640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 0x7ff9bc03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80098c0 tx=0x7ff9c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:30.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:30.951+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff9c4040030 con 0x7ff9d4106540 2024-09-12T03:10:31.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:31.128+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}) -- 0x7ff9d4106a00 con 0x7ff9bc0390c0 2024-09-12T03:10:31.866 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:31 smithi019 ceph-mon[30953]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi135", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi121", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:33.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: Deploying cephadm binary to smithi135 2024-09-12T03:10:33.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-12T03:10:33.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: Updating smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:10:33.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: Updating smithi121:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:33.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: Updating smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:10:33.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:33.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:33.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:34.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:34 smithi019 ceph-mon[30953]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:35.025 INFO:teuthology.orchestra.run.smithi019.stdout:Added host 'smithi135' with addr '172.21.15.135' 2024-09-12T03:10:35.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.025+0000 7ff9d1ffb640 1 -- 172.21.15.19:0/230347528 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff9d4106a00 con 0x7ff9bc0390c0 2024-09-12T03:10:35.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 msgr2=0x7ff9bc03b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:35.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 0x7ff9bc03b580 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff9c80098c0 tx=0x7ff9c8006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 msgr2=0x7ff9d4197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40045e0 tx=0x7ff9c4004610 comp rx=0 tx=0).stop 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 shutdown_connections 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff9bc0390c0 0x7ff9bc03b580 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 --2- 172.21.15.19:0/230347528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff9d4106540 0x7ff9d4197110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:35.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.027+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 >> 172.21.15.19:0/230347528 conn(0x7ff9d4077310 msgr2=0x7ff9d40696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:35.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.028+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 shutdown_connections 2024-09-12T03:10:35.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:35.028+0000 7ff9dab49640 1 -- 172.21.15.19:0/230347528 wait complete. 2024-09-12T03:10:35.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T03:10:35.883 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:36.250 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:36.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:36 smithi019 ceph-mon[30953]: Added host smithi135 2024-09-12T03:10:36.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:36.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:36.251 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 -- 172.21.15.19:0/1827558703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 msgr2=0x7f52b0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:36.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/1827558703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0107a40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009880 tx=0x7f52a402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 -- 172.21.15.19:0/1827558703 shutdown_connections 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/1827558703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0107a40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 -- 172.21.15.19:0/1827558703 >> 172.21.15.19:0/1827558703 conn(0x7f52b00fd290 msgr2=0x7f52b00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 -- 172.21.15.19:0/1827558703 shutdown_connections 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.333+0000 7f52b6e3e640 1 -- 172.21.15.19:0/1827558703 wait complete. 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b6e3e640 1 Processor -- start 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b6e3e640 1 -- start start 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b6e3e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b6e3e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f52b010bc90 con 0x7f52b0107660 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b4bda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b4bda640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57578/0 (socket says 172.21.15.19:57578) 2024-09-12T03:10:36.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b4bda640 1 -- 172.21.15.19:0/593934297 learned_addr learned my addr 172.21.15.19:0/593934297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.334+0000 7f52b4bda640 1 -- 172.21.15.19:0/593934297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b01976c0 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52b4bda640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009230 tx=0x7f52a4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a403f070 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52a403bc80 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b0197950 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52a4035030 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.335+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b0197db0 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.336+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f52a4042070 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.336+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b0102cd0 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.336+0000 7f52adffb640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 0x7f52a003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.336+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f52a406cdd0 con 0x7f52b0107660 2024-09-12T03:10:36.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.336+0000 7f52affff640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 0x7f52a003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.337+0000 7f52affff640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 0x7f52a003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098c0 tx=0x7f529c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:36.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.341+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52a4070030 con 0x7f52b0107660 2024-09-12T03:10:36.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.522+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f52b0198540 con 0x7f52a00390c0 2024-09-12T03:10:36.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.524+0000 7f52adffb640 1 -- 172.21.15.19:0/593934297 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f52b0198540 con 0x7f52a00390c0 2024-09-12T03:10:36.524 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:10:36.524 INFO:teuthology.orchestra.run.smithi019.stdout:[{"addr": "172.21.15.19", "hostname": "smithi019", "labels": [], "status": ""}, {"addr": "172.21.15.121", "hostname": "smithi121", "labels": [], "status": ""}, {"addr": "172.21.15.135", "hostname": "smithi135", "labels": [], "status": ""}] 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.526+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 msgr2=0x7f52a003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.526+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 0x7f52a003b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f529c0098c0 tx=0x7f529c006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.526+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 msgr2=0x7f52b0197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.526+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009230 tx=0x7f52a4003d40 comp rx=0 tx=0).stop 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 shutdown_connections 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f52a00390c0 0x7f52a003b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 --2- 172.21.15.19:0/593934297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f52b0107660 0x7f52b0197180 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:36.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 >> 172.21.15.19:0/593934297 conn(0x7f52b00fd290 msgr2=0x7f52b00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:36.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 shutdown_connections 2024-09-12T03:10:36.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:36.527+0000 7f52b6e3e640 1 -- 172.21.15.19:0/593934297 wait complete. 2024-09-12T03:10:37.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:37 smithi019 ceph-mon[30953]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:37.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:37.093 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T03:10:37.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T03:10:37.344 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 -- 172.21.15.19:0/2719195840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 msgr2=0x7f57e0107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 --2- 172.21.15.19:0/2719195840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e0107a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009880 tx=0x7f57d002a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 -- 172.21.15.19:0/2719195840 shutdown_connections 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 --2- 172.21.15.19:0/2719195840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e0107a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 -- 172.21.15.19:0/2719195840 >> 172.21.15.19:0/2719195840 conn(0x7f57e00fd290 msgr2=0x7f57e00ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 -- 172.21.15.19:0/2719195840 shutdown_connections 2024-09-12T03:10:37.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.789+0000 7f57e6035640 1 -- 172.21.15.19:0/2719195840 wait complete. 2024-09-12T03:10:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57e6035640 1 Processor -- start 2024-09-12T03:10:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57e6035640 1 -- start start 2024-09-12T03:10:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57e6035640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:37.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57e6035640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57e010bc90 con 0x7f57e0107660 2024-09-12T03:10:37.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57590/0 (socket says 172.21.15.19:57590) 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.790+0000 7f57df7fe640 1 -- 172.21.15.19:0/464113117 learned_addr learned my addr 172.21.15.19:0/464113117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57df7fe640 1 -- 172.21.15.19:0/464113117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e0197600 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57df7fe640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009230 tx=0x7f57d0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d003f070 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d003bc80 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e0197890 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f57d0035030 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.791+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e019b990 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.792+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f57d0042070 con 0x7f57e0107660 2024-09-12T03:10:37.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.792+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e0102cd0 con 0x7f57e0107660 2024-09-12T03:10:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.792+0000 7f57dcff9640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 0x7f57c803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.792+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f57d006ce20 con 0x7f57e0107660 2024-09-12T03:10:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.793+0000 7f57deffd640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 0x7f57c803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:37.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.793+0000 7f57deffd640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 0x7f57c803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098c0 tx=0x7f57d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:37.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.797+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57d004c350 con 0x7f57e0107660 2024-09-12T03:10:37.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:37.976+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f57e0107ac0 con 0x7f57e0107660 2024-09-12T03:10:38.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.029+0000 7f57dcff9640 1 -- 172.21.15.19:0/464113117 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f57d00314c0 con 0x7f57e0107660 2024-09-12T03:10:38.029 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-09-12T03:10:38.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 msgr2=0x7f57c803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 0x7f57c803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f57d40098c0 tx=0x7f57d4006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 msgr2=0x7f57e01970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:38.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f57d0009230 tx=0x7f57d0003d40 comp rx=0 tx=0).stop 2024-09-12T03:10:38.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:38 smithi019 ceph-mon[30953]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:10:38.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/464113117' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:10:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 shutdown_connections 2024-09-12T03:10:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c8039060 0x7f57c803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 --2- 172.21.15.19:0/464113117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57e0107660 0x7f57e01970c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.032+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 >> 172.21.15.19:0/464113117 conn(0x7f57e00fd290 msgr2=0x7f57e00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:38.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.033+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 shutdown_connections 2024-09-12T03:10:38.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:10:38.033+0000 7f57e6035640 1 -- 172.21.15.19:0/464113117 wait complete. 2024-09-12T03:10:38.565 INFO:tasks.cephadm:Adding mon.a on smithi019 2024-09-12T03:10:38.566 INFO:tasks.cephadm:Adding mon.b on smithi121 2024-09-12T03:10:38.566 INFO:tasks.cephadm:Adding mon.c on smithi135 2024-09-12T03:10:38.566 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c' 2024-09-12T03:10:39.135 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:10:39.181 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T03:10:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/464113117' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:10:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:39.321 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:39.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:39.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:39.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:39.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:39.322 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.680+0000 7f6e992d7640 1 -- 172.21.15.135:0/3363200611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 msgr2=0x7f6e940ffd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.680+0000 7f6e992d7640 1 --2- 172.21.15.135:0/3363200611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e940ffd00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009880 tx=0x7f6e8402a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.684 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.680+0000 7f6e992d7640 1 -- 172.21.15.135:0/3363200611 shutdown_connections 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.680+0000 7f6e992d7640 1 --2- 172.21.15.135:0/3363200611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e940ffd00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.680+0000 7f6e992d7640 1 -- 172.21.15.135:0/3363200611 >> 172.21.15.135:0/3363200611 conn(0x7f6e940fb450 msgr2=0x7f6e940fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.681+0000 7f6e992d7640 1 -- 172.21.15.135:0/3363200611 shutdown_connections 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.681+0000 7f6e992d7640 1 -- 172.21.15.135:0/3363200611 wait complete. 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.681+0000 7f6e992d7640 1 Processor -- start 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e992d7640 1 -- start start 2024-09-12T03:10:39.685 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e992d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e992d7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941076e0 con 0x7f6e940ff900 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e92ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e92ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.135:54492/0 (socket says 172.21.15.135:54492) 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e92ffd640 1 -- 172.21.15.135:0/4229531471 learned_addr learned my addr 172.21.15.135:0/4229531471 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e92ffd640 1 -- 172.21.15.135:0/4229531471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e941026a0 con 0x7f6e940ff900 2024-09-12T03:10:39.686 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.682+0000 7f6e92ffd640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e84004f60 tx=0x7f6e840048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:39.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.683+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e8403f070 con 0x7f6e940ff900 2024-09-12T03:10:39.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.683+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e94100890 con 0x7f6e940ff900 2024-09-12T03:10:39.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.683+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e84004d40 con 0x7f6e940ff900 2024-09-12T03:10:39.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.683+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e84035030 con 0x7f6e940ff900 2024-09-12T03:10:39.687 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.683+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e94100dd0 con 0x7f6e940ff900 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.684+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f6e84042070 con 0x7f6e940ff900 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.684+0000 7f6e7bfff640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 0x7f6e7c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.684+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e94107e60 con 0x7f6e940ff900 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.684+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6e84003ae0 con 0x7f6e940ff900 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.684+0000 7f6e927fc640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 0x7f6e7c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:39.689 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.685+0000 7f6e927fc640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 0x7f6e7c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e880098c0 tx=0x7f6e88006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:39.692 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.691+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e84042360 con 0x7f6e940ff900 2024-09-12T03:10:39.890 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.889+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}) -- 0x7f6e94190220 con 0x7f6e7c039060 2024-09-12T03:10:39.896 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.895+0000 7f6e7bfff640 1 -- 172.21.15.135:0/4229531471 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6e94190220 con 0x7f6e7c039060 2024-09-12T03:10:39.896 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mon update... 2024-09-12T03:10:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 msgr2=0x7f6e7c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 0x7f6e7c03b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6e880098c0 tx=0x7f6e88006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 msgr2=0x7f6e94102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:39.898 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e84004f60 tx=0x7f6e840048f0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 shutdown_connections 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6e7c039060 0x7f6e7c03b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 --2- 172.21.15.135:0/4229531471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6e940ff900 0x7f6e94102160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 >> 172.21.15.135:0/4229531471 conn(0x7f6e940fb450 msgr2=0x7f6e940fbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 shutdown_connections 2024-09-12T03:10:39.899 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:39.897+0000 7f6e992d7640 1 -- 172.21.15.135:0/4229531471 wait complete. 2024-09-12T03:10:40.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:10:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:40.416 DEBUG:teuthology.orchestra.run.smithi121:mon.b> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T03:10:40.419 DEBUG:teuthology.orchestra.run.smithi135:mon.c> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T03:10:40.422 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:10:40.422 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:10:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:10:41.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:41.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:10:41.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:41.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c;count:3 2024-09-12T03:10:41.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:41 smithi019 ceph-mon[30953]: Deploying daemon mon.c on smithi135 2024-09-12T03:10:42.060 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 -- 172.21.15.135:0/4274386778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 msgr2=0x7ff0f8107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/4274386778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f8107a30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009880 tx=0x7ff0e802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 -- 172.21.15.135:0/4274386778 shutdown_connections 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/4274386778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f8107a30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 -- 172.21.15.135:0/4274386778 >> 172.21.15.135:0/4274386778 conn(0x7ff0f80fd260 msgr2=0x7ff0f80ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.040 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 -- 172.21.15.135:0/4274386778 shutdown_connections 2024-09-12T03:10:43.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.038+0000 7ff0fdc26640 1 -- 172.21.15.135:0/4274386778 wait complete. 2024-09-12T03:10:43.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.039+0000 7ff0fdc26640 1 Processor -- start 2024-09-12T03:10:43.041 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.039+0000 7ff0fdc26640 1 -- start start 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.039+0000 7ff0fdc26640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.039+0000 7ff0fdc26640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f810bc80 con 0x7ff0f8107650 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.135:54504/0 (socket says 172.21.15.135:54504) 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f77fe640 1 -- 172.21.15.135:0/1094392249 learned_addr learned my addr 172.21.15.135:0/1094392249 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f77fe640 1 -- 172.21.15.135:0/1094392249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f80763e0 con 0x7ff0f8107650 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f77fe640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009230 tx=0x7ff0e8003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.042 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.040+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e803f070 con 0x7ff0f8107650 2024-09-12T03:10:43.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.041+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f8074a30 con 0x7ff0f8107650 2024-09-12T03:10:43.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.041+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0e803bc80 con 0x7ff0f8107650 2024-09-12T03:10:43.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.041+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0e8035030 con 0x7ff0f8107650 2024-09-12T03:10:43.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.041+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f8074ef0 con 0x7ff0f8107650 2024-09-12T03:10:43.043 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.041+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f8102cc0 con 0x7ff0f8107650 2024-09-12T03:10:43.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.042+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff0e8042070 con 0x7ff0f8107650 2024-09-12T03:10:43.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.042+0000 7ff0f4ff9640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 0x7ff0e003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:43.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.042+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff0e806cee0 con 0x7ff0f8107650 2024-09-12T03:10:43.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.042+0000 7ff0f6ffd640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 0x7ff0e003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:43.044 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.042+0000 7ff0f6ffd640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 0x7ff0e003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0098c0 tx=0x7ff0ec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:43.048 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.047+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0e804c350 con 0x7ff0f8107650 2024-09-12T03:10:43.305 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.304+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff0f8075a20 con 0x7ff0f8107650 2024-09-12T03:10:43.306 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.305+0000 7ff0f4ff9640 1 -- 172.21.15.135:0/1094392249 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7ff0e80314c0 con 0x7ff0f8107650 2024-09-12T03:10:43.306 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:10:43.306 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":1,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:09:09.637034Z","created":"2024-09-12T03:09:09.637034Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T03:10:43.307 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 1 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.308+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 msgr2=0x7ff0e003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.308+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 0x7ff0e003b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec0098c0 tx=0x7ff0ec006db0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.308+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 msgr2=0x7ff0f80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.308+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8009230 tx=0x7ff0e8003d40 comp rx=0 tx=0).stop 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 shutdown_connections 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0e0039060 0x7ff0e003b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 --2- 172.21.15.135:0/1094392249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0f8107650 0x7ff0f80744f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 >> 172.21.15.135:0/1094392249 conn(0x7ff0f80fd260 msgr2=0x7ff0f80fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 shutdown_connections 2024-09-12T03:10:43.310 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:43.309+0000 7ff0fdc26640 1 -- 172.21.15.135:0/1094392249 wait complete. 2024-09-12T03:10:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:43 smithi019 ceph-mon[30953]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:44.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:44 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/1094392249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:44 smithi135 systemd[1]: Started Ceph mon.c for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:10:45.208 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:10:45.209 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:10:45.462 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:10:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 -- 172.21.15.135:0/1818939571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 msgr2=0x7f142c005580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 --2- 172.21.15.135:0/1818939571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f142c005580 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f143c014d10 tx=0x7f143c02c570 comp rx=0 tx=0).stop 2024-09-12T03:10:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 -- 172.21.15.135:0/1818939571 shutdown_connections 2024-09-12T03:10:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 --2- 172.21.15.135:0/1818939571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f142c005580 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.459 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 -- 172.21.15.135:0/1818939571 >> 172.21.15.135:0/1818939571 conn(0x7f14400fc220 msgr2=0x7f14400fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 -- 172.21.15.135:0/1818939571 shutdown_connections 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.457+0000 7f14480a6640 1 -- 172.21.15.135:0/1818939571 wait complete. 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.458+0000 7f14480a6640 1 Processor -- start 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.458+0000 7f14480a6640 1 -- start start 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f14480a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f14480a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f14480a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1440108670 con 0x7f1440106630 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f14480a6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14401084f0 con 0x7f1440197790 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445e42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:46472/0 (socket says 172.21.15.135:46472) 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 -- 172.21.15.135:0/187990449 learned_addr learned my addr 172.21.15.135:0/187990449 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445e42640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.135:56960/0 (socket says 172.21.15.135:56960) 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 -- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 msgr2=0x7f144019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 -- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 msgr2=0x7f144019bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445641640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445e42640 1 -- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 msgr2=0x7f144019bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445e42640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.459+0000 7f1445e42640 1 -- 172.21.15.135:0/187990449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1440197db0 con 0x7f1440106630 2024-09-12T03:10:49.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f1445e42640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f143c035040 tx=0x7f143c0050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f143c004660 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f143c03f470 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f143c012450 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144019c060 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.460+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144019c620 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.461+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f143c046020 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.461+0000 7f1436ffd640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 0x7f142403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.461+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1440101ca0 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.461+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f143c06e5b0 con 0x7f1440106630 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.461+0000 7f1445641640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 0x7f142403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:49.463 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.462+0000 7f1445641640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 0x7f142403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1430009100 tx=0x7f1430008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:49.466 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.466+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f143c04d030 con 0x7f1440106630 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: mon.a calling monitor election 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: mon.c calling monitor election 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:49.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:10:49.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: monmap epoch 2 2024-09-12T03:10:49.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:49.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: last_changed 2024-09-12T03:10:44.386781+0000 2024-09-12T03:10:49.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:49.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: min_mon_release 19 (squid) 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: election_strategy: 1 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: fsmap 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: mgrmap e13: x(active, since 38s) 2024-09-12T03:10:49.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: overall HEALTH_OK 2024-09-12T03:10:49.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:10:49.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:49.701 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.701+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f144019c900 con 0x7f1440106630 2024-09-12T03:10:49.702 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:49.702 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.702 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: mon.a calling monitor election 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: mon.c calling monitor election 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: monmap epoch 2 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: last_changed 2024-09-12T03:10:44.386781+0000 2024-09-12T03:10:49.703 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: min_mon_release 19 (squid) 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: election_strategy: 1 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: fsmap 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: mgrmap e13: x(active, since 38s) 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: overall HEALTH_OK 2024-09-12T03:10:49.704 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.705 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.705 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:49.705 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:10:49.705 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:49.705 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.702+0000 7f1436ffd640 1 -- 172.21.15.135:0/187990449 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f143c03b580 con 0x7f1440106630 2024-09-12T03:10:49.705 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:10:49.706 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:10:44.386781Z","created":"2024-09-12T03:09:09.637034Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:10:49.706 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 2 2024-09-12T03:10:49.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 msgr2=0x7f142403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 0x7f142403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1430009100 tx=0x7f1430008040 comp rx=0 tx=0).stop 2024-09-12T03:10:49.706 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 msgr2=0x7f1440197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f143c035040 tx=0x7f143c0050e0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 shutdown_connections 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1440197790 0x7f144019bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1424038e50 0x7f142403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 --2- 172.21.15.135:0/187990449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1440106630 0x7f1440197250 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 >> 172.21.15.135:0/187990449 conn(0x7f14400fc220 msgr2=0x7f144006a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 shutdown_connections 2024-09-12T03:10:49.707 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:49.704+0000 7f14480a6640 1 -- 172.21.15.135:0/187990449 wait complete. 2024-09-12T03:10:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:50 smithi135 ceph-mon[35474]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:50 smithi135 ceph-mon[35474]: Deploying daemon mon.b on smithi121 2024-09-12T03:10:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:50 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/187990449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:50 smithi019 ceph-mon[30953]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:50 smithi019 ceph-mon[30953]: Deploying daemon mon.b on smithi121 2024-09-12T03:10:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:50 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/187990449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:51.283 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:10:51.283 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:10:51.538 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- 172.21.15.135:0/1484659985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec106650 msgr2=0x7f4cec106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 --2- 172.21.15.135:0/1484659985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec106650 0x7f4cec106a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009880 tx=0x7f4cdc02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- 172.21.15.135:0/1484659985 shutdown_connections 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 --2- 172.21.15.135:0/1484659985 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec106650 0x7f4cec106a30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- 172.21.15.135:0/1484659985 >> 172.21.15.135:0/1484659985 conn(0x7f4cec0fc280 msgr2=0x7f4cec0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- 172.21.15.135:0/1484659985 shutdown_connections 2024-09-12T03:10:51.910 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- 172.21.15.135:0/1484659985 wait complete. 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 Processor -- start 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.910+0000 7f4cf243c640 1 -- start start 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cf243c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cf243c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cf243c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10ac80 con 0x7f4cec197980 2024-09-12T03:10:51.912 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cf243c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec10ab00 con 0x7f4cec106650 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cebfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.135:56980/0 (socket says 172.21.15.135:56980) 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 -- 172.21.15.135:0/3779453071 learned_addr learned my addr 172.21.15.135:0/3779453071 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cebfff640 1 -- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 msgr2=0x7f4cec197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cebfff640 1 -- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 msgr2=0x7f4cec197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cebfff640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4cebfff640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 -- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 msgr2=0x7f4cec197440 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.911+0000 7f4ceb7fe640 1 -- 172.21.15.135:0/3779453071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec197fa0 con 0x7f4cec197980 2024-09-12T03:10:51.913 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4ceb7fe640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b4a0 tx=0x7f4ce000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4ce0010610 con 0x7f4cec197980 2024-09-12T03:10:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ce0010c90 con 0x7f4cec197980 2024-09-12T03:10:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec19c250 con 0x7f4cec197980 2024-09-12T03:10:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4ce0017e60 con 0x7f4cec197980 2024-09-12T03:10:51.914 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.912+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec19c790 con 0x7f4cec197980 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.913+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f4ce0017070 con 0x7f4cec197980 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.913+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cec101cc0 con 0x7f4cec197980 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.913+0000 7f4ce97fa640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 0x7f4cd403b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.913+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4ce000d040 con 0x7f4cec197980 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.913+0000 7f4cebfff640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 0x7f4cd403b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:51.915 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.914+0000 7f4cebfff640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 0x7f4cd403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0314e0 tx=0x7f4cdc0058b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:51.918 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:51.918+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4ce001e020 con 0x7f4cec197980 2024-09-12T03:10:52.145 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.145+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4cec19ca50 con 0x7f4cec197980 2024-09-12T03:10:52.146 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.146+0000 7f4ce97fa640 1 -- 172.21.15.135:0/3779453071 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4ce0018e90 con 0x7f4cec197980 2024-09-12T03:10:52.146 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:10:52.146 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":2,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:10:44.386781Z","created":"2024-09-12T03:09:09.637034Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T03:10:52.146 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 2 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 msgr2=0x7f4cd403b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 0x7f4cd403b2c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc0314e0 tx=0x7f4cdc0058b0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 msgr2=0x7f4cec19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000b4a0 tx=0x7f4ce000b960 comp rx=0 tx=0).stop 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 shutdown_connections 2024-09-12T03:10:52.148 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4cd4038e00 0x7f4cd403b2c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4cec197980 0x7f4cec19bd10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 --2- 172.21.15.135:0/3779453071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4cec106650 0x7f4cec197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 >> 172.21.15.135:0/3779453071 conn(0x7f4cec0fc280 msgr2=0x7f4cec104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 shutdown_connections 2024-09-12T03:10:52.149 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:52.148+0000 7f4cf243c640 1 -- 172.21.15.135:0/3779453071 wait complete. 2024-09-12T03:10:52.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:52 smithi135 ceph-mon[35474]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:52.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:52 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/3779453071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:52 smithi019 ceph-mon[30953]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:52.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:52 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/3779453071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.412 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: load: jerasure load: lrc 2024-09-12T03:10:53.787 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T03:10:53.787 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: RocksDB version: 7.9.2 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Git sha 0 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: DB SUMMARY 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: DB Session ID: BKXBC7SNS8DCAWME46AW 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: CURRENT file: CURRENT 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T03:10:53.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.error_if_exists: 0 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.create_if_missing: 0 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.paranoid_checks: 1 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.env: 0x55c139626940 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.info_log: 0x55c13ad3b6e0 2024-09-12T03:10:53.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.statistics: (nil) 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.use_fsync: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_log_file_size: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_fallocate: 1 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.use_direct_reads: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T03:10:53.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.db_log_dir: 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.wal_dir: 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T03:10:53.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T03:10:53.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.write_buffer_manager: 0x55c13ad4adc0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T03:10:53.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.unordered_write: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.row_cache: None 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.wal_filter: None 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.two_write_queues: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.wal_compression: 0 2024-09-12T03:10:53.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.atomic_flush: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.log_readahead_size: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_background_jobs: 2 2024-09-12T03:10:53.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_background_compactions: -1 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_subcompactions: 1 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_open_files: -1 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T03:10:53.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_background_flushes: -1 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Compression algorithms supported: 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kZSTD supported: 0 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kXpressCompression supported: 0 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kBZip2Compression supported: 0 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kLZ4Compression supported: 1 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kZlibCompression supported: 1 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: kSnappyCompression supported: 1 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T03:10:53.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.merge_operator: 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_filter: None 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c13ad3b320) 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout: cache_index_and_filter_blocks: 1 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T03:10:53.910 INFO:journalctl@ceph.mon.b.smithi121.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: pin_top_level_index_and_filter: 1 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_type: 0 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: data_block_index_type: 0 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_shortening: 1 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: checksum: 4 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: no_block_cache: 0 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache: 0x55c13ad2f1f0 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_name: BinnedLRUCache 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_options: 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: capacity : 536870912 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: num_shard_bits : 4 2024-09-12T03:10:53.911 INFO:journalctl@ceph.mon.b.smithi121.stdout: strict_capacity_limit : 0 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: high_pri_pool_ratio: 0.000 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_cache_compressed: (nil) 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: persistent_cache: (nil) 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_size: 4096 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_size_deviation: 10 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_restart_interval: 16 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: index_block_restart_interval: 1 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: metadata_block_size: 4096 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: partition_filters: 0 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: use_delta_encoding: 1 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: filter_policy: bloomfilter 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: whole_key_filtering: 1 2024-09-12T03:10:53.912 INFO:journalctl@ceph.mon.b.smithi121.stdout: verify_compression: 0 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: read_amp_bytes_per_bit: 0 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: format_version: 5 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: enable_index_compression: 1 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: block_align: 0 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: max_auto_readahead_size: 262144 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: prepopulate_block_cache: 0 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: initial_auto_readahead_size: 8192 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression: NoCompression 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T03:10:53.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.num_levels: 7 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:10:53.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T03:10:53.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T03:10:53.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T03:10:53.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.inplace_update_support: 0 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T03:10:53.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.bloom_locality: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.max_successive_merges: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.ttl: 2592000 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enable_blob_files: false 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.min_blob_size: 0 2024-09-12T03:10:53.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6d1063e5-4c92-4fff-af1b-e2ca8211e1e2 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110653414658, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T03:10:53.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110653415085, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726110653, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6d1063e5-4c92-4fff-af1b-e2ca8211e1e2", "db_session_id": "BKXBC7SNS8DCAWME46AW", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726110653415176, "job": 1, "event": "recovery_finished"} 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c13ad5ce00 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: DB pointer 0x55c13ae58000 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** DB Stats ** 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T03:10:53.921 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** Compaction Stats [default] ** 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** Compaction Stats [default] ** 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T03:10:53.922 INFO:journalctl@ceph.mon.b.smithi121.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T03:10:53.923 INFO:journalctl@ceph.mon.b.smithi121.stdout: Block cache BinnedLRUCache@0x55c13ad2f1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: using public_addr v2:172.21.15.121:0/0 -> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: starting mon.b rank -1 at public addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] at bind addrs [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(???) e0 preinit fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).mds e1 new map 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: e1 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: btime 2024-09-12T03:09:21:563894+0000 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:10:53.924 INFO:journalctl@ceph.mon.b.smithi121.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout: legacy client fscid: -1 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout: 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout: No filesystems configured 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:10:53.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/464113117' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/464113117' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:53.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Updating smithi135:/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T03:10:53.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Saving service mon spec with placement smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c;count:3 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Deploying daemon mon.c on smithi135 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/1094392249' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.a calling monitor election 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.c calling monitor election 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: monmap epoch 2 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: last_changed 2024-09-12T03:10:44.386781+0000 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: min_mon_release 19 (squid) 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: election_strategy: 1 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: fsmap 2024-09-12T03:10:53.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mgrmap e13: x(active, since 38s) 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: overall HEALTH_OK 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: Deploying daemon mon.b on smithi121 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/187990449' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/3779453071' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:53.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 ceph-mon[34963]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T03:10:53.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 bash[34944]: 9d07ff7117278aa67d51cb99cc006a278a4f4932ed3948103734dfa2f735b92a 2024-09-12T03:10:53.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:53 smithi121 systemd[1]: Started Ceph mon.b for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:10:54.038 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:10:58.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: mon.a calling monitor election 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: mon.c calling monitor election 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: mon.b calling monitor election 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: monmap epoch 3 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: last_changed 2024-09-12T03:10:53.440729+0000 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: min_mon_release 19 (squid) 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: election_strategy: 1 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: fsmap 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:58.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: mgrmap e13: x(active, since 47s) 2024-09-12T03:10:58.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: overall HEALTH_OK 2024-09-12T03:10:58.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:58.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: mon.a calling monitor election 2024-09-12T03:10:58.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: mon.c calling monitor election 2024-09-12T03:10:58.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: mon.b calling monitor election 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: monmap epoch 3 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: last_changed 2024-09-12T03:10:53.440729+0000 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: min_mon_release 19 (squid) 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: election_strategy: 1 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-12T03:10:58.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: fsmap 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: mgrmap e13: x(active, since 47s) 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: overall HEALTH_OK 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: mon.a calling monitor election 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: mon.c calling monitor election 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: mon.b calling monitor election 2024-09-12T03:10:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: monmap epoch 3 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:10:58.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: last_changed 2024-09-12T03:10:53.440729+0000 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: created 2024-09-12T03:09:09.637034+0000 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: min_mon_release 19 (squid) 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: election_strategy: 1 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: 0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: 1: [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] mon.c 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: 2: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.b 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: fsmap 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: mgrmap e13: x(active, since 47s) 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: overall HEALTH_OK 2024-09-12T03:10:58.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:10:58.997 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 -- 172.21.15.135:0/493811768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa144003530 msgr2=0x7fa1440059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 --2- 172.21.15.135:0/493811768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa144003530 0x7fa1440059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa15c1016a0 tx=0x7fa14c02bfe0 comp rx=0 tx=0).stop 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 -- 172.21.15.135:0/493811768 shutdown_connections 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 --2- 172.21.15.135:0/493811768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa144003530 0x7fa1440059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 -- 172.21.15.135:0/493811768 >> 172.21.15.135:0/493811768 conn(0x7fa15c0fc220 msgr2=0x7fa15c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 -- 172.21.15.135:0/493811768 shutdown_connections 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 -- 172.21.15.135:0/493811768 wait complete. 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.995+0000 7fa161d71640 1 Processor -- start 2024-09-12T03:10:58.998 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 -- start start 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c108660 con 0x7fa15c1978a0 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1084e0 con 0x7fa144003530 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa161d71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c1087e0 con 0x7fa15c19c170 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.135:44182/0 (socket says 172.21.15.135:44182) 2024-09-12T03:10:58.999 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 -- 172.21.15.135:0/3920208142 learned_addr learned my addr 172.21.15.135:0/3920208142 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.135:45950/0 (socket says 172.21.15.135:45950) 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15bfff640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 msgr2=0x7fa15c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 msgr2=0x7fa15c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:10:59.000 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15b7fe640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15bfff640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 msgr2=0x7fa15c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15bfff640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 msgr2=0x7fa15c19c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15bfff640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.996+0000 7fa15bfff640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 msgr2=0x7fa15c19c620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 msgr2=0x7fa15c197360 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 -- 172.21.15.135:0/3920208142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1a3ee0 con 0x7fa15c1978a0 2024-09-12T03:10:59.001 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa15affd640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b450 tx=0x7fa15000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150004230 con 0x7fa15c1978a0 2024-09-12T03:10:59.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c1a4110 con 0x7fa15c1978a0 2024-09-12T03:10:59.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1500043d0 con 0x7fa15c1978a0 2024-09-12T03:10:59.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150003cb0 con 0x7fa15c1978a0 2024-09-12T03:10:59.002 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.997+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c1a4620 con 0x7fa15c1978a0 2024-09-12T03:10:59.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.998+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa15000f050 con 0x7fa15c1978a0 2024-09-12T03:10:59.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.998+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c109750 con 0x7fa15c1978a0 2024-09-12T03:10:59.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.998+0000 7fa158ff9640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 0x7fa14803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:10:59.003 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.998+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa15004be80 con 0x7fa15c1978a0 2024-09-12T03:10:59.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.999+0000 7fa15b7fe640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 0x7fa14803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:10:59.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:58.999+0000 7fa15b7fe640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 0x7fa14803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009b30 tx=0x7fa14c005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:10:59.004 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.004+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa15000d040 con 0x7fa15c1978a0 2024-09-12T03:10:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.235+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa15c1a4a10 con 0x7fa15c1978a0 2024-09-12T03:10:59.237 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.236+0000 7fa158ff9640 1 -- 172.21.15.135:0/3920208142 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1285 (secure 0 0 0) 0x7fa150003e50 con 0x7fa15c1978a0 2024-09-12T03:10:59.238 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:10:59.238 INFO:teuthology.orchestra.run.smithi135.stdout:{"epoch":3,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","modified":"2024-09-12T03:10:53.440729Z","created":"2024-09-12T03:09:09.637034Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:3300","nonce":0},{"type":"v1","addr":"172.21.15.135:6789","nonce":0}]},"addr":"172.21.15.135:6789/0","public_addr":"172.21.15.135:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-12T03:10:59.239 INFO:teuthology.orchestra.run.smithi135.stderr:dumped monmap epoch 3 2024-09-12T03:10:59.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 msgr2=0x7fa14803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.239 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 0x7fa14803b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009b30 tx=0x7fa14c005c50 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 msgr2=0x7fa15c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa15000b450 tx=0x7fa15000b910 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 shutdown_connections 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa15c19c170 0x7fa15c19c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1480394f0 0x7fa14803b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa15c1978a0 0x7fa15c19bc30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 --2- 172.21.15.135:0/3920208142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa144003530 0x7fa15c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 >> 172.21.15.135:0/3920208142 conn(0x7fa15c0fc220 msgr2=0x7fa15c1058f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 shutdown_connections 2024-09-12T03:10:59.240 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:10:59.238+0000 7fa161d71640 1 -- 172.21.15.135:0/3920208142 wait complete. 2024-09-12T03:10:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:59 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/3920208142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:10:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:59.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:59 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/3920208142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:10:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:10:59.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:59 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/3920208142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T03:10:59.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:10:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:11:00.257 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T03:11:00.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T03:11:00.517 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:00.572 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:11:00.440+0000 7faa161b2640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-12T03:11:00.879 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.880 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: Updating smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.881 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:00.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:00.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: Updating smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:00.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:01.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 -- 172.21.15.19:0/944643718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 msgr2=0x7f53cc106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/944643718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc106940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009880 tx=0x7f53c802a5d0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 -- 172.21.15.19:0/944643718 shutdown_connections 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/944643718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc106940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 -- 172.21.15.19:0/944643718 >> 172.21.15.19:0/944643718 conn(0x7f53cc0fc1b0 msgr2=0x7f53cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 -- 172.21.15.19:0/944643718 shutdown_connections 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.963+0000 7f53d3c1f640 1 -- 172.21.15.19:0/944643718 wait complete. 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 Processor -- start 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 -- start start 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 0x7f53cc19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 0x7f53cc19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10ab90 con 0x7f53cc106560 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10aa10 con 0x7f53cc199a80 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d3c1f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10ad10 con 0x7f53cc19e350 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d19bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d19bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54692/0 (socket says 172.21.15.19:54692) 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d19bb640 1 -- 172.21.15.19:0/2980049032 learned_addr learned my addr 172.21.15.19:0/2980049032 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d11ba640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 0x7f53cc19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d21bc640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 0x7f53cc19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.964+0000 7f53d19bb640 1 -- 172.21.15.19:0/2980049032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 msgr2=0x7f53cc19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d19bb640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 0x7f53cc19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d19bb640 1 -- 172.21.15.19:0/2980049032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 msgr2=0x7f53cc19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d19bb640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 0x7f53cc19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d19bb640 1 -- 172.21.15.19:0/2980049032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc1a60c0 con 0x7f53cc106560 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d21bc640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 0x7f53cc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d11ba640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 0x7f53cc19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d19bb640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006010 tx=0x7f53c8003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c803f070 con 0x7f53cc106560 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc1a6290 con 0x7f53cc106560 2024-09-12T03:11:01.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53c803b460 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8035030 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.965+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc1a67c0 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.966+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f53c8042070 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.966+0000 7f53c2ffd640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 0x7f53b803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.966+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53c806ccd0 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.966+0000 7f53d11ba640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 0x7f53b803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.966+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53cc1069c0 con 0x7f53cc106560 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.967+0000 7f53d11ba640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 0x7f53b803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009890 tx=0x7f53bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:01.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:00.971+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53c8039510 con 0x7f53cc106560 2024-09-12T03:11:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi019:/etc/ceph/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi121:/etc/ceph/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi135:/etc/ceph/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: Updating smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:01.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:01.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.153+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f53cc02d7c0 con 0x7f53cc106560 2024-09-12T03:11:01.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.153+0000 7f53c2ffd640 1 -- 172.21.15.19:0/2980049032 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f53c8039770 con 0x7f53cc106560 2024-09-12T03:11:01.153 INFO:teuthology.orchestra.run.smithi019.stdout:# minimal ceph.conf for 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:01.154 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-09-12T03:11:01.154 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:11:01.154 INFO:teuthology.orchestra.run.smithi019.stdout: mon_host = [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] 2024-09-12T03:11:01.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 msgr2=0x7f53b803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 0x7f53b803b9b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009890 tx=0x7f53bc008040 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 msgr2=0x7f53cc199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53c8006010 tx=0x7f53c8003c00 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 shutdown_connections 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f53cc19e350 0x7f53cc19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53cc199a80 0x7f53cc19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f53b80394f0 0x7f53b803b9b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 --2- 172.21.15.19:0/2980049032 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f53cc106560 0x7f53cc199540 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 >> 172.21.15.19:0/2980049032 conn(0x7f53cc0fc1b0 msgr2=0x7f53cc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.155+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 shutdown_connections 2024-09-12T03:11:01.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:01.156+0000 7f53d3c1f640 1 -- 172.21.15.19:0/2980049032 wait complete. 2024-09-12T03:11:01.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:01 smithi019 ceph-mon[30953]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:11:01.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:01 smithi019 ceph-mon[30953]: Reconfiguring daemon mon.a on smithi019 2024-09-12T03:11:01.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2980049032' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:01.836 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.865 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T03:11:01.866 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:11:01.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:11:01.895 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:11:01.895 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:01.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:01 smithi121 ceph-mon[34963]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:11:01.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:01 smithi121 ceph-mon[34963]: Reconfiguring daemon mon.a on smithi019 2024-09-12T03:11:01.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2980049032' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:01.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:01.961 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:11:01.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:11:01.990 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:11:01.990 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:02.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:01 smithi135 ceph-mon[35474]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T03:11:02.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:01 smithi135 ceph-mon[35474]: Reconfiguring daemon mon.a on smithi019 2024-09-12T03:11:02.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2980049032' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:02.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:02.055 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:11:02.055 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T03:11:02.087 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:11:02.087 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T03:11:02.157 INFO:tasks.cephadm:Adding mgr.x on smithi019 2024-09-12T03:11:02.158 INFO:tasks.cephadm:Adding mgr.y on smithi121 2024-09-12T03:11:02.158 INFO:tasks.cephadm:Adding mgr.z on smithi135 2024-09-12T03:11:02.158 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi019=x;smithi121=y;smithi135=z' 2024-09-12T03:11:02.440 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:11:02.740 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:02 smithi019 ceph-mon[30953]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:02 smithi135 ceph-mon[35474]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:02.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 -- 172.21.15.135:0/484445776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a01005f0 msgr2=0x7f24a01009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:02.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 --2- 172.21.15.135:0/484445776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a01005f0 0x7f24a01009f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f249c009880 tx=0x7f249c02a5d0 comp rx=0 tx=0).stop 2024-09-12T03:11:02.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 -- 172.21.15.135:0/484445776 shutdown_connections 2024-09-12T03:11:02.829 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 --2- 172.21.15.135:0/484445776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a01005f0 0x7f24a01009f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:02.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 -- 172.21.15.135:0/484445776 >> 172.21.15.135:0/484445776 conn(0x7f24a00fbda0 msgr2=0x7f24a00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:02.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 -- 172.21.15.135:0/484445776 shutdown_connections 2024-09-12T03:11:02.830 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.829+0000 7f24a8009640 1 -- 172.21.15.135:0/484445776 wait complete. 2024-09-12T03:11:02.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 Processor -- start 2024-09-12T03:11:02.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 -- start start 2024-09-12T03:11:02.831 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a0195300 0x7f24a019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a0195780 0x7f24a019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0105600 con 0x7f24a0195780 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0105480 con 0x7f24a0195300 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a8009640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24a0105780 con 0x7f24a01005f0 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a5da5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a5da5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:60632/0 (socket says 172.21.15.135:60632) 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.830+0000 7f24a5da5640 1 -- 172.21.15.135:0/1943252751 learned_addr learned my addr 172.21.15.135:0/1943252751 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a65a6640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a0195780 0x7f24a019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a55a4640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a0195300 0x7f24a019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:02.832 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 -- 172.21.15.135:0/1943252751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a0195300 msgr2=0x7f24a019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a0195300 0x7f24a019a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 -- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a0195780 msgr2=0x7f24a019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a0195780 0x7f24a019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 -- 172.21.15.135:0/1943252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a019d1c0 con 0x7f24a01005f0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a5da5640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f249c005ab0 tx=0x7f249c032d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c03f030 con 0x7f24a01005f0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a01a1860 con 0x7f24a01005f0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f249c003ce0 con 0x7f24a01005f0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c035070 con 0x7f24a01005f0 2024-09-12T03:11:02.833 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.831+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a01a1dc0 con 0x7f24a01005f0 2024-09-12T03:11:02.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.832+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f249c042030 con 0x7f24a01005f0 2024-09-12T03:11:02.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.832+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a018e3c0 con 0x7f24a01005f0 2024-09-12T03:11:02.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.832+0000 7f2492ffd640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 0x7f248c03b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:02.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.832+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f249c06cb50 con 0x7f24a01005f0 2024-09-12T03:11:02.834 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.833+0000 7f24a55a4640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 0x7f248c03b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:02.835 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.833+0000 7f24a55a4640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 0x7f248c03b9a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24a01962e0 tx=0x7f2494009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:02.837 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:02.837+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f249c03be20 con 0x7f24a01005f0 2024-09-12T03:11:02.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:02 smithi121 ceph-mon[34963]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:03.013 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.013+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi121=y;smithi135=z", "target": ["mon-mgr", ""]}) -- 0x7f24a00626c0 con 0x7f248c0394e0 2024-09-12T03:11:03.018 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.018+0000 7f2492ffd640 1 -- 172.21.15.135:0/1943252751 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f24a00626c0 con 0x7f248c0394e0 2024-09-12T03:11:03.018 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mgr update... 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 msgr2=0x7f248c03b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 0x7f248c03b9a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24a01962e0 tx=0x7f2494009190 comp rx=0 tx=0).stop 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 msgr2=0x7f24a0194dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f249c005ab0 tx=0x7f249c032d00 comp rx=0 tx=0).stop 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 shutdown_connections 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f248c0394e0 0x7f248c03b9a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f24a0195780 0x7f24a019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:03.021 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24a0195300 0x7f24a019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:03.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 --2- 172.21.15.135:0/1943252751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f24a01005f0 0x7f24a0194dc0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:03.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 >> 172.21.15.135:0/1943252751 conn(0x7f24a00fbda0 msgr2=0x7f24a00fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:03.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 shutdown_connections 2024-09-12T03:11:03.022 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:03.021+0000 7f24a8009640 1 -- 172.21.15.135:0/1943252751 wait complete. 2024-09-12T03:11:03.627 DEBUG:teuthology.orchestra.run.smithi121:mgr.y> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T03:11:03.630 DEBUG:teuthology.orchestra.run.smithi135:mgr.z> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T03:11:03.633 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T03:11:03.633 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:11:03.633 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:11:03.653 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:11:03.653 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-09-12T03:11:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-12 03:10:21.751604391 +0000 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-12 03:06:04.791868821 +0000 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-12 03:06:04.791868821 +0000 2024-09-12T03:11:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-12 03:06:04.791868821 +0000 2024-09-12T03:11:03.714 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:11:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-12T03:11:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-12T03:11:03.784 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000239071 s, 2.1 MB/s 2024-09-12T03:11:03.785 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:11:03.845 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-12 03:10:21.779604131 +0000 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-12 03:06:05.141865849 +0000 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-12 03:06:05.141865849 +0000 2024-09-12T03:11:03.903 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-12 03:06:05.141865849 +0000 2024-09-12T03:11:03.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:11:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-12T03:11:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-12T03:11:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000184705 s, 2.8 MB/s 2024-09-12T03:11:03.971 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:11:04.031 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: Reconfiguring daemon mon.b on smithi121 2024-09-12T03:11:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi121=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:04.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: Saving service mgr spec with placement smithi019=x;smithi121=y;smithi135=z;count:3 2024-09-12T03:11:04.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-12 03:10:21.807603871 +0000 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-12 03:06:05.476863004 +0000 2024-09-12T03:11:04.089 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-12 03:06:05.476863004 +0000 2024-09-12T03:11:04.090 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-12 03:06:05.476863004 +0000 2024-09-12T03:11:04.090 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: Reconfiguring daemon mon.b on smithi121 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi121=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: Saving service mgr spec with placement smithi019=x;smithi121=y;smithi135=z;count:3 2024-09-12T03:11:04.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: Reconfiguring mon.b (monmap changed)... 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:04.153 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: Reconfiguring daemon mon.b on smithi121 2024-09-12T03:11:04.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi019=x;smithi121=y;smithi135=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:04.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: Saving service mgr spec with placement smithi019=x;smithi121=y;smithi135=z;count:3 2024-09-12T03:11:04.154 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-12T03:11:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-12T03:11:04.156 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000228148 s, 2.2 MB/s 2024-09-12T03:11:04.157 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:11:04.216 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-12 03:10:21.834603620 +0000 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-12 03:06:05.841859905 +0000 2024-09-12T03:11:04.274 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-12 03:06:05.841859905 +0000 2024-09-12T03:11:04.275 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-12 03:06:05.841859905 +0000 2024-09-12T03:11:04.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:11:04.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-12T03:11:04.339 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-12T03:11:04.340 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000223913 s, 2.3 MB/s 2024-09-12T03:11:04.341 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:11:04.399 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-09-12T03:11:04.399 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:11:04.417 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:11:04.417 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-12 03:10:52.864221843 +0000 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-12 03:06:18.896161434 +0000 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-12 03:06:18.896161434 +0000 2024-09-12T03:11:04.473 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-12 03:06:18.896161434 +0000 2024-09-12T03:11:04.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:11:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-12T03:11:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-12T03:11:04.538 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000220398 s, 2.3 MB/s 2024-09-12T03:11:04.540 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:11:04.596 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-09-12T03:11:04.652 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:11:04.652 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:04.652 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T03:11:04.652 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-12 03:10:52.864221843 +0000 2024-09-12T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-12 03:06:19.232156908 +0000 2024-09-12T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-12 03:06:19.232156908 +0000 2024-09-12T03:11:04.653 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-12 03:06:19.232156908 +0000 2024-09-12T03:11:04.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:11:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-12T03:11:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-12T03:11:04.715 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.00026696 s, 1.9 MB/s 2024-09-12T03:11:04.717 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:11:04.772 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:04.828 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-12 03:10:52.865221829 +0000 2024-09-12T03:11:04.829 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-12 03:06:19.565152423 +0000 2024-09-12T03:11:04.829 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-12 03:06:19.565152423 +0000 2024-09-12T03:11:04.829 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-12 03:06:19.565152423 +0000 2024-09-12T03:11:04.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:11:04.891 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-12T03:11:04.891 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-12T03:11:04.891 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000188081 s, 2.7 MB/s 2024-09-12T03:11:04.893 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:11:04.949 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-09-12 03:10:52.865221829 +0000 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-09-12 03:06:19.885148113 +0000 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-09-12 03:06:19.885148113 +0000 2024-09-12T03:11:05.004 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-09-12 03:06:19.885148113 +0000 2024-09-12T03:11:05.005 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:11:05.067 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-09-12T03:11:05.067 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-09-12T03:11:05.067 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000213555 s, 2.4 MB/s 2024-09-12T03:11:05.068 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:11:05.125 DEBUG:teuthology.orchestra.run.smithi135:> set -ex 2024-09-12T03:11:05.125 DEBUG:teuthology.orchestra.run.smithi135:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T03:11:05.142 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T03:11:05.142 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_1 2024-09-12T03:11:05.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:05.197 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T03:11:05.197 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:05.197 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-12T03:11:05.197 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:05.197 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:05.198 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-12 03:10:43.440779919 +0000 2024-09-12T03:11:05.198 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-12 03:06:20.136055992 +0000 2024-09-12T03:11:05.198 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-12 03:06:20.136055992 +0000 2024-09-12T03:11:05.198 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-12 03:06:20.136055992 +0000 2024-09-12T03:11:05.198 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T03:11:05.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:05.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:05.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:05.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:05.262 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-12T03:11:05.262 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-12T03:11:05.262 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000266637 s, 1.9 MB/s 2024-09-12T03:11:05.263 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T03:11:05.319 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_2 2024-09-12T03:11:05.375 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-12 03:10:43.440779919 +0000 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-12 03:06:20.463049076 +0000 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-12 03:06:20.463049076 +0000 2024-09-12T03:11:05.376 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-12 03:06:20.463049076 +0000 2024-09-12T03:11:05.376 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T03:11:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T03:11:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T03:11:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:05.440 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-12T03:11:05.441 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-12T03:11:05.441 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.00017868 s, 2.9 MB/s 2024-09-12T03:11:05.442 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T03:11:05.497 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_3 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-12 03:10:43.440779919 +0000 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-12 03:06:20.784042288 +0000 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-12 03:06:20.784042288 +0000 2024-09-12T03:11:05.554 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-12 03:06:20.784042288 +0000 2024-09-12T03:11:05.555 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T03:11:05.618 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-12T03:11:05.618 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-12T03:11:05.618 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000177312 s, 2.9 MB/s 2024-09-12T03:11:05.619 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T03:11:05.676 DEBUG:teuthology.orchestra.run.smithi135:> stat /dev/vg_nvme/lv_4 2024-09-12T03:11:05.732 INFO:teuthology.orchestra.run.smithi135.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Access: 2024-09-12 03:10:43.440779919 +0000 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Modify: 2024-09-12 03:06:21.115035288 +0000 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout:Change: 2024-09-12 03:06:21.115035288 +0000 2024-09-12T03:11:05.733 INFO:teuthology.orchestra.run.smithi135.stdout: Birth: 2024-09-12 03:06:21.115035288 +0000 2024-09-12T03:11:05.733 DEBUG:teuthology.orchestra.run.smithi135:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T03:11:05.798 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records in 2024-09-12T03:11:05.999 INFO:teuthology.orchestra.run.smithi135.stderr:1+0 records out 2024-09-12T03:11:05.999 INFO:teuthology.orchestra.run.smithi135.stderr:512 bytes copied, 0.000207547 s, 2.5 MB/s 2024-09-12T03:11:06.001 DEBUG:teuthology.orchestra.run.smithi135:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T03:11:06.060 INFO:tasks.cephadm:Deploying osd.0 on smithi019 with /dev/vg_nvme/lv_4... 2024-09-12T03:11:06.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: Reconfiguring daemon mon.c on smithi135 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:06.168 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:06.169 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:06.326 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: Reconfiguring daemon mon.c on smithi135 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:06.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:06.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:06.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: Reconfiguring mon.c (monmap changed)... 2024-09-12T03:11:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: Reconfiguring daemon mon.c on smithi135 2024-09-12T03:11:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:06.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:07.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:07 smithi135 ceph-mon[35474]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:07.158 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:07 smithi135 ceph-mon[35474]: Deploying daemon mgr.z on smithi135 2024-09-12T03:11:07.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:07 smithi019 ceph-mon[30953]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:07.296 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:07 smithi019 ceph-mon[30953]: Deploying daemon mgr.z on smithi135 2024-09-12T03:11:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:07 smithi121 ceph-mon[34963]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:07.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:07 smithi121 ceph-mon[34963]: Deploying daemon mgr.z on smithi135 2024-09-12T03:11:07.532 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 systemd[1]: Starting Ceph mgr.z for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:11:07.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 podman[37265]: 2024-09-12 03:11:07.4350676 +0000 UTC m=+0.015869418 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:11:07.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 podman[37265]: 2024-09-12 03:11:07.540101613 +0000 UTC m=+0.120903461 container create f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:11:07.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 podman[37265]: 2024-09-12 03:11:07.734269172 +0000 UTC m=+0.315070982 container init f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:11:07.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 podman[37265]: 2024-09-12 03:11:07.737878273 +0000 UTC m=+0.318680076 container start f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-12T03:11:07.795 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 ceph-mgr[37284]: -- 172.21.15.135:0/2661555064 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571d38c90e0 con 0x5571d38a7000 2024-09-12T03:11:08.073 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 bash[37265]: f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b 2024-09-12T03:11:08.073 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 systemd[1]: Started Ceph mgr.z for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:08.074 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:07.862+0000 7f9e0b2b4040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:11:08.074 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:07.925+0000 7f9e0b2b4040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:08.074 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.073+0000 7f9e0b2b4040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:11:08.425 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.139+0000 7f9e0b2b4040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:11:08.425 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.285+0000 7f9e0b2b4040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:08.425 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.424+0000 7f9e0b2b4040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:11:08.673 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:11:08.701 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_4 2024-09-12T03:11:08.953 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:11:09.044 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.737+0000 7f9e0b2b4040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:11:09.044 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.798+0000 7f9e0b2b4040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:08.982+0000 7f9e0b2b4040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:09.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:09.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:09.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:08 smithi135 ceph-mon[35474]: Deploying daemon mgr.y on smithi121 2024-09-12T03:11:09.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:09.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:08 smithi019 ceph-mon[30953]: Deploying daemon mgr.y on smithi121 2024-09-12T03:11:09.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:08 smithi121 ceph-mon[34963]: Deploying daemon mgr.y on smithi121 2024-09-12T03:11:09.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- 172.21.15.19:0/3932432285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100dd0 msgr2=0x7f5458101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:09.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3932432285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100dd0 0x7f5458101230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5448009900 tx=0x7f544802a410 comp rx=0 tx=0).stop 2024-09-12T03:11:09.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- 172.21.15.19:0/3932432285 shutdown_connections 2024-09-12T03:11:09.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3932432285 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 0x7f5458101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:09.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3932432285 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100dd0 0x7f5458101230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3932432285 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100490 0x7f5458100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- 172.21.15.19:0/3932432285 >> 172.21.15.19:0/3932432285 conn(0x7f54580fc2a0 msgr2=0x7f54580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- 172.21.15.19:0/3932432285 shutdown_connections 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- 172.21.15.19:0/3932432285 wait complete. 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 Processor -- start 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 -- start start 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100dd0 0x7f545819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.409+0000 7f545f3d0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 0x7f54581a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545f3d0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f545810fb30 con 0x7f5458100490 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545f3d0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f545810f9b0 con 0x7f5458100dd0 2024-09-12T03:11:09.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545f3d0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f545810fcb0 con 0x7f5458101770 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54704/0 (socket says 172.21.15.19:54704) 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 -- 172.21.15.19:0/3019184171 learned_addr learned my addr 172.21.15.19:0/3019184171 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d96d640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 0x7f54581a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545c96b640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100dd0 0x7f545819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 -- 172.21.15.19:0/3019184171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 msgr2=0x7f54581a1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 0x7f54581a1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 -- 172.21.15.19:0/3019184171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100dd0 msgr2=0x7f545819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100dd0 0x7f545819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 -- 172.21.15.19:0/3019184171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54581a1d80 con 0x7f5458100490 2024-09-12T03:11:09.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545d16c640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4a0 tx=0x7f545400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454014070 con 0x7f5458100490 2024-09-12T03:11:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5454010af0 con 0x7f5458100490 2024-09-12T03:11:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54581a2070 con 0x7f5458100490 2024-09-12T03:11:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5454017d10 con 0x7f5458100490 2024-09-12T03:11:09.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.410+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54581a2650 con 0x7f5458100490 2024-09-12T03:11:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.411+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5454017070 con 0x7f5458100490 2024-09-12T03:11:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.411+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f545810b1e0 con 0x7f5458100490 2024-09-12T03:11:09.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.412+0000 7f544e7fc640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 0x7f544403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.412+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f545404b6d0 con 0x7f5458100490 2024-09-12T03:11:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.412+0000 7f545c96b640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 0x7f544403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.412+0000 7f545c96b640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 0x7f544403b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5448031440 tx=0x7f54480315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:09.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.416+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f545401c120 con 0x7f5458100490 2024-09-12T03:11:09.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:09.601+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5458062630 con 0x7f54440392c0 2024-09-12T03:11:09.884 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:09 smithi121 systemd[1]: Starting Ceph mgr.y for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:11:10.138 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:10.138 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:10.138 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:10.139 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:10.139 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:10.139 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:10.140 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:09 smithi121 podman[36318]: 2024-09-12 03:11:09.809521126 +0000 UTC m=+0.015811070 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:11:10.140 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:09 smithi121 podman[36318]: 2024-09-12 03:11:09.910420621 +0000 UTC m=+0.116710556 container create 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T03:11:10.140 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 podman[36318]: 2024-09-12 03:11:10.112963964 +0000 UTC m=+0.319253898 container init 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:11:10.140 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 podman[36318]: 2024-09-12 03:11:10.115804418 +0000 UTC m=+0.322094354 container start 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:11:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:10.294 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:09.856+0000 7f9e0b2b4040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:11:10.294 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:09.926+0000 7f9e0b2b4040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:11:10.399 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mgr[36337]: -- 172.21.15.121:0/2535906912 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bdada250e0 con 0x55bdada03400 2024-09-12T03:11:10.399 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 bash[36318]: 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 2024-09-12T03:11:10.400 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 systemd[1]: Started Ceph mgr.y for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:11:10.400 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.245+0000 7f3b2b4e8040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T03:11:10.400 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.313+0000 7f3b2b4e8040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:10.850 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.468+0000 7f3b2b4e8040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T03:11:10.850 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.537+0000 7f3b2b4e8040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T03:11:10.850 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.696+0000 7f3b2b4e8040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:10.850 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:10.849+0000 7f3b2b4e8040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T03:11:10.955 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:10.675+0000 7f9e0b2b4040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:11:10.955 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:10.804+0000 7f9e0b2b4040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:10.955+0000 7f9e0b2b4040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.956 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:10.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:11.121 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:11.122 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:11.243 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.016+0000 7f9e0b2b4040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:11:11.243 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.173+0000 7f9e0b2b4040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:11:11.243 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.243+0000 7f9e0b2b4040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:11:11.400 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:11.173+0000 7f3b2b4e8040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T03:11:11.400 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:11.236+0000 7f3b2b4e8040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T03:11:11.541 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.303+0000 7f9e0b2b4040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:11:11.542 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.463+0000 7f9e0b2b4040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:11:11.542 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.541+0000 7f9e0b2b4040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:11:11.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.608+0000 7f9e0b2b4040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:11:11.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.695+0000 7f9e0b2b4040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:11.794 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.754+0000 7f9e0b2b4040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:11:11.899 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:11.423+0000 7f3b2b4e8040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T03:11:12.222 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:11.815+0000 7f9e0b2b4040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:11:12.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:12.534 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.535 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: Reconfiguring daemon mgr.x on smithi019 2024-09-12T03:11:12.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]: dispatch 2024-09-12T03:11:12.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]': finished 2024-09-12T03:11:12.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:11:12.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:11:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: Reconfiguring daemon mgr.x on smithi019 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]: dispatch 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]': finished 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:11:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:12.649 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:12.316+0000 7f3b2b4e8040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:12.382+0000 7f3b2b4e8040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: Reconfiguring daemon mgr.x on smithi019 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]: dispatch 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1276187901' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c60c7b8-10fe-4d61-8f59-09d8bb612f89"}]': finished 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T03:11:12.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:13.044 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:11:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z[37280]: 2024-09-12T03:11:12.603+0000 7f9e0b2b4040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:11:13.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:13.232+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f545401ecf0 con 0x7f5458100490 2024-09-12T03:11:13.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2659145590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:13.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: Standby manager daemon z started 2024-09-12T03:11:13.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:11:13.348 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:13.349 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:11:13.349 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:13.349 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.093+0000 7f3b2b4e8040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T03:11:13.349 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.205+0000 7f3b2b4e8040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T03:11:13.349 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.348+0000 7f3b2b4e8040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2659145590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: Standby manager daemon z started 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:11:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:13 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:13.624 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.409+0000 7f3b2b4e8040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T03:11:13.624 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.557+0000 7f3b2b4e8040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T03:11:13.624 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.624+0000 7f3b2b4e8040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T03:11:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2659145590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: Standby manager daemon z started 2024-09-12T03:11:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T03:11:13.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:13.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T03:11:13.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:13 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.135:0/2399789488' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:13.899 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.683+0000 7f3b2b4e8040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T03:11:13.899 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.838+0000 7f3b2b4e8040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T03:11:14.168 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.903+0000 7f3b2b4e8040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T03:11:14.168 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:13.968+0000 7f3b2b4e8040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T03:11:14.169 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:14.050+0000 7f3b2b4e8040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T03:11:14.169 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:14.108+0000 7f3b2b4e8040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T03:11:14.169 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:14.168+0000 7f3b2b4e8040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T03:11:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:14 smithi135 ceph-mon[35474]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:14 smithi135 ceph-mon[35474]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:11:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:14 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:11:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-mon[34963]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-mon[34963]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:11:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:11:14.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:14 smithi019 ceph-mon[30953]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:14.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:14 smithi019 ceph-mon[30953]: mgrmap e14: x(active, since 62s), standbys: z 2024-09-12T03:11:14.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:14 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T03:11:15.229 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:11:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y[36333]: 2024-09-12T03:11:14.949+0000 7f3b2b4e8040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T03:11:15.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:15.238+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mon.0 v2:172.21.15.19:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5454010610 con 0x7f5458100490 2024-09-12T03:11:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:15 smithi135 ceph-mon[35474]: Standby manager daemon y started 2024-09-12T03:11:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:15 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:11:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:15 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:15 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:11:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:15 smithi135 ceph-mon[35474]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:15 smithi121 ceph-mon[34963]: Standby manager daemon y started 2024-09-12T03:11:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:15 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:11:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:15 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:15 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:11:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:15 smithi121 ceph-mon[34963]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:15 smithi019 ceph-mon[30953]: Standby manager daemon y started 2024-09-12T03:11:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:15 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T03:11:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:15 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T03:11:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:15 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T03:11:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:15 smithi019 ceph-mon[30953]: from='mgr.? 172.21.15.121:0/4266434182' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T03:11:16.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:16 smithi019 ceph-mon[30953]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:16.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:16 smithi019 ceph-mon[30953]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:11:16.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:11:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:16 smithi135 ceph-mon[35474]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:16 smithi135 ceph-mon[35474]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:11:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:11:16.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:16 smithi121 ceph-mon[34963]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:16.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:16 smithi121 ceph-mon[34963]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-12T03:11:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:19.276 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:19.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:19.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:19.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:19.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:19.625 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:21.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:21 smithi019 ceph-mon[30953]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:21.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:11:21.227 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:21 smithi135 ceph-mon[35474]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:11:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:21.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:21 smithi121 ceph-mon[34963]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:21.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T03:11:21.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:22.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:22 smithi019 ceph-mon[30953]: Deploying daemon osd.0 on smithi019 2024-09-12T03:11:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:22 smithi135 ceph-mon[35474]: Deploying daemon osd.0 on smithi019 2024-09-12T03:11:22.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:22 smithi121 ceph-mon[34963]: Deploying daemon osd.0 on smithi019 2024-09-12T03:11:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:23 smithi135 ceph-mon[35474]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:23.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:23 smithi019 ceph-mon[30953]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:23.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:23 smithi121 ceph-mon[34963]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:24 smithi135 ceph-mon[35474]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:24.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:24 smithi121 ceph-mon[34963]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:24.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:24 smithi019 ceph-mon[30953]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:26.537 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:26 smithi019 ceph-mon[30953]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:26.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:26 smithi135 ceph-mon[35474]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:26.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:26 smithi121 ceph-mon[34963]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:27.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:27.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:27.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:28.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:28 smithi019 ceph-mon[30953]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:28 smithi135 ceph-mon[35474]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:29.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:28 smithi121 ceph-mon[34963]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:30.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:30.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.969+0000 7f544e7fc640 1 -- 172.21.15.19:0/3019184171 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5458062630 con 0x7f54440392c0 2024-09-12T03:11:30.971 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 0 on host 'smithi019' 2024-09-12T03:11:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 msgr2=0x7f544403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 0x7f544403b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5448031440 tx=0x7f54480315b0 comp rx=0 tx=0).stop 2024-09-12T03:11:30.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 msgr2=0x7f545819a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f545400b4a0 tx=0x7f545400b960 comp rx=0 tx=0).stop 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 shutdown_connections 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5458101770 0x7f54581a1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5458100dd0 0x7f545819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54440392c0 0x7f544403b780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 --2- 172.21.15.19:0/3019184171 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5458100490 0x7f545819a150 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 >> 172.21.15.19:0/3019184171 conn(0x7f54580fc2a0 msgr2=0x7f54580fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 shutdown_connections 2024-09-12T03:11:30.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:11:30.973+0000 7f545f3d0640 1 -- 172.21.15.19:0/3019184171 wait complete. 2024-09-12T03:11:31.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:31 smithi019 ceph-mon[30953]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:31.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:31.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:31 smithi121 ceph-mon[34963]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:31.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:31.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:31 smithi135 ceph-mon[35474]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:31.601 DEBUG:teuthology.orchestra.run.smithi019:osd.0> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T03:11:31.603 INFO:tasks.cephadm:Deploying osd.1 on smithi121 with /dev/vg_nvme/lv_4... 2024-09-12T03:11:31.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:11:31.840 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:11:32.414 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:11:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:11:32.280+0000 7f68e41a5640 -1 osd.0 0 log_to_monitors true 2024-09-12T03:11:33.198 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:33 smithi121 ceph-mon[34963]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:33.199 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:33 smithi121 ceph-mon[34963]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:11:33.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:33 smithi019 ceph-mon[30953]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:33.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:33 smithi019 ceph-mon[30953]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:11:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:33 smithi135 ceph-mon[35474]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:33 smithi135 ceph-mon[35474]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T03:11:34.229 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:11:34.253 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:34 smithi121 ceph-mon[34963]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:11:34.253 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:34 smithi121 ceph-mon[34963]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:11:34.253 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:34 smithi121 ceph-mon[34963]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:11:34.253 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:34.253 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:34 smithi121 ceph-mon[34963]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:11:34.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_4 2024-09-12T03:11:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-mon[30953]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:11:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-mon[30953]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:11:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-mon[30953]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:11:34.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:34.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-mon[30953]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:11:34.504 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:11:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:34 smithi135 ceph-mon[35474]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T03:11:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:34 smithi135 ceph-mon[35474]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T03:11:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:34 smithi135 ceph-mon[35474]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:11:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:34 smithi135 ceph-mon[35474]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:11:34.734 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:11:34.560+0000 7f68e0128640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T03:11:34.734 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:11:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:11:34.563+0000 7f68d7f48640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/1430005943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74100ad0 msgr2=0x7f6f74100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/1430005943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74100ad0 0x7f6f74100ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009900 tx=0x7f6f6402a410 comp rx=0 tx=0).stop 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/1430005943 shutdown_connections 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/1430005943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 0x7f6f74108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/1430005943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74101cd0 0x7f6f74102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/1430005943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74100ad0 0x7f6f74100ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/1430005943 >> 172.21.15.121:0/1430005943 conn(0x7f6f740fc280 msgr2=0x7f6f740fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:34.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/1430005943 shutdown_connections 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/1430005943 wait complete. 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.972+0000 7f6f7a8ca640 1 Processor -- start 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 -- start start 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74100ad0 0x7f6f74197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 0x7f6f7419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410c800 con 0x7f6f74102690 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410c680 con 0x7f6f74101cd0 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f7a8ca640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6f7410c980 con 0x7f6f74100ad0 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55910/0 (socket says 172.21.15.121:55910) 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 -- 172.21.15.121:0/3223114017 learned_addr learned my addr 172.21.15.121:0/3223114017 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f78e67640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 0x7f6f7419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f73fff640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74100ad0 0x7f6f74197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 -- 172.21.15.121:0/3223114017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74100ad0 msgr2=0x7f6f74197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74100ad0 0x7f6f74197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 -- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 msgr2=0x7f6f7419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 0x7f6f7419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.973+0000 7f6f737fe640 1 -- 172.21.15.121:0/3223114017 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f7419fbd0 con 0x7f6f74101cd0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f737fe640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800b4a0 tx=0x7f6f6800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68014070 con 0x7f6f74101cd0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f7419fec0 con 0x7f6f74101cd0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f68010af0 con 0x7f6f74101cd0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f68017d10 con 0x7f6f74101cd0 2024-09-12T03:11:34.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.974+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f741a03f0 con 0x7f6f74101cd0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.975+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f741a0680 con 0x7f6f74101cd0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.976+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f6f68017070 con 0x7f6f74101cd0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.977+0000 7f6f717fa640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 0x7f6f5c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.977+0000 7f6f73fff640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 0x7f6f5c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.977+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f6f680cc7d0 con 0x7f6f74101cd0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.978+0000 7f6f73fff640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 0x7f6f5c0a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402aa00 tx=0x7f6f64031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:34.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:34.982+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f6809b2c0 con 0x7f6f74101cd0 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' 2024-09-12T03:11:35.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:35.168+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6f741a0850 con 0x7f6f5c0a2b20 2024-09-12T03:11:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' 2024-09-12T03:11:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T03:11:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955]' entity='osd.0' 2024-09-12T03:11:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi019 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] boot 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:11:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:11:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:11:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi019 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] boot 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:11:36.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi019 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi019 to 17092M 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: osd.0 [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] boot 2024-09-12T03:11:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T03:11:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T03:11:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:37 smithi121 ceph-mon[34963]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:37 smithi121 ceph-mon[34963]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:37 smithi019 ceph-mon[30953]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:37 smithi019 ceph-mon[30953]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:37 smithi135 ceph-mon[35474]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:11:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:37 smithi135 ceph-mon[35474]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/420610814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]': finished 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:11:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/3988025817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/420610814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]': finished 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:11:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:11:38.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/3988025817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/420610814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]: dispatch 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963d3ee4-bad4-4896-8b30-38c5d33c0333"}]': finished 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:11:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/3988025817' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:11:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:39 smithi121 ceph-mon[34963]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:39 smithi019 ceph-mon[30953]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:39 smithi135 ceph-mon[35474]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:41 smithi121 ceph-mon[34963]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:41 smithi019 ceph-mon[30953]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:41 smithi135 ceph-mon[35474]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:43.176 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:43 smithi121 ceph-mon[34963]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:43 smithi019 ceph-mon[30953]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:43 smithi135 ceph-mon[35474]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:45.329 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:45 smithi121 ceph-mon[34963]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:45 smithi019 ceph-mon[30953]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:45 smithi135 ceph-mon[35474]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:46.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:11:46.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:46.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:11:46.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:46.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T03:11:46.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:47.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:47 smithi121 ceph-mon[34963]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:47 smithi121 ceph-mon[34963]: Deploying daemon osd.1 on smithi121 2024-09-12T03:11:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:47 smithi019 ceph-mon[30953]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:47 smithi019 ceph-mon[30953]: Deploying daemon osd.1 on smithi121 2024-09-12T03:11:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:47 smithi135 ceph-mon[35474]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:47 smithi135 ceph-mon[35474]: Deploying daemon osd.1 on smithi121 2024-09-12T03:11:49.246 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:49 smithi121 ceph-mon[34963]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:49 smithi019 ceph-mon[30953]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:49 smithi135 ceph-mon[35474]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:51.276 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:51 smithi121 ceph-mon[34963]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:51 smithi019 ceph-mon[30953]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:51 smithi135 ceph-mon[35474]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:53.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:53 smithi121 ceph-mon[34963]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:53 smithi019 ceph-mon[30953]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:53.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:53.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:53 smithi135 ceph-mon[35474]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:55 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:55.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:55 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:11:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:55 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:55.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.870+0000 7f6f717fa640 1 -- 172.21.15.121:0/3223114017 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f741a0850 con 0x7f6f5c0a2b20 2024-09-12T03:11:55.871 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 1 on host 'smithi121' 2024-09-12T03:11:55.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 msgr2=0x7f6f5c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 0x7f6f5c0a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6f6402aa00 tx=0x7f6f64031400 comp rx=0 tx=0).stop 2024-09-12T03:11:55.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 msgr2=0x7f6f741984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:55.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f6800b4a0 tx=0x7f6f6800b960 comp rx=0 tx=0).stop 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 shutdown_connections 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f5c0a2b20 0x7f6f5c0a4fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6f74102690 0x7f6f7419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6f74101cd0 0x7f6f741984e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 --2- 172.21.15.121:0/3223114017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6f74100ad0 0x7f6f74197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.872+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 >> 172.21.15.121:0/3223114017 conn(0x7f6f740fc280 msgr2=0x7f6f74069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:55.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.873+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 shutdown_connections 2024-09-12T03:11:55.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:11:55.873+0000 7f6f7a8ca640 1 -- 172.21.15.121:0/3223114017 wait complete. 2024-09-12T03:11:56.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:56 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:56 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:56 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:56 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:56 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:56 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:56 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:11:56.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:56 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:56 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:11:56.561 DEBUG:teuthology.orchestra.run.smithi121:osd.1> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T03:11:56.563 INFO:tasks.cephadm:Deploying osd.2 on smithi135 with /dev/vg_nvme/lv_4... 2024-09-12T03:11:56.563 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T03:11:56.816 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:11:57.220 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:57 smithi135 ceph-mon[35474]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:57 smithi121 ceph-mon[34963]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:57 smithi019 ceph-mon[30953]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:58.400 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:11:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:11:58.184+0000 7f87b58a9640 -1 osd.1 0 log_to_monitors true 2024-09-12T03:11:59.183 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:11:59.205 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:59 smithi135 ceph-mon[35474]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:59.205 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:59 smithi135 ceph-mon[35474]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.205 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:11:59 smithi135 ceph-mon[35474]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.207 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_4 2024-09-12T03:11:59.355 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:59 smithi121 ceph-mon[34963]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:59.355 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:59 smithi121 ceph-mon[34963]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.355 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:11:59 smithi121 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:59 smithi019 ceph-mon[30953]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:11:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:59 smithi019 ceph-mon[30953]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:11:59 smithi019 ceph-mon[30953]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T03:11:59.452 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:11:59.868 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- 172.21.15.135:0/3997281041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4102e00 msgr2=0x7efef4103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 --2- 172.21.15.135:0/3997281041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4102e00 0x7efef4103280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efee8009900 tx=0x7efee802a410 comp rx=0 tx=0).stop 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- 172.21.15.135:0/3997281041 shutdown_connections 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 --2- 172.21.15.135:0/3997281041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 0x7efef410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 --2- 172.21.15.135:0/3997281041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4102e00 0x7efef4103280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 --2- 172.21.15.135:0/3997281041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4101c00 0x7efef4102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- 172.21.15.135:0/3997281041 >> 172.21.15.135:0/3997281041 conn(0x7efef40fd390 msgr2=0x7efef40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- 172.21.15.135:0/3997281041 shutdown_connections 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- 172.21.15.135:0/3997281041 wait complete. 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 Processor -- start 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.867+0000 7efefa98d640 1 -- start start 2024-09-12T03:11:59.869 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4102e00 0x7efef41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 0x7efef419f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d930 con 0x7efef4102e00 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d7b0 con 0x7efef41037c0 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efefa98d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efef410dab0 con 0x7efef4101c00 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:43840/0 (socket says 172.21.15.135:43840) 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 -- 172.21.15.135:0/1860754936 learned_addr learned my addr 172.21.15.135:0/1860754936 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef8f2a640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 0x7efef419f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef37fe640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4102e00 0x7efef41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 -- 172.21.15.135:0/1860754936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 msgr2=0x7efef419f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.870 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 0x7efef419f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 -- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4102e00 msgr2=0x7efef41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4102e00 0x7efef41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 -- 172.21.15.135:0/1860754936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef419fbe0 con 0x7efef4101c00 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.868+0000 7efef3fff640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efee400b450 tx=0x7efee400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.869+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4015020 con 0x7efef4101c00 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.869+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efee4011d70 con 0x7efef4101c00 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.869+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef4074480 con 0x7efef4101c00 2024-09-12T03:11:59.871 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.869+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400f340 con 0x7efef4101c00 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.869+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef4074a10 con 0x7efef4101c00 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.870+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef4076440 con 0x7efef4101c00 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.871+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7efee4018020 con 0x7efef4101c00 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.872+0000 7efef17fa640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 0x7efedc0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.872+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7efee40cd490 con 0x7efef4101c00 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.872+0000 7efef37fe640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 0x7efedc0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:11:59.872 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.872+0000 7efef37fe640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 0x7efedc0a4fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efee80314e0 tx=0x7efee8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:11:59.875 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:11:59.875+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efee401f020 con 0x7efef4101c00 2024-09-12T03:12:00.065 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:00.065+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7efef4062630 con 0x7efedc0a2b20 2024-09-12T03:12:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:00.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:00.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:00.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:00 smithi019 ceph-mon[30953]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:12:00.461 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:12:00.461 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:12:00.461 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:00 smithi135 ceph-mon[35474]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.537 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:00.538 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-mon[34963]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:12:00.899 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:12:00.535+0000 7f87b182c640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T03:12:00.900 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:12:00.539+0000 7f87a964c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi121 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: from='osd.1 ' entity='osd.1' 2024-09-12T03:12:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi121 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: from='osd.1 ' entity='osd.1' 2024-09-12T03:12:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi121 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi121 to 17087M 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: from='osd.1 ' entity='osd.1' 2024-09-12T03:12:01.447 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:02 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:12:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:02 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:12:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:02 smithi135 ceph-mon[35474]: osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] boot 2024-09-12T03:12:02.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:02 smithi135 ceph-mon[35474]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:12:02.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:02.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:02 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:12:02.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:02 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:12:02.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:02 smithi121 ceph-mon[34963]: osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] boot 2024-09-12T03:12:02.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:02 smithi121 ceph-mon[34963]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:12:02.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:02 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:12:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:02 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:12:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:02 smithi019 ceph-mon[30953]: osd.1 [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] boot 2024-09-12T03:12:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:02 smithi019 ceph-mon[30953]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T03:12:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/1318045703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]': finished 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/4290652124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/1318045703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]': finished 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/4290652124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-12T03:12:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/1318045703' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]: dispatch 2024-09-12T03:12:03.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aca96f86-8aca-451b-8be2-0d18ed46afae"}]': finished 2024-09-12T03:12:03.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: osdmap e14: 3 total, 2 up, 3 in 2024-09-12T03:12:03.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:03.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/4290652124' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:04 smithi135 ceph-mon[35474]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:04.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:04 smithi121 ceph-mon[34963]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:04.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:04 smithi019 ceph-mon[30953]: pgmap v60: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:06.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:06 smithi135 ceph-mon[35474]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:06.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:06 smithi121 ceph-mon[34963]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:06.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:06 smithi019 ceph-mon[30953]: pgmap v61: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:08.358 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:08 smithi135 ceph-mon[35474]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:08.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:08 smithi121 ceph-mon[34963]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:08.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:08 smithi019 ceph-mon[30953]: pgmap v62: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:10.524 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:10 smithi135 ceph-mon[35474]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:10.524 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:12:10.524 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:10.524 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:10 smithi135 ceph-mon[35474]: Deploying daemon osd.2 on smithi135 2024-09-12T03:12:10.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:10 smithi121 ceph-mon[34963]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:10.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:12:10.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:10.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:10 smithi121 ceph-mon[34963]: Deploying daemon osd.2 on smithi135 2024-09-12T03:12:10.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:10 smithi019 ceph-mon[30953]: pgmap v63: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:10.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T03:12:10.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:10.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:10 smithi019 ceph-mon[30953]: Deploying daemon osd.2 on smithi135 2024-09-12T03:12:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:12 smithi135 ceph-mon[35474]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:12.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:12 smithi121 ceph-mon[34963]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:12.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:12 smithi019 ceph-mon[30953]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:14 smithi135 ceph-mon[35474]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:14 smithi121 ceph-mon[34963]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:14.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:14 smithi019 ceph-mon[30953]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:16 smithi135 ceph-mon[35474]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:16.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:16 smithi121 ceph-mon[34963]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:16.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:16 smithi019 ceph-mon[30953]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:18.646 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:18 smithi135 ceph-mon[35474]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:18.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:18 smithi121 ceph-mon[34963]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:18 smithi019 ceph-mon[30953]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:20.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:20.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.446 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.446+0000 7efef17fa640 1 -- 172.21.15.135:0/1860754936 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efef4062630 con 0x7efedc0a2b20 2024-09-12T03:12:20.447 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 2 on host 'smithi135' 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 msgr2=0x7efedc0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 0x7efedc0a4fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efee80314e0 tx=0x7efee8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 msgr2=0x7efef4197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efee400b450 tx=0x7efee400b910 comp rx=0 tx=0).stop 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 shutdown_connections 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efef41037c0 0x7efef419f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efedc0a2b20 0x7efedc0a4fe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.449 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efef4102e00 0x7efef41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 --2- 172.21.15.135:0/1860754936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efef4101c00 0x7efef4197fb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 >> 172.21.15.135:0/1860754936 conn(0x7efef40fd390 msgr2=0x7efef41061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 shutdown_connections 2024-09-12T03:12:20.450 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:12:20.449+0000 7efefa98d640 1 -- 172.21.15.135:0/1860754936 wait complete. 2024-09-12T03:12:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:20 smithi135 ceph-mon[35474]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:21.045 DEBUG:teuthology.orchestra.run.smithi135:osd.2> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T03:12:21.048 INFO:tasks.cephadm:Deploying osd.3 on smithi019 with /dev/vg_nvme/lv_3... 2024-09-12T03:12:21.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:12:21.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:20 smithi019 ceph-mon[30953]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:21.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:21.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:21.073 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:21.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:20 smithi121 ceph-mon[34963]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:21.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:21.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:21.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:21.301 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:22.354 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:12:22.196+0000 7f3b94847640 -1 osd.2 0 log_to_monitors true 2024-09-12T03:12:22.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:22 smithi019 ceph-mon[30953]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:22.824 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:22 smithi019 ceph-mon[30953]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:22.825 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:22 smithi019 ceph-mon[30953]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:22 smithi135 ceph-mon[35474]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:22 smithi135 ceph-mon[35474]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:22 smithi135 ceph-mon[35474]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:23.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:22 smithi121 ceph-mon[34963]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:23.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:22 smithi121 ceph-mon[34963]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:23.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:22 smithi121 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T03:12:23.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:23 smithi019 ceph-mon[30953]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:12:23.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:23 smithi019 ceph-mon[30953]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:12:23.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:23.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:23 smithi019 ceph-mon[30953]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:23.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:23 smithi019 ceph-mon[30953]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-mon[35474]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:12:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-mon[35474]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:12:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-mon[35474]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-mon[35474]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:24.045 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:12:23.702+0000 7f3b907ca640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T03:12:24.045 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:12:23.706+0000 7f3b8c5f2640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:12:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:12:24.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:23 smithi121 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T03:12:24.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:23 smithi121 ceph-mon[34963]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T03:12:24.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:24.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:23 smithi121 ceph-mon[34963]: from='osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:24.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:23 smithi121 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:12:24.160 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_3 2024-09-12T03:12:24.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:12:24.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi135 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:24.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:24.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 -- 172.21.15.19:0/4004150962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 msgr2=0x7f6b90101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 --2- 172.21.15.19:0/4004150962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90101910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800aa10 tx=0x7f6b7802b720 comp rx=0 tx=0).stop 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 -- 172.21.15.19:0/4004150962 shutdown_connections 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 --2- 172.21.15.19:0/4004150962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 0x7f6b901086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 --2- 172.21.15.19:0/4004150962 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90101910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 --2- 172.21.15.19:0/4004150962 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 0x7f6b90100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 -- 172.21.15.19:0/4004150962 >> 172.21.15.19:0/4004150962 conn(0x7f6b900fba40 msgr2=0x7f6b900fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 -- 172.21.15.19:0/4004150962 shutdown_connections 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.037+0000 7f6b94e14640 1 -- 172.21.15.19:0/4004150962 wait complete. 2024-09-12T03:12:25.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 Processor -- start 2024-09-12T03:12:25.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 -- start start 2024-09-12T03:12:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 0x7f6b90104220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 0x7f6b90102db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010b6b0 con 0x7f6b90101490 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010b530 con 0x7f6b90101e50 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b94e14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9010b830 con 0x7f6b90100290 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b85d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b85d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35626/0 (socket says 172.21.15.19:35626) 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b85d9b640 1 -- 172.21.15.19:0/2889611399 learned_addr learned my addr 172.21.15.19:0/2889611399 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b8e59c640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 0x7f6b90104220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.038+0000 7f6b8ed9d640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 0x7f6b90102db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 -- 172.21.15.19:0/2889611399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 msgr2=0x7f6b90104220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 0x7f6b90104220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 -- 172.21.15.19:0/2889611399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 msgr2=0x7f6b90102db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 0x7f6b90102db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 -- 172.21.15.19:0/2889611399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b90103670 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b85d9b640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005940 tx=0x7f6b78030e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7803f070 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b78032400 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b901a20d0 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78036040 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.039+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b901a2580 con 0x7f6b90101490 2024-09-12T03:12:25.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.040+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b90100690 con 0x7f6b90101490 2024-09-12T03:12:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.041+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f6b78042070 con 0x7f6b90101490 2024-09-12T03:12:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.042+0000 7f6b87fff640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 0x7f6b7c0a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.042+0000 7f6b8e59c640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 0x7f6b7c0a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.042+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2851+0+0 (secure 0 0 0) 0x7f6b780ee1a0 con 0x7f6b90101490 2024-09-12T03:12:25.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.042+0000 7f6b8e59c640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 0x7f6b7c0a4fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b901012f0 tx=0x7f6b8000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi135 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:25.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.046+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b780bc7c0 con 0x7f6b90101490 2024-09-12T03:12:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi135 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi135 to 5815M 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:25.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:25.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:25.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:25.232+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f6b901081e0 con 0x7f6b7c0a2af0 2024-09-12T03:12:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] boot 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:25.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] boot 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:26.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:26.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:26.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: osd.2 [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] boot 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: osdmap e17: 3 total, 3 up, 3 in 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:26.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:26.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:26 smithi019 ceph-mon[30953]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:26.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:26 smithi019 ceph-mon[30953]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:26.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:26.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:26 smithi019 ceph-mon[30953]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:12:26.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:26 smithi135 ceph-mon[35474]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:26 smithi135 ceph-mon[35474]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:26 smithi135 ceph-mon[35474]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:12:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:27.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:26 smithi121 ceph-mon[34963]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-09-12T03:12:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:26 smithi121 ceph-mon[34963]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:26 smithi121 ceph-mon[34963]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T03:12:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:12:27.664 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49677]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:27.664 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:27.665 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49677]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:27.665 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49677]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:27.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:27.727+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6b780b8730 con 0x7f6b90101490 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]: dispatch 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]': finished 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:12:27.978 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]: dispatch 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]': finished 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:12:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:28.055 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:28.056 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:28.056 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49705]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:28.056 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:12:27 smithi019 sudo[49705]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: osdmap e19: 3 total, 3 up, 3 in 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]: dispatch 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1304444527' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4eccbc69-1f18-4b12-be71-e115af1f187f"}]': finished 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T03:12:28.056 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:28.234 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:27 smithi121 sudo[43294]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:28.234 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:27 smithi121 sudo[43294]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:28.234 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:27 smithi121 sudo[43294]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:28.234 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:27 smithi121 sudo[43294]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:28.649 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:28 smithi121 sudo[43297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:28.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:28 smithi121 sudo[43297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:28.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:28 smithi121 sudo[43297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:28.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:12:28 smithi121 sudo[43297]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:28.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:28 smithi135 ceph-mon[35474]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:28.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:28 smithi135 ceph-mon[35474]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:12:28.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:28.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:28 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2365930567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:28.758 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:28.758 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44184]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:28.758 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44184]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:28.758 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44184]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:28.758 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44184]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:29.044 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44187]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:29.044 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44187]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:29.044 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44187]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:29.044 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:12:28 smithi135 sudo[44187]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:29.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:28 smithi121 ceph-mon[34963]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:29.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:28 smithi121 ceph-mon[34963]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:12:29.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:29.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:28 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2365930567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:29.152 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:29.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:28 smithi019 ceph-mon[30953]: pgmap v77: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:29.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:28 smithi019 ceph-mon[30953]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T03:12:29.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:29.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:28 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2365930567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:29.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[49969]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[49969]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[49969]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:29.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[49969]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:30.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[50155]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:30.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[50155]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:30.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[50155]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:30.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:29 smithi019 sudo[50155]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:30.183 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:29 smithi135 sudo[44190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:30.183 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:29 smithi135 sudo[44190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:30.183 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:29 smithi135 sudo[44190]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:30.183 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:29 smithi135 sudo[44190]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 sudo[44193]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 sudo[44193]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 sudo[44193]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 sudo[44193]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:30.682 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T03:12:30.682 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:30.682 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43300]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:30.683 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43300]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:30 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 sudo[43303]: pam_unix(sudo:session): session closed for user root 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:31.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:30 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:31.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:31.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:31.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:30 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:32.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:32.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:31 smithi135 ceph-mon[35474]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:31 smithi121 ceph-mon[34963]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: pgmap v79: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:32.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T03:12:32.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T03:12:32.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T03:12:32.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T03:12:32.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:31 smithi019 ceph-mon[30953]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T03:12:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:32 smithi019 ceph-mon[30953]: pgmap v80: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:32 smithi135 ceph-mon[35474]: pgmap v80: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:33.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:32 smithi121 ceph-mon[34963]: pgmap v80: 1 pgs: 1 active+clean; 449 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:34.850 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:34 smithi019 ceph-mon[30953]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-12T03:12:35.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:34 smithi135 ceph-mon[35474]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-12T03:12:35.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:34 smithi121 ceph-mon[34963]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 20 op/s 2024-09-12T03:12:36.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:36 smithi019 ceph-mon[30953]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-12T03:12:36.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:12:36.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:36.806 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:36 smithi019 ceph-mon[30953]: Deploying daemon osd.3 on smithi019 2024-09-12T03:12:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:36 smithi135 ceph-mon[35474]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-12T03:12:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:12:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:36 smithi135 ceph-mon[35474]: Deploying daemon osd.3 on smithi019 2024-09-12T03:12:37.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:36 smithi121 ceph-mon[34963]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 15 op/s 2024-09-12T03:12:37.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T03:12:37.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:37.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:36 smithi121 ceph-mon[34963]: Deploying daemon osd.3 on smithi019 2024-09-12T03:12:39.012 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:38 smithi019 ceph-mon[30953]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-12T03:12:39.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:38 smithi135 ceph-mon[35474]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-12T03:12:39.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:38 smithi121 ceph-mon[34963]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-09-12T03:12:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:40 smithi135 ceph-mon[35474]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-09-12T03:12:41.068 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:40 smithi019 ceph-mon[30953]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-09-12T03:12:41.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:40 smithi121 ceph-mon[34963]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.1 KiB/s rd, 156 KiB/s wr, 10 op/s 2024-09-12T03:12:43.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:42 smithi019 ceph-mon[30953]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:43.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:43.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:43.014 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:43.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:42 smithi135 ceph-mon[35474]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:43.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:43.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:43.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:43.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:42 smithi121 ceph-mon[34963]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:43.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:43.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:43.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:44 smithi135 ceph-mon[35474]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:45.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:44 smithi121 ceph-mon[34963]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:44 smithi019 ceph-mon[30953]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-09-12T03:12:45.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.850+0000 7f6b87fff640 1 -- 172.21.15.19:0/2889611399 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6b901081e0 con 0x7f6b7c0a2af0 2024-09-12T03:12:45.851 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 3 on host 'smithi019' 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 msgr2=0x7f6b7c0a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 0x7f6b7c0a4fb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6b901012f0 tx=0x7f6b8000b390 comp rx=0 tx=0).stop 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 msgr2=0x7f6b90102870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005940 tx=0x7f6b78030e70 comp rx=0 tx=0).stop 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 shutdown_connections 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b90101e50 0x7f6b90102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b7c0a2af0 0x7f6b7c0a4fb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b90101490 0x7f6b90102870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 --2- 172.21.15.19:0/2889611399 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b90100290 0x7f6b90104220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:45.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.852+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 >> 172.21.15.19:0/2889611399 conn(0x7f6b900fba40 msgr2=0x7f6b90106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:45.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.853+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 shutdown_connections 2024-09-12T03:12:45.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:12:45.853+0000 7f6b94e14640 1 -- 172.21.15.19:0/2889611399 wait complete. 2024-09-12T03:12:46.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:46.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:46.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:46.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:46.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:46.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:46.226 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:46.508 DEBUG:teuthology.orchestra.run.smithi019:osd.3> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T03:12:46.510 INFO:tasks.cephadm:Deploying osd.4 on smithi121 with /dev/vg_nvme/lv_3... 2024-09-12T03:12:46.510 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:12:46.768 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:12:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:46 smithi121 ceph-mon[34963]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:46 smithi019 ceph-mon[30953]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.171 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:46 smithi135 ceph-mon[35474]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:47.171 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:12:47.171 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.171 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:47.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:47 smithi121 ceph-mon[34963]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:12:47.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:47 smithi019 ceph-mon[30953]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:12:47.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:12:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:12:47.615+0000 7fac4a86a640 -1 osd.3 0 log_to_monitors true 2024-09-12T03:12:48.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:47 smithi135 ceph-mon[35474]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T03:12:48.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:48 smithi121 ceph-mon[34963]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:48.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:48 smithi121 ceph-mon[34963]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:12:48.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:48 smithi121 ceph-mon[34963]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:12:48.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:48 smithi121 ceph-mon[34963]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:12:48.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:48 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:48.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:12:48.842+0000 7fac467ed640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T03:12:48.915 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:12:48.849+0000 7fac42615640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:12:48.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-mon[30953]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:48.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-mon[30953]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:12:48.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-mon[30953]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:12:48.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-mon[30953]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:12:48.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:48 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:48 smithi135 ceph-mon[35474]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:48 smithi135 ceph-mon[35474]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T03:12:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:48 smithi135 ceph-mon[35474]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T03:12:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:48 smithi135 ceph-mon[35474]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:12:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:48 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:49.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:12:49.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_3 2024-09-12T03:12:49.726 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:12:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:12:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:12:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:49.889 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] boot 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T03:12:49.890 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] boot 2024-09-12T03:12:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T03:12:50.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.235+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/2633556960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 msgr2=0x7fdd98108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.235+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/2633556960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd98108f20 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ca0 tx=0x7fdd9002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/2633556960 shutdown_connections 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/2633556960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd98108f20 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/2633556960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98101cd0 0x7fdd98102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/2633556960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98100ad0 0x7fdd98100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/2633556960 >> 172.21.15.121:0/2633556960 conn(0x7fdd980fc280 msgr2=0x7fdd980fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/2633556960 shutdown_connections 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/2633556960 wait complete. 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 Processor -- start 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 -- start start 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.236+0000 7fdd9d4fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 0x7fdd9806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98101cd0 0x7fdd9819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c750 con 0x7fdd98100ad0 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c5d0 con 0x7fdd98102690 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9810c8d0 con 0x7fdd98101cd0 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:50.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55402/0 (socket says 172.21.15.121:55402) 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 -- 172.21.15.121:0/3749807753 learned_addr learned my addr 172.21.15.121:0/3749807753 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd967fc640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98101cd0 0x7fdd9819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd96ffd640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 0x7fdd9806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 -- 172.21.15.121:0/3749807753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98101cd0 msgr2=0x7fdd9819cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98101cd0 0x7fdd9819cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 -- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 msgr2=0x7fdd9806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 0x7fdd9806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 -- 172.21.15.121:0/3749807753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9819faa0 con 0x7fdd98102690 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd96ffd640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 0x7fdd9806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd977fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098d0 tx=0x7fdd9002f840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd9002fc30 con 0x7fdd98102690 2024-09-12T03:12:50.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9819fcd0 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd90003e50 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd90004810 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.237+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a0210 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.238+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd98100f00 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.240+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdd90004b40 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.240+0000 7fdd7ffff640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 0x7fdd800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.240+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3702+0+0 (secure 0 0 0) 0x7fdd900ef350 con 0x7fdd98102690 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.241+0000 7fdd96ffd640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 0x7fdd800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.241+0000 7fdd96ffd640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 0x7fdd800a5040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd880058c0 tx=0x7fdd88005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:12:50.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.244+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd98100f00 con 0x7fdd98102690 2024-09-12T03:12:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:12:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:12:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: osd.3 [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] boot 2024-09-12T03:12:50.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: osdmap e23: 4 total, 4 up, 4 in 2024-09-12T03:12:50.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T03:12:50.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:12:50.424+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdd98062630 con 0x7fdd800a2b80 2024-09-12T03:12:51.128 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:12:51.128 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:12:51.128 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:51.128 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi019 2024-09-12T03:12:51.128 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-12T03:12:51.129 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:51.129 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:51.129 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:51.129 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:50 smithi121 ceph-mon[34963]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:12:51.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:12:51.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:12:51.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:51.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi019 2024-09-12T03:12:51.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-12T03:12:51.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:51.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:51.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:51.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:50 smithi019 ceph-mon[30953]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi019 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi019 to 8546M 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:12:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:12:51.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:50 smithi135 ceph-mon[35474]: osdmap e24: 4 total, 4 up, 4 in 2024-09-12T03:12:52.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:51 smithi121 ceph-mon[34963]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:52.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:51 smithi121 ceph-mon[34963]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:12:52.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:51 smithi019 ceph-mon[30953]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:52.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:51 smithi019 ceph-mon[30953]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:12:52.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:51 smithi135 ceph-mon[35474]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:12:52.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:51 smithi135 ceph-mon[35474]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T03:12:52.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:52 smithi121 ceph-mon[34963]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:12:52.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:52 smithi121 ceph-mon[34963]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:12:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:52 smithi019 ceph-mon[30953]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:12:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:52 smithi019 ceph-mon[30953]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:12:53.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:52 smithi135 ceph-mon[35474]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:12:53.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:52 smithi135 ceph-mon[35474]: osdmap e26: 4 total, 4 up, 4 in 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/1528103540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]': finished 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:12:54.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/1296960935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/1528103540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]': finished 2024-09-12T03:12:54.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:12:54.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:12:54.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:53 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/1296960935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:54.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/1528103540' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]: dispatch 2024-09-12T03:12:54.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf3b091d-b06c-466a-b160-49acb3bb2917"}]': finished 2024-09-12T03:12:54.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T03:12:54.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:12:54.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:53 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/1296960935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:12:55.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:54 smithi135 ceph-mon[35474]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:12:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:54 smithi121 ceph-mon[34963]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:12:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:54 smithi019 ceph-mon[30953]: pgmap v98: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:12:57.238 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:56 smithi135 ceph-mon[35474]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:12:57.274 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:56 smithi121 ceph-mon[34963]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:12:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:56 smithi019 ceph-mon[30953]: pgmap v99: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:12:59.102 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:12:58 smithi121 ceph-mon[34963]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:12:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:12:58 smithi135 ceph-mon[35474]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:12:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:12:58 smithi019 ceph-mon[30953]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:13:01.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:00 smithi135 ceph-mon[35474]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:13:01.386 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:00 smithi121 ceph-mon[34963]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:13:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:00 smithi019 ceph-mon[30953]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:13:02.234 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:13:02.234 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:02.235 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:01 smithi121 ceph-mon[34963]: Deploying daemon osd.4 on smithi121 2024-09-12T03:13:02.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:13:02.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:02.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:01 smithi135 ceph-mon[35474]: Deploying daemon osd.4 on smithi121 2024-09-12T03:13:02.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T03:13:02.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:02.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:01 smithi019 ceph-mon[30953]: Deploying daemon osd.4 on smithi121 2024-09-12T03:13:03.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:02 smithi121 ceph-mon[34963]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:02 smithi135 ceph-mon[35474]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:03.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:02 smithi019 ceph-mon[30953]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:04 smithi121 ceph-mon[34963]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:05.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:04 smithi135 ceph-mon[35474]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:04 smithi019 ceph-mon[30953]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:06 smithi135 ceph-mon[35474]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:07.327 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:06 smithi121 ceph-mon[34963]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:06 smithi019 ceph-mon[30953]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:08.290 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:08.291 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:08.291 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:08.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:08.315 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:08.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:08 smithi135 ceph-mon[35474]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:09.313 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:08 smithi121 ceph-mon[34963]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:08 smithi019 ceph-mon[30953]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:11.280 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:11.281 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.281 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.281 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.281 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:11.281 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:11.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.594+0000 7fdd7ffff640 1 -- 172.21.15.121:0/3749807753 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdd98062630 con 0x7fdd800a2b80 2024-09-12T03:13:11.595 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 4 on host 'smithi121' 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 msgr2=0x7fdd800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 0x7fdd800a5040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdd880058c0 tx=0x7fdd88005ec0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 msgr2=0x7fdd9819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdd900098d0 tx=0x7fdd9002f840 comp rx=0 tx=0).stop 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 shutdown_connections 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd98102690 0x7fdd9819f190 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd98101cd0 0x7fdd9819cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd800a2b80 0x7fdd800a5040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 --2- 172.21.15.121:0/3749807753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd98100ad0 0x7fdd9806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.596+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 >> 172.21.15.121:0/3749807753 conn(0x7fdd980fc280 msgr2=0x7fdd98104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.597+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 shutdown_connections 2024-09-12T03:13:11.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:13:11.597+0000 7fdd9d4fe640 1 -- 172.21.15.121:0/3749807753 wait complete. 2024-09-12T03:13:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:11 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:11 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:11 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.376 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:11 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:12.376 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:11 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.376 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:11 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.402 DEBUG:teuthology.orchestra.run.smithi121:osd.4> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T03:13:12.405 INFO:tasks.cephadm:Deploying osd.5 on smithi135 with /dev/vg_nvme/lv_3... 2024-09-12T03:13:12.405 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T03:13:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:11 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:12.656 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:13.030 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:12 smithi135 ceph-mon[35474]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:12 smithi121 ceph-mon[34963]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:12 smithi019 ceph-mon[30953]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:13.894 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:13:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:13:13.618+0000 7f8ea6dcc640 -1 osd.4 0 log_to_monitors true 2024-09-12T03:13:14.086 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:13 smithi135 ceph-mon[35474]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:14.086 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:13 smithi135 ceph-mon[35474]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:14.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:13 smithi121 ceph-mon[34963]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:14.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:13 smithi121 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:14.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:13 smithi019 ceph-mon[30953]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:14.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:13 smithi019 ceph-mon[30953]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T03:13:15.240 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:13:15.009+0000 7f8ea2d4f640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T03:13:15.240 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:13:15.017+0000 7f8e9eb77640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:13:15.240 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:15.240 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:13:15.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:13:15.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:15.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:15 smithi121 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.285 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:15.286 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:13:15.286 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:13:15.286 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.286 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:15.286 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:15 smithi135 ceph-mon[35474]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.289 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:13:15.318 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_3 2024-09-12T03:13:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T03:13:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T03:13:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: from='osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:15 smithi019 ceph-mon[30953]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:13:15.570 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:13:16.090 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 -- 172.21.15.135:0/3500018860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 msgr2=0x7fa33c102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 --2- 172.21.15.135:0/3500018860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c102150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa330009900 tx=0x7fa33002a410 comp rx=0 tx=0).stop 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 -- 172.21.15.135:0/3500018860 shutdown_connections 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 --2- 172.21.15.135:0/3500018860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 0x7fa33c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 --2- 172.21.15.135:0/3500018860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c102150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 --2- 172.21.15.135:0/3500018860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 -- 172.21.15.135:0/3500018860 >> 172.21.15.135:0/3500018860 conn(0x7fa33c0fc280 msgr2=0x7fa33c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 -- 172.21.15.135:0/3500018860 shutdown_connections 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.052+0000 7fa3415c8640 1 -- 172.21.15.135:0/3500018860 wait complete. 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 Processor -- start 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 -- start start 2024-09-12T03:13:16.091 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 0x7fa33c1a1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10c750 con 0x7fa33c102690 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10c5d0 con 0x7fa33c100ad0 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa3415c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c10c8d0 con 0x7fa33c101cd0 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa33a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa33a7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:57518/0 (socket says 172.21.15.135:57518) 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa33a7fc640 1 -- 172.21.15.135:0/2859866314 learned_addr learned my addr 172.21.15.135:0/2859866314 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.053+0000 7fa33b7fe640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 0x7fa33c1a1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33affd640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.092 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 -- 172.21.15.135:0/2859866314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 msgr2=0x7fa33c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 -- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 msgr2=0x7fa33c1a1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 0x7fa33c1a1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 -- 172.21.15.135:0/2859866314 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c1a1d20 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33affd640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa33a7fc640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33003f030 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa330003ce0 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c1a1fb0 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa330035070 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.054+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c1a2400 con 0x7fa33c101cd0 2024-09-12T03:13:16.093 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.055+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c107ec0 con 0x7fa33c101cd0 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.057+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa330042030 con 0x7fa33c101cd0 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.058+0000 7fa323fff640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 0x7fa3240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.058+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4101+0+0 (secure 0 0 0) 0x7fa3300ee9c0 con 0x7fa33c101cd0 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.058+0000 7fa33affd640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 0x7fa3240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.058+0000 7fa33affd640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 0x7fa3240a5040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa33c06b050 tx=0x7fa32c005e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:16.094 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.062+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa3300f0e20 con 0x7fa33c101cd0 2024-09-12T03:13:16.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:13:16.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:13:16.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.094 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:16.095 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.244 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:16.244+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa33c062630 con 0x7fa3240a2b80 2024-09-12T03:13:16.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:13:16.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:13:16.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:16.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:13:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T03:13:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:13:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:13:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi121 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] boot 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: osdmap e30: 5 total, 5 up, 5 in 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:17.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:17 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi121 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] boot 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: osdmap e30: 5 total, 5 up, 5 in 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:17.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:17.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:17 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:13:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:13:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi121 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi121 to 8543M 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: osd.4 [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] boot 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: osdmap e30: 5 total, 5 up, 5 in 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:17 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:18.245 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:18 smithi135 ceph-mon[35474]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:18.245 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:18 smithi135 ceph-mon[35474]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T03:13:18.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:18 smithi121 ceph-mon[34963]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:18.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:18 smithi121 ceph-mon[34963]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T03:13:18.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:18 smithi019 ceph-mon[30953]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:18.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:18 smithi019 ceph-mon[30953]: osdmap e31: 5 total, 5 up, 5 in 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/1187592918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]': finished 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T03:13:19.157 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:19.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/1187592918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]': finished 2024-09-12T03:13:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T03:13:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/1187592918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]: dispatch 2024-09-12T03:13:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ac0c634-1484-497b-bee3-58e37c79b212"}]': finished 2024-09-12T03:13:19.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: osdmap e32: 6 total, 5 up, 6 in 2024-09-12T03:13:19.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:20 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/1377571630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:20.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:20 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/1377571630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:20.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:20 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/1377571630' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:21 smithi135 ceph-mon[35474]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:21.321 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:21 smithi121 ceph-mon[34963]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:21.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:21 smithi019 ceph-mon[30953]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 141 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:23.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:23 smithi135 ceph-mon[35474]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:23 smithi121 ceph-mon[34963]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:23 smithi019 ceph-mon[30953]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:25.139 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:25 smithi135 ceph-mon[35474]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:25 smithi121 ceph-mon[34963]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:25 smithi019 ceph-mon[30953]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:27 smithi121 ceph-mon[34963]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:27 smithi019 ceph-mon[30953]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:27.495 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:27 smithi135 ceph-mon[35474]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:28.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:13:28.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:28.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:28 smithi121 ceph-mon[34963]: Deploying daemon osd.5 on smithi135 2024-09-12T03:13:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:13:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:28 smithi019 ceph-mon[30953]: Deploying daemon osd.5 on smithi135 2024-09-12T03:13:28.510 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T03:13:28.510 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:28.510 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:28 smithi135 ceph-mon[35474]: Deploying daemon osd.5 on smithi135 2024-09-12T03:13:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:29 smithi135 ceph-mon[35474]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:29 smithi121 ceph-mon[34963]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:29 smithi019 ceph-mon[30953]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:31 smithi135 ceph-mon[35474]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:31 smithi121 ceph-mon[34963]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:31 smithi019 ceph-mon[30953]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:33.187 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:33 smithi135 ceph-mon[35474]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:33 smithi121 ceph-mon[34963]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:33 smithi019 ceph-mon[30953]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:34.227 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:34.227 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:34.227 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:34.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:34.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:34.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:35.194 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:35 smithi135 ceph-mon[35474]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:35 smithi121 ceph-mon[34963]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:35 smithi019 ceph-mon[30953]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:37.415 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:37.458 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.458+0000 7fa323fff640 1 -- 172.21.15.135:0/2859866314 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa33c062630 con 0x7fa3240a2b80 2024-09-12T03:13:37.458 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 5 on host 'smithi135' 2024-09-12T03:13:37.460 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 msgr2=0x7fa3240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 0x7fa3240a5040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa33c06b050 tx=0x7fa32c005e90 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 msgr2=0x7fa33c19f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa330009230 tx=0x7fa330004360 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 shutdown_connections 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa3240a2b80 0x7fa3240a5040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa33c102690 0x7fa33c1a1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa33c101cd0 0x7fa33c19f000 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 --2- 172.21.15.135:0/2859866314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c100ad0 0x7fa33c06ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.460+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 >> 172.21.15.135:0/2859866314 conn(0x7fa33c0fc280 msgr2=0x7fa33c105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:37.461 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.461+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 shutdown_connections 2024-09-12T03:13:37.462 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:13:37.461+0000 7fa3415c8640 1 -- 172.21.15.135:0/2859866314 wait complete. 2024-09-12T03:13:38.137 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:38.138 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.138 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.165 DEBUG:teuthology.orchestra.run.smithi135:osd.5> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T03:13:38.168 INFO:tasks.cephadm:Deploying osd.6 on smithi019 with /dev/vg_nvme/lv_2... 2024-09-12T03:13:38.168 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:13:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:38.436 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:39.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:39 smithi019 ceph-mon[30953]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:39 smithi121 ceph-mon[34963]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:39.449 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:39 smithi135 ceph-mon[35474]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:39.450 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:13:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:13:39.300+0000 7f304c01f640 -1 osd.5 0 log_to_monitors true 2024-09-12T03:13:40.338 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:40 smithi135 ceph-mon[35474]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.338 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:40 smithi135 ceph-mon[35474]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:40 smithi121 ceph-mon[34963]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:40 smithi121 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:40 smithi019 ceph-mon[30953]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:40 smithi019 ceph-mon[30953]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T03:13:40.785 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:13:40.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_2 2024-09-12T03:13:41.088 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:13:41.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:13:41.104+0000 7f3047fa2640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T03:13:41.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:13:41.111+0000 7f3043dca640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:41.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T03:13:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T03:13:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:13:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:13:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:13:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.546+0000 7fb7a36ef640 1 -- 172.21.15.19:0/44321165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c100ad0 msgr2=0x7fb79c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.546+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/44321165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c100ad0 0x7fb79c100ed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb798009900 tx=0x7fb79802a410 comp rx=0 tx=0).stop 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.546+0000 7fb7a36ef640 1 -- 172.21.15.19:0/44321165 shutdown_connections 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.546+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/44321165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c102690 0x7fb79c108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.546+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/44321165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 0x7fb79c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/44321165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c100ad0 0x7fb79c100ed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 -- 172.21.15.19:0/44321165 >> 172.21.15.19:0/44321165 conn(0x7fb79c0fc280 msgr2=0x7fb79c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 -- 172.21.15.19:0/44321165 shutdown_connections 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 -- 172.21.15.19:0/44321165 wait complete. 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 Processor -- start 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.547+0000 7fb7a36ef640 1 -- start start 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c100ad0 0x7fb79c104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 0x7fb79c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10c750 con 0x7fb79c102690 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54374/0 (socket says 172.21.15.19:54374) 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 -- 172.21.15.19:0/3492350221 learned_addr learned my addr 172.21.15.19:0/3492350221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:13:41.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10c5d0 con 0x7fb79c100ad0 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb79c10c8d0 con 0x7fb79c101cd0 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a0c8a640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 0x7fb79c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a148b640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c100ad0 0x7fb79c104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 -- 172.21.15.19:0/3492350221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 msgr2=0x7fb79c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 0x7fb79c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 -- 172.21.15.19:0/3492350221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c100ad0 msgr2=0x7fb79c104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c100ad0 0x7fb79c104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.548+0000 7fb7a1c8c640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79c103eb0 con 0x7fb79c102690 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb7a1c8c640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb79400eaa0 tx=0x7fb79400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79400cdb0 con 0x7fb79c102690 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79c1a20c0 con 0x7fb79c102690 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb794005020 con 0x7fb79c102690 2024-09-12T03:13:41.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7940116b0 con 0x7fb79c102690 2024-09-12T03:13:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.549+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb79c1a2580 con 0x7fb79c102690 2024-09-12T03:13:41.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.550+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb79c1a27b0 con 0x7fb79c102690 2024-09-12T03:13:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.551+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb794005560 con 0x7fb79c102690 2024-09-12T03:13:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.552+0000 7fb78e7fc640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 0x7fb7880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:13:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.552+0000 7fb7a148b640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 0x7fb7880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:13:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.552+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7fb7940ce2d0 con 0x7fb79c102690 2024-09-12T03:13:41.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.553+0000 7fb7a148b640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 0x7fb7880a5040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb798005ab0 tx=0x7fb798000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:13:41.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.557+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb794016030 con 0x7fb79c102690 2024-09-12T03:13:41.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:13:41.742+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb79c1a2980 con 0x7fb7880a2b80 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi135 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:42.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi135 2024-09-12T03:13:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-12T03:13:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:13:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:13:42.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:42.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:42.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi135 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi135 to 2907M 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:42.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:13:42.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:13:42.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:13:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:13:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:43.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] boot 2024-09-12T03:13:43.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T03:13:43.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] boot 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T03:13:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: osd.5 [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] boot 2024-09-12T03:13:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: osdmap e35: 6 total, 6 up, 6 in 2024-09-12T03:13:43.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T03:13:44.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:44 smithi121 ceph-mon[34963]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:13:44.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:44 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]: dispatch 2024-09-12T03:13:44.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:44 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]': finished 2024-09-12T03:13:44.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:44 smithi121 ceph-mon[34963]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T03:13:44.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:44 smithi019 ceph-mon[30953]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:13:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:44 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]: dispatch 2024-09-12T03:13:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:44 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]': finished 2024-09-12T03:13:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:44 smithi019 ceph-mon[30953]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T03:13:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:44 smithi135 ceph-mon[35474]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T03:13:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:44 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]: dispatch 2024-09-12T03:13:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:44 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1510383016' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "10489254-1ec9-4c5d-acf1-abc03c7e1167"}]': finished 2024-09-12T03:13:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:44 smithi135 ceph-mon[35474]: osdmap e37: 7 total, 6 up, 7 in 2024-09-12T03:13:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:45.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:45 smithi121 ceph-mon[34963]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:45.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:45 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2927593736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:45.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:45 smithi121 ceph-mon[34963]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T03:13:45.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:45 smithi019 ceph-mon[30953]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:45 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2927593736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:45 smithi019 ceph-mon[30953]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T03:13:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:45.448 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:45 smithi135 ceph-mon[35474]: pgmap v132: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:45.448 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:45 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2927593736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:13:45.448 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:45 smithi135 ceph-mon[35474]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T03:13:45.448 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:46 smithi135 ceph-mon[35474]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:46 smithi135 ceph-mon[35474]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:13:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:47.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:46 smithi121 ceph-mon[34963]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:46 smithi121 ceph-mon[34963]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:13:47.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:46 smithi019 ceph-mon[30953]: pgmap v135: 1 pgs: 1 remapped+peering; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:13:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:46 smithi019 ceph-mon[30953]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T03:13:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:13:49.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:48 smithi019 ceph-mon[30953]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T03:13:49.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:48 smithi135 ceph-mon[35474]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T03:13:49.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:48 smithi121 ceph-mon[34963]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-12T03:13:51.022 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:50 smithi019 ceph-mon[30953]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:13:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:50 smithi135 ceph-mon[35474]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:13:51.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:50 smithi121 ceph-mon[34963]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:13:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:52 smithi135 ceph-mon[35474]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T03:13:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:52 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:13:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:52 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:53.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:52 smithi019 ceph-mon[30953]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T03:13:53.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:52 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:13:53.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:52 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:53.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:52 smithi121 ceph-mon[34963]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-12T03:13:53.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:52 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T03:13:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:52 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:13:54.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:53 smithi135 ceph-mon[35474]: Deploying daemon osd.6 on smithi019 2024-09-12T03:13:54.078 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:53 smithi019 ceph-mon[30953]: Deploying daemon osd.6 on smithi019 2024-09-12T03:13:54.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:53 smithi121 ceph-mon[34963]: Deploying daemon osd.6 on smithi019 2024-09-12T03:13:55.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:54 smithi135 ceph-mon[35474]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:13:55.098 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:54 smithi019 ceph-mon[30953]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:13:55.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:54 smithi121 ceph-mon[34963]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-12T03:13:57.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:56 smithi135 ceph-mon[35474]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:13:57.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:56 smithi019 ceph-mon[30953]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:13:57.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:56 smithi121 ceph-mon[34963]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T03:13:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:58 smithi135 ceph-mon[35474]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:13:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:13:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:58 smithi019 ceph-mon[30953]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:13:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:59.145 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:13:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:59.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:58 smithi121 ceph-mon[34963]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-12T03:13:59.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:13:59.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:13:59.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:13:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:00.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:00 smithi019 ceph-mon[30953]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:00 smithi135 ceph-mon[35474]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:01.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:00 smithi121 ceph-mon[34963]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:01.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.919+0000 7fb78e7fc640 1 -- 172.21.15.19:0/3492350221 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb79c1a2980 con 0x7fb7880a2b80 2024-09-12T03:14:01.919 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 6 on host 'smithi019' 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 msgr2=0x7fb7880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 0x7fb7880a5040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb798005ab0 tx=0x7fb798000bc0 comp rx=0 tx=0).stop 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 msgr2=0x7fb79c1035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb79400eaa0 tx=0x7fb79400ef60 comp rx=0 tx=0).stop 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 shutdown_connections 2024-09-12T03:14:01.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7880a2b80 0x7fb7880a5040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb79c102690 0x7fb79c1035f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb79c101cd0 0x7fb79c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 --2- 172.21.15.19:0/3492350221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb79c100ad0 0x7fb79c104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 >> 172.21.15.19:0/3492350221 conn(0x7fb79c0fc280 msgr2=0x7fb79c069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 shutdown_connections 2024-09-12T03:14:01.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:01.922+0000 7fb7a36ef640 1 -- 172.21.15.19:0/3492350221 wait complete. 2024-09-12T03:14:02.600 DEBUG:teuthology.orchestra.run.smithi019:osd.6> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T03:14:02.602 INFO:tasks.cephadm:Deploying osd.7 on smithi121 with /dev/vg_nvme/lv_2... 2024-09-12T03:14:02.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:14:02.877 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:14:03.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:02 smithi019 ceph-mon[30953]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:03.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.228 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:02 smithi121 ceph-mon[34963]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:03.229 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.229 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.229 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.229 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:02 smithi135 ceph-mon[35474]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:04.387 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:14:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:14:04.249+0000 7f35f59f4640 -1 osd.6 0 log_to_monitors true 2024-09-12T03:14:05.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:04 smithi135 ceph-mon[35474]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:05.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:04 smithi135 ceph-mon[35474]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:14:05.349 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:04 smithi121 ceph-mon[34963]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:05.349 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:04 smithi121 ceph-mon[34963]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:14:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:04 smithi019 ceph-mon[30953]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:04 smithi019 ceph-mon[30953]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T03:14:05.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:14:05.673 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_2 2024-09-12T03:14:05.915 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:14:06.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:14:05.924+0000 7f35f1977640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T03:14:06.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:14:05.936+0000 7f35ed79f640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.184 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.185 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:06.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 -- 172.21.15.121:0/3317437995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 msgr2=0x7fc588102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 --2- 172.21.15.121:0/3317437995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588102000 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc578009900 tx=0x7fc57802a410 comp rx=0 tx=0).stop 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 -- 172.21.15.121:0/3317437995 shutdown_connections 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 --2- 172.21.15.121:0/3317437995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 --2- 172.21.15.121:0/3317437995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc588103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 --2- 172.21.15.121:0/3317437995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588102000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 -- 172.21.15.121:0/3317437995 >> 172.21.15.121:0/3317437995 conn(0x7fc5880fd390 msgr2=0x7fc5880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 -- 172.21.15.121:0/3317437995 shutdown_connections 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.394+0000 7fc58d661640 1 -- 172.21.15.121:0/3317437995 wait complete. 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 Processor -- start 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 -- start start 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc5881983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58819f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810d860 con 0x7fc588102e00 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810d6e0 con 0x7fc588101c00 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc58d661640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc58810d9e0 con 0x7fc5881037c0 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34408/0 (socket says 172.21.15.121:34408) 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 -- 172.21.15.121:0/3832791834 learned_addr learned my addr 172.21.15.121:0/3832791834 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc5867fc640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc5881983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 -- 172.21.15.121:0/3832791834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 msgr2=0x7fc58819f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc5877fe640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58819f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58819f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 -- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 msgr2=0x7fc5881983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:06.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc5881983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc586ffd640 1 -- 172.21.15.121:0/3832791834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819fab0 con 0x7fc588101c00 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc5867fc640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc5881983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.395+0000 7fc5877fe640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc586ffd640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc578031610 tx=0x7fc5780043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57803f030 con 0x7fc588101c00 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc578003ce0 con 0x7fc588101c00 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578035070 con 0x7fc588101c00 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819fd40 con 0x7fc588101c00 2024-09-12T03:14:06.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.396+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881a0170 con 0x7fc588101c00 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.397+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc588108fa0 con 0x7fc588101c00 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.398+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc578042030 con 0x7fc588101c00 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.399+0000 7fc56ffff640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 0x7fc5700a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.399+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4685+0+0 (secure 0 0 0) 0x7fc5780ee890 con 0x7fc588101c00 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.399+0000 7fc5867fc640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 0x7fc5700a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:06.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.399+0000 7fc5867fc640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 0x7fc5700a5070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0058c0 tx=0x7fc57c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:06.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.403+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc5780bc620 con 0x7fc588101c00 2024-09-12T03:14:06.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:06.576+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc588062630 con 0x7fc5700a2bb0 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi019 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.241 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:07.242 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:07.242 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:07.242 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:07.242 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.242 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:06 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi019 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:07.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:06 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi019 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi019 to 5697M 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:07.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:06 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] boot 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: osdmap e42: 7 total, 7 up, 7 in 2024-09-12T03:14:08.020 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:14:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:14:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] boot 2024-09-12T03:14:08.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: osdmap e42: 7 total, 7 up, 7 in 2024-09-12T03:14:08.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:08.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:14:08.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:14:08.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:08.316 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: osd.6 [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] boot 2024-09-12T03:14:08.317 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: osdmap e42: 7 total, 7 up, 7 in 2024-09-12T03:14:08.317 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T03:14:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:08 smithi121 ceph-mon[34963]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:08 smithi121 ceph-mon[34963]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:14:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:08 smithi135 ceph-mon[35474]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:08 smithi135 ceph-mon[35474]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:14:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:08 smithi019 ceph-mon[30953]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-12T03:14:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:08 smithi019 ceph-mon[30953]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/4210063724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]': finished 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:09 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/3837350087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/4210063724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]': finished 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:10.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:09 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/3837350087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:10.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/4210063724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]: dispatch 2024-09-12T03:14:10.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17cbf654-fee4-4cfd-a5f3-32a8e312fabc"}]': finished 2024-09-12T03:14:10.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: osdmap e44: 8 total, 7 up, 8 in 2024-09-12T03:14:10.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:10.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:09 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/3837350087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:11 smithi121 ceph-mon[34963]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:11 smithi019 ceph-mon[30953]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:11 smithi135 ceph-mon[35474]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:13 smithi121 ceph-mon[34963]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:13 smithi019 ceph-mon[30953]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:13 smithi135 ceph-mon[35474]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:15.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:15 smithi121 ceph-mon[34963]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:15.346 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:15 smithi135 ceph-mon[35474]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:15 smithi019 ceph-mon[30953]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:17 smithi121 ceph-mon[34963]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:17 smithi019 ceph-mon[30953]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:17 smithi135 ceph-mon[35474]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:18.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:14:18.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:18.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:14:18.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T03:14:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:19.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:19 smithi121 ceph-mon[34963]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:19.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:19 smithi121 ceph-mon[34963]: Deploying daemon osd.7 on smithi121 2024-09-12T03:14:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:19 smithi019 ceph-mon[30953]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:19 smithi019 ceph-mon[30953]: Deploying daemon osd.7 on smithi121 2024-09-12T03:14:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:19 smithi135 ceph-mon[35474]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:19 smithi135 ceph-mon[35474]: Deploying daemon osd.7 on smithi121 2024-09-12T03:14:21.334 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:21 smithi121 ceph-mon[34963]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:21.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:21 smithi019 ceph-mon[30953]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:21 smithi135 ceph-mon[35474]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:23.377 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:23 smithi121 ceph-mon[34963]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:23 smithi019 ceph-mon[30953]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:23 smithi135 ceph-mon[35474]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:24.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:25 smithi121 ceph-mon[34963]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:25 smithi019 ceph-mon[30953]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:25 smithi135 ceph-mon[35474]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:27.839 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 7 on host 'smithi121' 2024-09-12T03:14:27.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.838+0000 7fc56ffff640 1 -- 172.21.15.121:0/3832791834 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc588062630 con 0x7fc5700a2bb0 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 msgr2=0x7fc5700a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 0x7fc5700a5070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0058c0 tx=0x7fc57c005850 comp rx=0 tx=0).stop 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 msgr2=0x7fc588197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc578031610 tx=0x7fc5780043b0 comp rx=0 tx=0).stop 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 shutdown_connections 2024-09-12T03:14:27.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc5881037c0 0x7fc58819f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc5700a2bb0 0x7fc5700a5070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc588102e00 0x7fc5881983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 --2- 172.21.15.121:0/3832791834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc588101c00 0x7fc588197e80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 >> 172.21.15.121:0/3832791834 conn(0x7fc5880fd390 msgr2=0x7fc588106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 shutdown_connections 2024-09-12T03:14:27.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:14:27.841+0000 7fc58d661640 1 -- 172.21.15.121:0/3832791834 wait complete. 2024-09-12T03:14:28.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.531 DEBUG:teuthology.orchestra.run.smithi121:osd.7> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T03:14:28.534 INFO:tasks.cephadm:Deploying osd.8 on smithi135 with /dev/vg_nvme/lv_2... 2024-09-12T03:14:28.534 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T03:14:28.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:28.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:28.787 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:14:29.161 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:29 smithi135 ceph-mon[35474]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:29 smithi121 ceph-mon[34963]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:29 smithi019 ceph-mon[30953]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:30.093 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:14:29 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:14:29.807+0000 7f6954685640 -1 osd.7 0 log_to_monitors true 2024-09-12T03:14:30.206 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:30 smithi135 ceph-mon[35474]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:30.206 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:30 smithi135 ceph-mon[35474]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:30.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:30 smithi121 ceph-mon[34963]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:30 smithi121 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:30 smithi019 ceph-mon[30953]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:30 smithi019 ceph-mon[30953]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T03:14:31.390 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:31.390 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:14:31.391 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:14:31.391 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:31.391 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.391 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:31 smithi135 ceph-mon[35474]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.393 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:14:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:14:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:14:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T03:14:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T03:14:31.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:31.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: from='osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:31 smithi019 ceph-mon[30953]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:14:31.418 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_2 2024-09-12T03:14:31.715 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:14:31.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:14:31.472+0000 7f6950e1c640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T03:14:31.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:14:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:14:31.484+0000 7f6948428640 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:14:32.247 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='osd.7 ' entity='osd.7' 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.248 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] boot 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T03:14:32.249 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- 172.21.15.135:0/2233426898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934101cd0 msgr2=0x7f4934102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 --2- 172.21.15.135:0/2233426898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934101cd0 0x7f4934102150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4924009900 tx=0x7f492402a410 comp rx=0 tx=0).stop 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- 172.21.15.135:0/2233426898 shutdown_connections 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 --2- 172.21.15.135:0/2233426898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934102690 0x7f4934108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 --2- 172.21.15.135:0/2233426898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934101cd0 0x7f4934102150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.281 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 --2- 172.21.15.135:0/2233426898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934100ad0 0x7f4934100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- 172.21.15.135:0/2233426898 >> 172.21.15.135:0/2233426898 conn(0x7f49340fc280 msgr2=0x7f49340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- 172.21.15.135:0/2233426898 shutdown_connections 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- 172.21.15.135:0/2233426898 wait complete. 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 Processor -- start 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.246+0000 7f493c286640 1 -- start start 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934100ad0 0x7f493406ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 0x7f493419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f493410c730 con 0x7f4934100ad0 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f493410c5b0 con 0x7f4934101cd0 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493c286640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f493410c8b0 con 0x7f4934102690 2024-09-12T03:14:32.282 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f4939821640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 0x7f493419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:43028/0 (socket says 172.21.15.135:43028) 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 -- 172.21.15.135:0/1182587180 learned_addr learned my addr 172.21.15.135:0/1182587180 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a022640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934100ad0 0x7f493406ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 -- 172.21.15.135:0/1182587180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 msgr2=0x7f493419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 0x7f493419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 -- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934100ad0 msgr2=0x7f493406ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934100ad0 0x7f493406ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 -- 172.21.15.135:0/1182587180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493419fa50 con 0x7f4934102690 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f4939821640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 0x7f493419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f493a823640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f492c00eaa0 tx=0x7f492c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.283 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.247+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c00cdb0 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.248+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f492c005020 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.248+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492c0116b0 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.248+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493419fd40 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.248+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49341a01a0 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.249+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4934107e80 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.250+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f492c005560 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.250+0000 7f492b7fe640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 0x7f49200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.250+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5430+0+0 (secure 0 0 0) 0x7f492c0ce6a0 con 0x7f4934102690 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.251+0000 7f493a022640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 0x7f49200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.251+0000 7f493a022640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 0x7f49200a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f493406b050 tx=0x7f4930005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:32.284 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.255+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f492c016030 con 0x7f4934102690 2024-09-12T03:14:32.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='osd.7 ' entity='osd.7' 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] boot 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T03:14:32.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='osd.7 ' entity='osd.7' 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:32.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: osd.7 [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] boot 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: osdmap e47: 8 total, 8 up, 8 in 2024-09-12T03:14:32.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T03:14:32.439 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:32.438+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4934062630 con 0x7f49200a2b80 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi121 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:33 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi121 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:33 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi121 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi121 to 5695M 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:33.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:33 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:34.243 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:34 smithi135 ceph-mon[35474]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:34.243 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:34 smithi135 ceph-mon[35474]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:14:34.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:34 smithi121 ceph-mon[34963]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:34.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:34 smithi121 ceph-mon[34963]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:14:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:34 smithi019 ceph-mon[30953]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:34 smithi019 ceph-mon[30953]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/390286454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]': finished 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:14:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/390286454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]': finished 2024-09-12T03:14:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:14:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:35.534 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: pgmap v169: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:35.534 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: osdmap e49: 8 total, 8 up, 8 in 2024-09-12T03:14:35.534 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: osdmap e50: 8 total, 8 up, 8 in 2024-09-12T03:14:35.534 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/390286454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.535 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]: dispatch 2024-09-12T03:14:35.535 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d7263722-4663-4725-8f31-833f02d73b8e"}]': finished 2024-09-12T03:14:35.535 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T03:14:35.535 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:36 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/4208179231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:36 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/4208179231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:36.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:36 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/4208179231' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:14:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:37 smithi121 ceph-mon[34963]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:37 smithi019 ceph-mon[30953]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:37 smithi135 ceph-mon[35474]: pgmap v173: 1 pgs: 1 remapped; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:39 smithi121 ceph-mon[34963]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:39 smithi019 ceph-mon[30953]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:39.533 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:39 smithi135 ceph-mon[35474]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:41.246 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:41 smithi135 ceph-mon[35474]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:41 smithi121 ceph-mon[34963]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:41 smithi019 ceph-mon[30953]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:43.253 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:43 smithi135 ceph-mon[35474]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:43 smithi121 ceph-mon[34963]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:43 smithi019 ceph-mon[30953]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:44.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:14:44.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:44.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:14:44.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:44.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T03:14:44.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:45.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:45 smithi135 ceph-mon[35474]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:45.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:45 smithi135 ceph-mon[35474]: Deploying daemon osd.8 on smithi135 2024-09-12T03:14:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:45 smithi019 ceph-mon[30953]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:45 smithi019 ceph-mon[30953]: Deploying daemon osd.8 on smithi135 2024-09-12T03:14:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:45 smithi121 ceph-mon[34963]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:45 smithi121 ceph-mon[34963]: Deploying daemon osd.8 on smithi135 2024-09-12T03:14:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:47 smithi019 ceph-mon[30953]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:47 smithi135 ceph-mon[35474]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:47 smithi121 ceph-mon[34963]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:49 smithi019 ceph-mon[30953]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:49 smithi135 ceph-mon[35474]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:49 smithi121 ceph-mon[34963]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:50.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:50.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:50.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:50.551 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:50.551 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:50.551 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:50.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:50.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:50.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:51 smithi135 ceph-mon[35474]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:51.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:51 smithi121 ceph-mon[34963]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:51 smithi019 ceph-mon[30953]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:53.171 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.170+0000 7f492b7fe640 1 -- 172.21.15.135:0/1182587180 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4934062630 con 0x7f49200a2b80 2024-09-12T03:14:53.171 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 8 on host 'smithi135' 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 msgr2=0x7f49200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 0x7f49200a5040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f493406b050 tx=0x7f4930005f90 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 msgr2=0x7f493419f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f492c00eaa0 tx=0x7f492c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 shutdown_connections 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4934102690 0x7f493419f190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4934101cd0 0x7f493419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49200a2b80 0x7f49200a5040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 --2- 172.21.15.135:0/1182587180 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4934100ad0 0x7f493406ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:53.173 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.172+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 >> 172.21.15.135:0/1182587180 conn(0x7f49340fc280 msgr2=0x7f4934105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:53.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.173+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 shutdown_connections 2024-09-12T03:14:53.174 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:14:53.173+0000 7f493c286640 1 -- 172.21.15.135:0/1182587180 wait complete. 2024-09-12T03:14:53.475 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:53.476 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:14:53.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:53.852 DEBUG:teuthology.orchestra.run.smithi135:osd.8> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T03:14:53.855 INFO:tasks.cephadm:Deploying osd.9 on smithi019 with /dev/vg_nvme/lv_1... 2024-09-12T03:14:53.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:14:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:54.461 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:54 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:54 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:54.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:54 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:55.288 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:55 smithi135 ceph-mon[35474]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:55.450 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:55 smithi019 ceph-mon[30953]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:55.544 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:14:55 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:14:55.520+0000 7efd82dbf640 -1 osd.8 0 log_to_monitors true 2024-09-12T03:14:55.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:55 smithi121 ceph-mon[34963]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:56.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:56 smithi019 ceph-mon[30953]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:56 smithi019 ceph-mon[30953]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:56 smithi135 ceph-mon[35474]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:56 smithi135 ceph-mon[35474]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:56 smithi121 ceph-mon[34963]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:56 smithi121 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T03:14:56.887 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:14:56.913 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi019:vg_nvme/lv_1 2024-09-12T03:14:57.173 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:14:57.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:57.441 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:57.442 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:14:57.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:57.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:14:57.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:14:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 -- 172.21.15.19:0/648919048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c1037d0 msgr2=0x7f8f7c10a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 --2- 172.21.15.19:0/648919048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c10a060 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009900 tx=0x7f8f7402a410 comp rx=0 tx=0).stop 2024-09-12T03:14:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 -- 172.21.15.19:0/648919048 shutdown_connections 2024-09-12T03:14:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 --2- 172.21.15.19:0/648919048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c10a060 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 --2- 172.21.15.19:0/648919048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c102e10 0x7f8f7c103290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 --2- 172.21.15.19:0/648919048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c101c10 0x7f8f7c102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 -- 172.21.15.19:0/648919048 >> 172.21.15.19:0/648919048 conn(0x7f8f7c0fd3c0 msgr2=0x7f8f7c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 -- 172.21.15.19:0/648919048 shutdown_connections 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 -- 172.21.15.19:0/648919048 wait complete. 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.635+0000 7f8f8147b640 1 Processor -- start 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 -- start start 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 0x7f8f7c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c19f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10d870 con 0x7f8f7c102e10 2024-09-12T03:14:57.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10d6f0 con 0x7f8f7c1037d0 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f8147b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7c10d9f0 con 0x7f8f7c101c10 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60598/0 (socket says 172.21.15.19:60598) 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 -- 172.21.15.19:0/3359754539 learned_addr learned my addr 172.21.15.19:0/3359754539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7affd640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 0x7f8f7c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7b7fe640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c19f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 -- 172.21.15.19:0/3359754539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 msgr2=0x7f8f7c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 0x7f8f7c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 -- 172.21.15.19:0/3359754539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c1037d0 msgr2=0x7f8f7c19f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:14:57.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c19f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7a7fc640 1 -- 172.21.15.19:0/3359754539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7c19fa60 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.636+0000 7f8f7affd640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 0x7f8f7c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f7a7fc640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d950 tx=0x7f8f7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70017070 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f70009c30 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7c19fd50 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000f6b0 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.637+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7c1a0230 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.638+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f7c108f40 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.639+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8f700109d0 con 0x7f8f7c102e10 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.640+0000 7f8f63fff640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 0x7f8f640a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:14:57.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.640+0000 7f8f7affd640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 0x7f8f640a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:14:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.640+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5549+0+0 (secure 0 0 0) 0x7f8f700ce600 con 0x7f8f7c102e10 2024-09-12T03:14:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.640+0000 7f8f7affd640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 0x7f8f640a5070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c102c70 tx=0x7f8f6c007870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:14:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.644+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8f7009bf90 con 0x7f8f7c102e10 2024-09-12T03:14:57.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:14:57.828+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8f7c062630 con 0x7f8f640a2bb0 2024-09-12T03:14:58.044 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:14:57.567+0000 7efd7ed42640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T03:14:58.044 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:14:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:14:57.579+0000 7efd76b62640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:14:58.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi135 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='osd.8 ' entity='osd.8' 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:58.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:58 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi135 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='osd.8 ' entity='osd.8' 2024-09-12T03:14:58.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:58.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:58.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:58 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:58.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi135 2024-09-12T03:14:58.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi135 to 1938M 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='osd.8 ' entity='osd.8' 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:14:58.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:58 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:14:59.337 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] boot 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T03:14:59.338 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:14:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] boot 2024-09-12T03:14:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T03:14:59.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:14:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:59.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi019:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: osd.8 [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] boot 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: osdmap e54: 9 total, 9 up, 9 in 2024-09-12T03:14:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:14:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T03:15:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:00 smithi019 ceph-mon[30953]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:15:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:00 smithi019 ceph-mon[30953]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:00 smithi135 ceph-mon[35474]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:15:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:00 smithi135 ceph-mon[35474]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:00.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:00 smithi121 ceph-mon[34963]: osdmap e55: 9 total, 9 up, 9 in 2024-09-12T03:15:00.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:00 smithi121 ceph-mon[34963]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 258 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:15:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]: dispatch 2024-09-12T03:15:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]': finished 2024-09-12T03:15:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:15:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3768281619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]: dispatch 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]': finished 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3768281619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]: dispatch 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2551092510' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "93a2fba3-453f-42ab-ac18-23faec974ede"}]': finished 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: osdmap e57: 10 total, 9 up, 10 in 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3768281619' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:02.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:02 smithi019 ceph-mon[30953]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:02.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:02 smithi019 ceph-mon[30953]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:15:02.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:02 smithi135 ceph-mon[35474]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:02 smithi135 ceph-mon[35474]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:15:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:02.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:02 smithi121 ceph-mon[34963]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:02.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:02 smithi121 ceph-mon[34963]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T03:15:02.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:03 smithi135 ceph-mon[35474]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:15:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:03.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:03 smithi121 ceph-mon[34963]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:15:03.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:03.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:03 smithi019 ceph-mon[30953]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T03:15:03.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:04 smithi135 ceph-mon[35474]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:15:04.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:04 smithi121 ceph-mon[34963]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:15:04.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:04 smithi019 ceph-mon[30953]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T03:15:06.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:06 smithi019 ceph-mon[30953]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:15:06.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:06 smithi135 ceph-mon[35474]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:15:06.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:06 smithi121 ceph-mon[34963]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-12T03:15:08.543 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:08 smithi019 ceph-mon[30953]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T03:15:08.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:08 smithi135 ceph-mon[35474]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T03:15:08.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:08 smithi121 ceph-mon[34963]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-09-12T03:15:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:15:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:09 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:15:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:09 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:09.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T03:15:09.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:09 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:10 smithi135 ceph-mon[35474]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:10 smithi135 ceph-mon[35474]: Deploying daemon osd.9 on smithi019 2024-09-12T03:15:10.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:10 smithi019 ceph-mon[30953]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.839 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:10 smithi019 ceph-mon[30953]: Deploying daemon osd.9 on smithi019 2024-09-12T03:15:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:10 smithi121 ceph-mon[34963]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T03:15:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:10 smithi121 ceph-mon[34963]: Deploying daemon osd.9 on smithi019 2024-09-12T03:15:12.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:12 smithi019 ceph-mon[30953]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T03:15:12.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:12 smithi135 ceph-mon[35474]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T03:15:12.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:12 smithi121 ceph-mon[34963]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-12T03:15:14.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:14 smithi019 ceph-mon[30953]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:14.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:14 smithi135 ceph-mon[35474]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:14 smithi121 ceph-mon[34963]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:15.709 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:15.709 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:15.709 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:15.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:15.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:15.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:16.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:16 smithi135 ceph-mon[35474]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:16.816 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:16 smithi019 ceph-mon[30953]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:16.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:16 smithi121 ceph-mon[34963]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:18.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:18.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:18 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:18 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:18 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.090+0000 7f8f63fff640 1 -- 172.21.15.19:0/3359754539 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f7c062630 con 0x7f8f640a2bb0 2024-09-12T03:15:19.090 INFO:teuthology.orchestra.run.smithi019.stdout:Created osd(s) 9 on host 'smithi019' 2024-09-12T03:15:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 msgr2=0x7f8f640a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 0x7f8f640a5070 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c102c70 tx=0x7f8f6c007870 comp rx=0 tx=0).stop 2024-09-12T03:15:19.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 msgr2=0x7f8f7c198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d950 tx=0x7f8f7000de10 comp rx=0 tx=0).stop 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 shutdown_connections 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8f7c1037d0 0x7f8f7c19f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8f640a2bb0 0x7f8f640a5070 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8f7c102e10 0x7f8f7c198370 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 --2- 172.21.15.19:0/3359754539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8f7c101c10 0x7f8f7c197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 >> 172.21.15.19:0/3359754539 conn(0x7f8f7c0fd3c0 msgr2=0x7f8f7c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 shutdown_connections 2024-09-12T03:15:19.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:15:19.092+0000 7f8f8147b640 1 -- 172.21.15.19:0/3359754539 wait complete. 2024-09-12T03:15:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.846 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:19.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.847 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.874 DEBUG:teuthology.orchestra.run.smithi019:osd.9> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T03:15:19.877 INFO:tasks.cephadm:Deploying osd.10 on smithi121 with /dev/vg_nvme/lv_1... 2024-09-12T03:15:19.877 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:15:19.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:19.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:19.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:20.131 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:15:21.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:21 smithi019 ceph-mon[30953]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:21 smithi121 ceph-mon[34963]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:21.421 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:15:21 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:15:21.164+0000 7ff4956e5640 -1 osd.9 0 log_to_monitors true 2024-09-12T03:15:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:21 smithi135 ceph-mon[35474]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:22.246 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:22 smithi019 ceph-mon[30953]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:15:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:22 smithi121 ceph-mon[34963]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:15:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:22 smithi135 ceph-mon[35474]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T03:15:22.829 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-09-12T03:15:22.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi121:vg_nvme/lv_1 2024-09-12T03:15:23.158 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:15:23.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:23.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:23.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.414 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:15:23.103+0000 7ff491668640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T03:15:23.414 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:15:23.117+0000 7ff48d490640 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:15:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:15:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:15:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:15:23.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:23.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:23.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]: dispatch 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:23.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 -- 172.21.15.121:0/1495670249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 msgr2=0x7fefc81024a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 --2- 172.21.15.121:0/1495670249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc81024a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009900 tx=0x7fefbc02a410 comp rx=0 tx=0).stop 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 -- 172.21.15.121:0/1495670249 shutdown_connections 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 --2- 172.21.15.121:0/1495670249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 0x7fefc8109040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 --2- 172.21.15.121:0/1495670249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 --2- 172.21.15.121:0/1495670249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc81024a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 -- 172.21.15.121:0/1495670249 >> 172.21.15.121:0/1495670249 conn(0x7fefc80fba40 msgr2=0x7fefc80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:23.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 -- 172.21.15.121:0/1495670249 shutdown_connections 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.803+0000 7fefcec71640 1 -- 172.21.15.121:0/1495670249 wait complete. 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 Processor -- start 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 -- start start 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 0x7fefc8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810c820 con 0x7fefc8100de0 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810c6a0 con 0x7fefc81020a0 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcec71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fefc810c9a0 con 0x7fefc81017a0 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcd20e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcd20e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:39640/0 (socket says 172.21.15.121:39640) 2024-09-12T03:15:23.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcd20e640 1 -- 172.21.15.121:0/1845845189 learned_addr learned my addr 172.21.15.121:0/1845845189 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcd20e640 1 -- 172.21.15.121:0/1845845189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 msgr2=0x7fefc8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefc7fff640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 0x7fefc8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcd20e640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 0x7fefc8198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.804+0000 7fefcca0d640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcd20e640 1 -- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 msgr2=0x7fefc8198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcd20e640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcd20e640 1 -- 172.21.15.121:0/1845845189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefc819fc60 con 0x7fefc81020a0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcca0d640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcd20e640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefc000eaa0 tx=0x7fefc000ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc000cdb0 con 0x7fefc81020a0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc0005020 con 0x7fefc81020a0 2024-09-12T03:15:23.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefc819ff50 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc00116b0 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.805+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefc81a0480 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.806+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefc81a0740 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.807+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fefc0005560 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.808+0000 7fefc5ffb640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 0x7fefb80a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.808+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5981+0+0 (secure 0 0 0) 0x7fefc00cea00 con 0x7fefc81020a0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.808+0000 7fefcca0d640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 0x7fefb80a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.808+0000 7fefcca0d640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 0x7fefb80a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000c00 tx=0x7fefbc009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:23.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.812+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefc0016030 con 0x7fefc81020a0 2024-09-12T03:15:23.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:23.985+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fefc81a0910 con 0x7fefb80a2bb0 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi019 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.397 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:24.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:24.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:24.398 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi019 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:24.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi019 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi019 to 4273M 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi019", "root=default"]}]': finished 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:24.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:24.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:24.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:25.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] boot 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:15:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:25 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:15:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:15:25.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:25.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:25.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] boot 2024-09-12T03:15:25.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:15:25.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:25 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:15:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:15:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-12T03:15:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi121:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:25.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: osd.9 [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] boot 2024-09-12T03:15:25.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: osdmap e62: 10 total, 10 up, 10 in 2024-09-12T03:15:25.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:25 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T03:15:26.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:26 smithi121 ceph-mon[34963]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:15:26.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:26 smithi019 ceph-mon[30953]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:15:26.426 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:26 smithi135 ceph-mon[35474]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/3939764085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]': finished 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.121:0/1155486479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/3939764085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]': finished 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.121:0/1155486479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/3939764085' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]: dispatch 2024-09-12T03:15:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17c4e2a4-2369-4f99-b966-e9deb821b99d"}]': finished 2024-09-12T03:15:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T03:15:27.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:27.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.121:0/1155486479' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:29 smithi121 ceph-mon[34963]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:29 smithi019 ceph-mon[30953]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:29 smithi135 ceph-mon[35474]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 286 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:31 smithi121 ceph-mon[34963]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:31 smithi019 ceph-mon[30953]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:31 smithi135 ceph-mon[35474]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:33 smithi121 ceph-mon[34963]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:33 smithi019 ceph-mon[30953]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:33 smithi135 ceph-mon[35474]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:35 smithi121 ceph-mon[34963]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:15:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:35 smithi019 ceph-mon[30953]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:15:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:35 smithi135 ceph-mon[35474]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T03:15:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:36 smithi019 ceph-mon[30953]: Deploying daemon osd.10 on smithi121 2024-09-12T03:15:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:36 smithi135 ceph-mon[35474]: Deploying daemon osd.10 on smithi121 2024-09-12T03:15:36.597 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:36 smithi121 ceph-mon[34963]: Deploying daemon osd.10 on smithi121 2024-09-12T03:15:37.208 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:37 smithi121 ceph-mon[34963]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:37 smithi019 ceph-mon[30953]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:37 smithi135 ceph-mon[35474]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:39 smithi121 ceph-mon[34963]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:39 smithi019 ceph-mon[30953]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:39 smithi135 ceph-mon[35474]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:41 smithi121 ceph-mon[34963]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:41 smithi019 ceph-mon[30953]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:41.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:41 smithi135 ceph-mon[35474]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:42 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:42 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:42.615 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:42.615 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:42.615 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:42 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:43 smithi121 ceph-mon[34963]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:43 smithi135 ceph-mon[35474]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:43 smithi019 ceph-mon[30953]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:44.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.706+0000 7fefc5ffb640 1 -- 172.21.15.121:0/1845845189 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fefc81a0910 con 0x7fefb80a2bb0 2024-09-12T03:15:44.707 INFO:teuthology.orchestra.run.smithi121.stdout:Created osd(s) 10 on host 'smithi121' 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 msgr2=0x7fefb80a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 0x7fefb80a5070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000c00 tx=0x7fefbc009560 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 msgr2=0x7fefc819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fefc000eaa0 tx=0x7fefc000ef60 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 shutdown_connections 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fefc81020a0 0x7fefc819f5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fefc81017a0 0x7fefc8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fefb80a2bb0 0x7fefb80a5070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 --2- 172.21.15.121:0/1845845189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fefc8100de0 0x7fefc8198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:44.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.708+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 >> 172.21.15.121:0/1845845189 conn(0x7fefc80fba40 msgr2=0x7fefc80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:44.710 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.709+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 shutdown_connections 2024-09-12T03:15:44.710 INFO:teuthology.orchestra.run.smithi121.stderr:2024-09-12T03:15:44.709+0000 7fefcec71640 1 -- 172.21.15.121:0/1845845189 wait complete. 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:45.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.386 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:45.386 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.386 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.387 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.413 DEBUG:teuthology.orchestra.run.smithi121:osd.10> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:15:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:45.415 INFO:tasks.cephadm:Deploying osd.11 on smithi135 with /dev/vg_nvme/lv_1... 2024-09-12T03:15:45.415 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T03:15:45.671 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:15:47.127 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:46 smithi135 ceph-mon[35474]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:46 smithi121 ceph-mon[34963]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:47.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:15:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:15:47.162+0000 7fb4d303b640 -1 osd.10 0 log_to_monitors true 2024-09-12T03:15:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:46 smithi019 ceph-mon[30953]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:48.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:47 smithi135 ceph-mon[35474]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:47 smithi135 ceph-mon[35474]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:47 smithi121 ceph-mon[34963]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:47 smithi121 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:47 smithi019 ceph-mon[30953]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:47 smithi019 ceph-mon[30953]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T03:15:48.463 INFO:teuthology.orchestra.run.smithi135.stdout: 2024-09-12T03:15:48.494 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi135:vg_nvme/lv_1 2024-09-12T03:15:48.749 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.259 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:49.260 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 -- 172.21.15.135:0/2940907722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 msgr2=0x7f8cb4109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:49.291 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 --2- 172.21.15.135:0/2940907722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb4109d50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009ca0 tx=0x7f8cac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 -- 172.21.15.135:0/2940907722 shutdown_connections 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 --2- 172.21.15.135:0/2940907722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb4109d50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 --2- 172.21.15.135:0/2940907722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 0x7f8cb41073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 --2- 172.21.15.135:0/2940907722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 0x7f8cb4104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 -- 172.21.15.135:0/2940907722 >> 172.21.15.135:0/2940907722 conn(0x7f8cb40fc2a0 msgr2=0x7f8cb40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 -- 172.21.15.135:0/2940907722 shutdown_connections 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.258+0000 7f8cba28a640 1 -- 172.21.15.135:0/2940907722 wait complete. 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 Processor -- start 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 -- start start 2024-09-12T03:15:49.292 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 0x7f8cb4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 0x7f8cb4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410d940 con 0x7f8cb41026a0 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410d7c0 con 0x7f8cb4104fd0 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cba28a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb410dac0 con 0x7f8cb4107900 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:37216/0 (socket says 172.21.15.135:37216) 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 -- 172.21.15.135:0/1108128759 learned_addr learned my addr 172.21.15.135:0/1108128759 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb3fff640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 0x7f8cb4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cabfff640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 0x7f8cb4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 -- 172.21.15.135:0/1108128759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 msgr2=0x7f8cb4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:49.293 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 0x7f8cb4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 -- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 msgr2=0x7f8cb4195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 0x7f8cb4195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.259+0000 7f8cb8827640 1 -- 172.21.15.135:0/1108128759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb41a61e0 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cb8827640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cac031ae0 tx=0x7f8cac031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac003b70 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb41a6470 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cac003d10 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac02fc20 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.260+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb41a6a20 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.261+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb41093f0 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.262+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8cac0046c0 con 0x7f8cb4107900 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.263+0000 7f8cb1ffb640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 0x7f8ca00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:15:49.294 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.263+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6414+0+0 (secure 0 0 0) 0x7f8cac0efcd0 con 0x7f8cb4107900 2024-09-12T03:15:49.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.263+0000 7f8cb3fff640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 0x7f8ca00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:15:49.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.264+0000 7f8cb3fff640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 0x7f8ca00a50c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009990 tx=0x7f8ca4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:15:49.295 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.266+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cac0bd430 con 0x7f8cb4107900 2024-09-12T03:15:49.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:15:49.005+0000 7fb4cefbe640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T03:15:49.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:15:49.017+0000 7fb4cade6640 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:49.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T03:15:49.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T03:15:49.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:49.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:15:49.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:15:49.445 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:15:49.445+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8cb4062630 con 0x7f8ca00a2c00 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi121 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] boot 2024-09-12T03:15:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:15:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi121 2024-09-12T03:15:50.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-12T03:15:50.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:15:50.303 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] boot 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:15:50.304 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi121 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi121 to 4271M 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi121", "root=default"]}]': finished 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: osd.10 [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] boot 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: osdmap e67: 11 total, 11 up, 11 in 2024-09-12T03:15:50.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T03:15:51.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:51 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:15:51.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:51 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:15:51.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:51 smithi135 ceph-mon[35474]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:51.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:51 smithi135 ceph-mon[35474]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:51.207 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:51 smithi135 ceph-mon[35474]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:15:51.315 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:51 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:15:51.315 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:51 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:15:51.315 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:51 smithi121 ceph-mon[34963]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:51.316 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:51 smithi121 ceph-mon[34963]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:51.316 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:51 smithi121 ceph-mon[34963]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:15:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:51 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:15:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:51 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:15:51.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:51 smithi019 ceph-mon[30953]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 287 MiB used, 894 GiB / 894 GiB avail 2024-09-12T03:15:51.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:51 smithi019 ceph-mon[30953]: from='client.24361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi135:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:15:51.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:51 smithi019 ceph-mon[30953]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/1586211586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]': finished 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:52 smithi135 ceph-mon[35474]: from='client.? 172.21.15.135:0/282425936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/1586211586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]': finished 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:52 smithi121 ceph-mon[34963]: from='client.? 172.21.15.135:0/282425936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:15:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T03:15:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/1586211586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]: dispatch 2024-09-12T03:15:53.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a58b445d-ac43-4862-9926-1bcd0a63537d"}]': finished 2024-09-12T03:15:53.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T03:15:53.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:53.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:52 smithi019 ceph-mon[30953]: from='client.? 172.21.15.135:0/282425936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T03:15:54.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:54 smithi121 ceph-mon[34963]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:15:54.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:54 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:54 smithi019 ceph-mon[30953]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:15:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:54 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:54 smithi135 ceph-mon[35474]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T03:15:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:54 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:15:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:55 smithi121 ceph-mon[34963]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:15:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:55 smithi019 ceph-mon[30953]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:15:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:55 smithi135 ceph-mon[35474]: pgmap v232: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 167 KiB/s, 0 objects/s recovering 2024-09-12T03:15:57.227 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:57 smithi135 ceph-mon[35474]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:15:57.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:57 smithi121 ceph-mon[34963]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:15:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:57 smithi019 ceph-mon[30953]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 129 KiB/s, 0 objects/s recovering 2024-09-12T03:15:59.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:15:59 smithi121 ceph-mon[34963]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:15:59 smithi019 ceph-mon[30953]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:15:59.532 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:15:59 smithi135 ceph-mon[35474]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T03:16:01.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:01 smithi135 ceph-mon[35474]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:16:01.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:16:01.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:01.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:01 smithi121 ceph-mon[34963]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:16:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:16:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:01 smithi019 ceph-mon[30953]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T03:16:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T03:16:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:02.378 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:02 smithi135 ceph-mon[35474]: Deploying daemon osd.11 on smithi135 2024-09-12T03:16:02.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:02 smithi121 ceph-mon[34963]: Deploying daemon osd.11 on smithi135 2024-09-12T03:16:02.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:02 smithi019 ceph-mon[30953]: Deploying daemon osd.11 on smithi135 2024-09-12T03:16:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:03 smithi135 ceph-mon[35474]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:16:03.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:03 smithi121 ceph-mon[34963]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:16:03.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:03 smithi019 ceph-mon[30953]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-12T03:16:05.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:05 smithi121 ceph-mon[34963]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:05 smithi019 ceph-mon[30953]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:05 smithi135 ceph-mon[35474]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:07.395 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:07 smithi135 ceph-mon[35474]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:07.395 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:07.395 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:07.395 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:07 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:07 smithi121 ceph-mon[34963]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:07.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:07 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:07 smithi019 ceph-mon[30953]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:07 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:09.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:09 smithi121 ceph-mon[34963]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:09 smithi019 ceph-mon[30953]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:09.494 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:09 smithi135 ceph-mon[35474]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:09.842 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.841+0000 7f8cb1ffb640 1 -- 172.21.15.135:0/1108128759 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8cb4062630 con 0x7f8ca00a2c00 2024-09-12T03:16:09.842 INFO:teuthology.orchestra.run.smithi135.stdout:Created osd(s) 11 on host 'smithi135' 2024-09-12T03:16:09.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 msgr2=0x7f8ca00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 0x7f8ca00a50c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4009990 tx=0x7f8ca4009110 comp rx=0 tx=0).stop 2024-09-12T03:16:09.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 msgr2=0x7f8cb41a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:09.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8cac031ae0 tx=0x7f8cac031ee0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.844 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 shutdown_connections 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8cb4107900 0x7f8cb41a5c70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8cb4104fd0 0x7f8cb4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8ca00a2c00 0x7f8ca00a50c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 --2- 172.21.15.135:0/1108128759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8cb41026a0 0x7f8cb4195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 >> 172.21.15.135:0/1108128759 conn(0x7f8cb40fc2a0 msgr2=0x7f8cb4101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 shutdown_connections 2024-09-12T03:16:09.845 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:09.843+0000 7f8cba28a640 1 -- 172.21.15.135:0/1108128759 wait complete. 2024-09-12T03:16:10.914 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:10 smithi135 ceph-mon[35474]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:10.914 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:10.914 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:10 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:10.946 DEBUG:teuthology.orchestra.run.smithi135:osd.11> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T03:16:10.949 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T03:16:10.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:16:11.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:10 smithi121 ceph-mon[34963]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:11.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:11.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:10 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:11.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:10 smithi019 ceph-mon[30953]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:11.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:11.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:10 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:11.212 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 -- 172.21.15.19:0/3209646717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 msgr2=0x7f7d10108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3209646717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d10108f20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009ca0 tx=0x7f7d0802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 -- 172.21.15.19:0/3209646717 shutdown_connections 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3209646717 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d10108f20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3209646717 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 0x7f7d10102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3209646717 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 0x7f7d10100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 -- 172.21.15.19:0/3209646717 >> 172.21.15.19:0/3209646717 conn(0x7f7d100fc280 msgr2=0x7f7d100fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:11.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 -- 172.21.15.19:0/3209646717 shutdown_connections 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.690+0000 7f7d1503e640 1 -- 172.21.15.19:0/3209646717 wait complete. 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 Processor -- start 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 -- start start 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 0x7f7d10197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 0x7f7d101983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010d840 con 0x7f7d10102690 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010d6c0 con 0x7f7d10101cd0 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d1503e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1010d9c0 con 0x7f7d10100ad0 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d0f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d0f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60006/0 (socket says 172.21.15.19:60006) 2024-09-12T03:16:11.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d0f59e640 1 -- 172.21.15.19:0/3669178421 learned_addr learned my addr 172.21.15.19:0/3669178421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d0ed9d640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 0x7f7d10197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.691+0000 7f7d07fff640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 0x7f7d101983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 -- 172.21.15.19:0/3669178421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 msgr2=0x7f7d10197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 0x7f7d10197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 -- 172.21.15.19:0/3669178421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 msgr2=0x7f7d101983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 0x7f7d101983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 -- 172.21.15.19:0/3669178421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1019fac0 con 0x7f7d10102690 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0f59e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d08031ae0 tx=0x7f7d08031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d08003c30 con 0x7f7d10102690 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1019fd50 con 0x7f7d10102690 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d08003dd0 con 0x7f7d10102690 2024-09-12T03:16:11.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0802fcb0 con 0x7f7d10102690 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.692+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d101a0200 con 0x7f7d10102690 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.693+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d10107ea0 con 0x7f7d10102690 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.694+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7d08004050 con 0x7f7d10102690 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.695+0000 7f7d0cd99640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 0x7f7cfc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.695+0000 7f7d0ed9d640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 0x7f7cfc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.695+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6809+0+0 (secure 0 0 0) 0x7f7d080efc20 con 0x7f7d10102690 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.696+0000 7f7d0ed9d640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 0x7f7cfc0a5070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d00005fd0 tx=0x7f7d0000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:11.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.700+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d080bd280 con 0x7f7d10102690 2024-09-12T03:16:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.877+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7d10062e90 con 0x7f7d10102690 2024-09-12T03:16:11.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.878+0000 7f7d0cd99640 1 -- 172.21.15.19:0/3669178421 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f7d080bcb80 con 0x7f7d10102690 2024-09-12T03:16:11.878 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 msgr2=0x7f7cfc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 0x7f7cfc0a5070 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7d00005fd0 tx=0x7f7d0000b410 comp rx=0 tx=0).stop 2024-09-12T03:16:11.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 msgr2=0x7f7d1019f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d08031ae0 tx=0x7f7d08031ee0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 shutdown_connections 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cfc0a2bb0 0x7f7cfc0a5070 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d10102690 0x7f7d1019f450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d10101cd0 0x7f7d101983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 --2- 172.21.15.19:0/3669178421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d10100ad0 0x7f7d10197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:11.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 >> 172.21.15.19:0/3669178421 conn(0x7f7d100fc280 msgr2=0x7f7d10105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 shutdown_connections 2024-09-12T03:16:11.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:11.880+0000 7f7d1503e640 1 -- 172.21.15.19:0/3669178421 wait complete. 2024-09-12T03:16:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:12 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3669178421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:12.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:12 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3669178421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:12.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:12 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3669178421' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:12.665 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":71,"num_osds":12,"num_up_osds":11,"osd_up_since":1726110949,"num_in_osds":12,"osd_in_since":1726110952,"num_remapped_pgs":0} 2024-09-12T03:16:12.869 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:16:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:16:12.721+0000 7f1aa69cc640 -1 osd.11 0 log_to_monitors true 2024-09-12T03:16:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:13 smithi121 ceph-mon[34963]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:13 smithi121 ceph-mon[34963]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:13 smithi121 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:13 smithi019 ceph-mon[30953]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:13 smithi019 ceph-mon[30953]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:13 smithi019 ceph-mon[30953]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:13 smithi135 ceph-mon[35474]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:13 smithi135 ceph-mon[35474]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:13 smithi135 ceph-mon[35474]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T03:16:13.667 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:16:13.936 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:14.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-mon[35474]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:16:14.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-mon[35474]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:16:14.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:14.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-mon[35474]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:14.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-mon[35474]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:14.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:14 smithi019 ceph-mon[30953]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:16:14.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:14 smithi019 ceph-mon[30953]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:16:14.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:14 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:14.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:14 smithi019 ceph-mon[30953]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:14.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:14 smithi019 ceph-mon[30953]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:14.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 -- 172.21.15.19:0/51829975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c102660 msgr2=0x7f515c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 --2- 172.21.15.19:0/51829975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c102660 0x7f515c104a50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f514c009900 tx=0x7f514c02a410 comp rx=0 tx=0).stop 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 -- 172.21.15.19:0/51829975 shutdown_connections 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 --2- 172.21.15.19:0/51829975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c1078c0 0x7f515c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 --2- 172.21.15.19:0/51829975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c104f90 0x7f515c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 --2- 172.21.15.19:0/51829975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c102660 0x7f515c104a50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.386+0000 7f5160e68640 1 -- 172.21.15.19:0/51829975 >> 172.21.15.19:0/51829975 conn(0x7f515c0fc280 msgr2=0x7f515c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.387+0000 7f5160e68640 1 -- 172.21.15.19:0/51829975 shutdown_connections 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.387+0000 7f5160e68640 1 -- 172.21.15.19:0/51829975 wait complete. 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 Processor -- start 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 -- start start 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c102660 0x7f515c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:14.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c1078c0 0x7f515c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10d8c0 con 0x7f515c104f90 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10d740 con 0x7f515c102660 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5160e68640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f515c10da40 con 0x7f515c1078c0 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5159d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5159d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60032/0 (socket says 172.21.15.19:60032) 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f5159d9b640 1 -- 172.21.15.19:0/3109609742 learned_addr learned my addr 172.21.15.19:0/3109609742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f515ad9d640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c1078c0 0x7f515c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.388+0000 7f515a59c640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c102660 0x7f515c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 -- 172.21.15.19:0/3109609742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c1078c0 msgr2=0x7f515c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c1078c0 0x7f515c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 -- 172.21.15.19:0/3109609742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c102660 msgr2=0x7f515c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c102660 0x7f515c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 -- 172.21.15.19:0/3109609742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515c19fb50 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5159d9b640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9c0 tx=0x7f515000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515000cd80 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5150005500 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f515c19fe40 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150010690 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.389+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f515c1a02f0 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.390+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515c0fdd70 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.392+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f51500046c0 con 0x7f515c104f90 2024-09-12T03:16:14.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.393+0000 7f514b7fe640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 0x7f51440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.393+0000 7f515a59c640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 0x7f51440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.393+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6847+0+0 (secure 0 0 0) 0x7f51500cf090 con 0x7f515c104f90 2024-09-12T03:16:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.394+0000 7f515a59c640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 0x7f51440a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f514c031440 tx=0x7f514c0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:14.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.396+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51500541b0 con 0x7f515c104f90 2024-09-12T03:16:14.574 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:16:14.561+0000 7f1aa294f640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T03:16:14.574 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:16:14 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:16:14.574+0000 7f1a9a76f640 -1 osd.11 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T03:16:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.574+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f515c02d940 con 0x7f515c104f90 2024-09-12T03:16:14.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.574+0000 7f514b7fe640 1 -- 172.21.15.19:0/3109609742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f515009c510 con 0x7f515c104f90 2024-09-12T03:16:14.575 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.576+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 msgr2=0x7f51440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.576+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 0x7f51440a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f514c031440 tx=0x7f514c0315b0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 msgr2=0x7f515c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f515000e9c0 tx=0x7f515000ee80 comp rx=0 tx=0).stop 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 shutdown_connections 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f515c1078c0 0x7f515c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51440a2bb0 0x7f51440a5070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f515c104f90 0x7f515c198460 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 --2- 172.21.15.19:0/3109609742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f515c102660 0x7f515c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 >> 172.21.15.19:0/3109609742 conn(0x7f515c0fc280 msgr2=0x7f515c102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 shutdown_connections 2024-09-12T03:16:14.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:14.577+0000 7f5160e68640 1 -- 172.21.15.19:0/3109609742 wait complete. 2024-09-12T03:16:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:14 smithi121 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T03:16:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:14 smithi121 ceph-mon[34963]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T03:16:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:14 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:14 smithi121 ceph-mon[34963]: from='osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:14 smithi121 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]: dispatch 2024-09-12T03:16:15.308 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726110949,"num_in_osds":12,"osd_in_since":1726110952,"num_remapped_pgs":0} 2024-09-12T03:16:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='osd.11 ' entity='osd.11' 2024-09-12T03:16:15.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3109609742' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:16:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:16:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:15.547 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.547 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.547 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.547 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='osd.11 ' entity='osd.11' 2024-09-12T03:16:15.547 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3109609742' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi135", "root=default"]}]': finished 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='osd.11 ' entity='osd.11' 2024-09-12T03:16:15.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3109609742' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:16.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: purged_snaps scrub starts 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: purged_snaps scrub ok 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: Detected new or changed devices on smithi135 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:16.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] boot 2024-09-12T03:16:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:16:16.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: purged_snaps scrub starts 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: purged_snaps scrub ok 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: Detected new or changed devices on smithi135 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] boot 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:16:16.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:16.578 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:16.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: purged_snaps scrub starts 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: purged_snaps scrub ok 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: Detected new or changed devices on smithi135 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi135 to 1453M 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: osd.11 [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] boot 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: osdmap e74: 12 total, 12 up, 12 in 2024-09-12T03:16:16.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T03:16:17.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 -- 172.21.15.19:0/3533159030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 msgr2=0x7fdb48104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3533159030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48104a90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009900 tx=0x7fdb4402a410 comp rx=0 tx=0).stop 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 -- 172.21.15.19:0/3533159030 shutdown_connections 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3533159030 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48107900 0x7fdb48109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3533159030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48104fd0 0x7fdb481073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3533159030 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48104a90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 -- 172.21.15.19:0/3533159030 >> 172.21.15.19:0/3533159030 conn(0x7fdb480fc2a0 msgr2=0x7fdb480fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 -- 172.21.15.19:0/3533159030 shutdown_connections 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.032+0000 7fdb502b0640 1 -- 172.21.15.19:0/3533159030 wait complete. 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 Processor -- start 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 -- start start 2024-09-12T03:16:17.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48104fd0 0x7fdb481984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48107900 0x7fdb4819f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4810d940 con 0x7fdb481026a0 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4810d7c0 con 0x7fdb48107900 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb502b0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4810dac0 con 0x7fdb48104fd0 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb4e04c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb4e04c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60050/0 (socket says 172.21.15.19:60050) 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb4e04c640 1 -- 172.21.15.19:0/3545423691 learned_addr learned my addr 172.21.15.19:0/3545423691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.033+0000 7fdb4d84b640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48104fd0 0x7fdb481984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e84d640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48107900 0x7fdb4819f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 -- 172.21.15.19:0/3545423691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48104fd0 msgr2=0x7fdb481984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48104fd0 0x7fdb481984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 -- 172.21.15.19:0/3545423691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48107900 msgr2=0x7fdb4819f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48107900 0x7fdb4819f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 -- 172.21.15.19:0/3545423691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819fbd0 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb4e04c640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009230 tx=0x7fdb44004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4403f030 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb44003ce0 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4819fe60 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44035070 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.034+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a02d0 con 0x7fdb481026a0 2024-09-12T03:16:17.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.035+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb480fdd90 con 0x7fdb481026a0 2024-09-12T03:16:17.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.036+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdb44042030 con 0x7fdb481026a0 2024-09-12T03:16:17.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.037+0000 7fdb3f7fe640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 0x7fdb340a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:17.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.037+0000 7fdb4d84b640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 0x7fdb340a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.037+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fdb440ef560 con 0x7fdb481026a0 2024-09-12T03:16:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.037+0000 7fdb4d84b640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 0x7fdb340a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb481994c0 tx=0x7fdb38009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:17.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.040+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb440bc960 con 0x7fdb481026a0 2024-09-12T03:16:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.213+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fdb48062ef0 con 0x7fdb481026a0 2024-09-12T03:16:17.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:17 smithi019 ceph-mon[30953]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:17.214 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:17 smithi019 ceph-mon[30953]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:16:17.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.214+0000 7fdb3f7fe640 1 -- 172.21.15.19:0/3545423691 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7fdb440bc260 con 0x7fdb481026a0 2024-09-12T03:16:17.215 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 msgr2=0x7fdb340a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 0x7fdb340a5070 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdb481994c0 tx=0x7fdb38009110 comp rx=0 tx=0).stop 2024-09-12T03:16:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 msgr2=0x7fdb48197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009230 tx=0x7fdb44004360 comp rx=0 tx=0).stop 2024-09-12T03:16:17.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 shutdown_connections 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb48107900 0x7fdb4819f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdb48104fd0 0x7fdb481984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb340a2bb0 0x7fdb340a5070 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 --2- 172.21.15.19:0/3545423691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdb481026a0 0x7fdb48197fa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 >> 172.21.15.19:0/3545423691 conn(0x7fdb480fc2a0 msgr2=0x7fdb48103060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 shutdown_connections 2024-09-12T03:16:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:17.216+0000 7fdb502b0640 1 -- 172.21.15.19:0/3545423691 wait complete. 2024-09-12T03:16:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:17 smithi135 ceph-mon[35474]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:17 smithi135 ceph-mon[35474]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:16:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:17 smithi121 ceph-mon[34963]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-09-12T03:16:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:17 smithi121 ceph-mon[34963]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T03:16:17.996 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726110975,"num_in_osds":12,"osd_in_since":1726110952,"num_remapped_pgs":0} 2024-09-12T03:16:17.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:16:18.255 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:18.376 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:18 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3545423691' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:18 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3545423691' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:18.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:18 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3545423691' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T03:16:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 -- 172.21.15.19:0/3570939205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 msgr2=0x7fbcd4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3570939205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd4103120 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009900 tx=0x7fbcbc02a480 comp rx=0 tx=0).stop 2024-09-12T03:16:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 -- 172.21.15.19:0/3570939205 shutdown_connections 2024-09-12T03:16:18.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3570939205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd410c170 0x7fbcd410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3570939205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd4103660 0x7fbcd410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3570939205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd4103120 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 -- 172.21.15.19:0/3570939205 >> 172.21.15.19:0/3570939205 conn(0x7fbcd40fc7a0 msgr2=0x7fbcd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 -- 172.21.15.19:0/3570939205 shutdown_connections 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.768+0000 7fbcdab11640 1 -- 172.21.15.19:0/3570939205 wait complete. 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 Processor -- start 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 -- start start 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd4103660 0x7fbcd4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 0x7fbcd4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4111f70 con 0x7fbcd4102d40 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd4111df0 con 0x7fbcd410c170 2024-09-12T03:16:18.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcdab11640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd41120f0 con 0x7fbcd4103660 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60078/0 (socket says 172.21.15.19:60078) 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 -- 172.21.15.19:0/3099282028 learned_addr learned my addr 172.21.15.19:0/3099282028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd90ae640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 0x7fbcd4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 -- 172.21.15.19:0/3099282028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd4103660 msgr2=0x7fbcd4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd4103660 0x7fbcd4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 -- 172.21.15.19:0/3099282028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 msgr2=0x7fbcd4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 0x7fbcd4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd88ad640 1 -- 172.21.15.19:0/3099282028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd41a6990 con 0x7fbcd4102d40 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.769+0000 7fbcd90ae640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 0x7fbcd4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:18.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcd88ad640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005ab0 tx=0x7fbcbc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc03f030 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd41a6c20 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcbc003ce0 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc035070 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.770+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd41a7100 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.771+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd4103120 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.772+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fbcbc042030 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.773+0000 7fbcd1ffb640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 0x7fbcc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.773+0000 7fbcd3fff640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 0x7fbcc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.773+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fbcbc0ef560 con 0x7fbcd4102d40 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.774+0000 7fbcd3fff640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 0x7fbcc00a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4102430 tx=0x7fbcc40071e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:18.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.777+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbcbc0bc960 con 0x7fbcd4102d40 2024-09-12T03:16:18.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.957+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcd4101ed0 con 0x7fbcd4102d40 2024-09-12T03:16:18.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.958+0000 7fbcd1ffb640 1 -- 172.21.15.19:0/3099282028 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+18492 (secure 0 0 0) 0x7fbcbc0bc260 con 0x7fbcd4102d40 2024-09-12T03:16:18.958 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:18.958 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":75,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:16:16.567646+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 msgr2=0x7fbcc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 0x7fbcc00a5040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4102430 tx=0x7fbcc40071e0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 msgr2=0x7fbcd41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc005ab0 tx=0x7fbcbc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 shutdown_connections 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcd410c170 0x7fbcd4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.960+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbcd4103660 0x7fbcd4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.961+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbcc00a2b80 0x7fbcc00a5040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.961+0000 7fbcdab11640 1 --2- 172.21.15.19:0/3099282028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbcd4102d40 0x7fbcd41009a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.961+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 >> 172.21.15.19:0/3099282028 conn(0x7fbcd40fc7a0 msgr2=0x7fbcd40fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:18.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.961+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 shutdown_connections 2024-09-12T03:16:18.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:18.961+0000 7fbcdab11640 1 -- 172.21.15.19:0/3099282028 wait complete. 2024-09-12T03:16:19.298 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:19 smithi019 ceph-mon[30953]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:19.299 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:19 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3099282028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:16:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:19 smithi135 ceph-mon[35474]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:19 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3099282028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:16:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:19 smithi121 ceph-mon[34963]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:19 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3099282028' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:16:19.706 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:12:25.696963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:16:19.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T03:16:19.961 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:20.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.499+0000 7fcf4fb43640 1 -- 172.21.15.19:0/2778869947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 msgr2=0x7fcf48112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.499+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/2778869947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf48112650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcf38009900 tx=0x7fcf3802a480 comp rx=0 tx=0).stop 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 -- 172.21.15.19:0/2778869947 shutdown_connections 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/2778869947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48112b90 0x7fcf48114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/2778869947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf48112650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/2778869947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48074590 0x7fcf48074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 -- 172.21.15.19:0/2778869947 >> 172.21.15.19:0/2778869947 conn(0x7fcf480fd8c0 msgr2=0x7fcf480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 -- 172.21.15.19:0/2778869947 shutdown_connections 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 -- 172.21.15.19:0/2778869947 wait complete. 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 Processor -- start 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.500+0000 7fcf4fb43640 1 -- start start 2024-09-12T03:16:20.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48074590 0x7fcf481a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48112b90 0x7fcf4819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcf48118650 con 0x7fcf48074eb0 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481184d0 con 0x7fcf48112b90 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcf481187d0 con 0x7fcf48074590 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54196/0 (socket says 172.21.15.19:54196) 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 -- 172.21.15.19:0/1798028131 learned_addr learned my addr 172.21.15.19:0/1798028131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d8df640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48074590 0x7fcf481a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4e0e0640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48112b90 0x7fcf4819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 -- 172.21.15.19:0/1798028131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48074590 msgr2=0x7fcf481a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48074590 0x7fcf481a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 -- 172.21.15.19:0/1798028131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48112b90 msgr2=0x7fcf4819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48112b90 0x7fcf4819ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 -- 172.21.15.19:0/1798028131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4819f180 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4d0de640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005ab0 tx=0x7fcf380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3803f030 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf38003ce0 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf38035070 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.501+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4819f410 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.502+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf481ab760 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.503+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf480749f0 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.504+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf38042030 con 0x7fcf48074eb0 2024-09-12T03:16:20.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.504+0000 7fcf3effd640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 0x7fcf340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.504+0000 7fcf4d8df640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 0x7fcf340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.504+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fcf380ef4e0 con 0x7fcf48074eb0 2024-09-12T03:16:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.505+0000 7fcf4d8df640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 0x7fcf340a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf4819fb90 tx=0x7fcf44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:20.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.509+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf380bc8e0 con 0x7fcf48074eb0 2024-09-12T03:16:20.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.687+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcf48105030 con 0x7fcf48074eb0 2024-09-12T03:16:20.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.688+0000 7fcf3effd640 1 -- 172.21.15.19:0/1798028131 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fcf380dae20 con 0x7fcf48074eb0 2024-09-12T03:16:20.688 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-12T03:16:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 msgr2=0x7fcf340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 0x7fcf340a50c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf4819fb90 tx=0x7fcf44008040 comp rx=0 tx=0).stop 2024-09-12T03:16:20.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 msgr2=0x7fcf481a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcf38005ab0 tx=0x7fcf380042f0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 shutdown_connections 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcf48112b90 0x7fcf4819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcf340a2c00 0x7fcf340a50c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcf48074eb0 0x7fcf481a4dd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 --2- 172.21.15.19:0/1798028131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcf48074590 0x7fcf481a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 >> 172.21.15.19:0/1798028131 conn(0x7fcf480fd8c0 msgr2=0x7fcf480ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 shutdown_connections 2024-09-12T03:16:20.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:20.690+0000 7fcf4fb43640 1 -- 172.21.15.19:0/1798028131 wait complete. 2024-09-12T03:16:21.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:21 smithi019 ceph-mon[30953]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:21.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1798028131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:16:21.452 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:21 smithi121 ceph-mon[34963]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:21.452 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1798028131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.a on smithi019 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.d on smithi019 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.g on smithi019 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.j on smithi019 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.b on smithi121 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.e on smithi121 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.h on smithi121 2024-09-12T03:16:21.479 INFO:tasks.cephadm:Adding mds.k on smithi121 2024-09-12T03:16:21.480 INFO:tasks.cephadm:Adding mds.c on smithi135 2024-09-12T03:16:21.480 INFO:tasks.cephadm:Adding mds.f on smithi135 2024-09-12T03:16:21.480 INFO:tasks.cephadm:Adding mds.i on smithi135 2024-09-12T03:16:21.480 INFO:tasks.cephadm:Adding mds.l on smithi135 2024-09-12T03:16:21.480 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l' 2024-09-12T03:16:21.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:21 smithi135 ceph-mon[35474]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:21.509 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1798028131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:16:21.739 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 -- 172.21.15.135:0/674443448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 msgr2=0x7f0f84100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/674443448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f84100890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8002a410 comp rx=0 tx=0).stop 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 -- 172.21.15.135:0/674443448 shutdown_connections 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/674443448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84101770 0x7f0f84101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/674443448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 0x7f0f84101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/674443448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f84100890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.275 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 -- 172.21.15.135:0/674443448 >> 172.21.15.135:0/674443448 conn(0x7f0f840fc2a0 msgr2=0x7f0f840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 -- 172.21.15.135:0/674443448 shutdown_connections 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.259+0000 7f0f8baf9640 1 -- 172.21.15.135:0/674443448 wait complete. 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 Processor -- start 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 -- start start 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 0x7f0f8419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84101770 0x7f0f841a1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410fa80 con 0x7f0f84101770 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410f900 con 0x7f0f84100dd0 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f8baf9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8410fc00 con 0x7f0f84100490 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.276 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.135:48952/0 (socket says 172.21.15.135:48952) 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 -- 172.21.15.135:0/1205926262 learned_addr learned my addr 172.21.15.135:0/1205926262 (peer_addr_for_me v2:172.21.15.135:0/0) 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89094640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 0x7f0f8419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 -- 172.21.15.135:0/1205926262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 msgr2=0x7f0f8419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 0x7f0f8419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 -- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84101770 msgr2=0x7f0f841a1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84101770 0x7f0f841a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.260+0000 7f0f89895640 1 -- 172.21.15.135:0/1205926262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f841a1d70 con 0x7f0f84100490 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f89895640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009230 tx=0x7f0f80004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8003f030 con 0x7f0f84100490 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f80003ce0 con 0x7f0f84100490 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80035070 con 0x7f0f84100490 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f841a2000 con 0x7f0f84100490 2024-09-12T03:16:22.277 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.261+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f841a2530 con 0x7f0f84100490 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.262+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8410b1e0 con 0x7f0f84100490 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.263+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0f80042030 con 0x7f0f84100490 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.264+0000 7f0f7affd640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 0x7f0f700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.264+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f0f800ef590 con 0x7f0f84100490 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.264+0000 7f0f89094640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 0x7f0f700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.264+0000 7f0f89094640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 0x7f0f700a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419b660 tx=0x7f0f7400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:22.278 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.267+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0f800bc990 con 0x7f0f84100490 2024-09-12T03:16:22.441 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.441+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}) -- 0x7f0f840627f0 con 0x7f0f700a2c30 2024-09-12T03:16:22.451 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.451+0000 7f0f7affd640 1 -- 172.21.15.135:0/1205926262 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f0f840627f0 con 0x7f0f700a2c30 2024-09-12T03:16:22.451 INFO:teuthology.orchestra.run.smithi135.stdout:Scheduled mds.all update... 2024-09-12T03:16:22.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 msgr2=0x7f0f700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.453 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 0x7f0f700a50f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f8419b660 tx=0x7f0f7400b390 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 msgr2=0x7f0f8419a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009230 tx=0x7f0f80004360 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 shutdown_connections 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f700a2c30 0x7f0f700a50f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f84101770 0x7f0f841a1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f84100dd0 0x7f0f8419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 --2- 172.21.15.135:0/1205926262 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f84100490 0x7f0f8419a140 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 >> 172.21.15.135:0/1205926262 conn(0x7f0f840fc2a0 msgr2=0x7f0f840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:22.454 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.453+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 shutdown_connections 2024-09-12T03:16:22.455 INFO:teuthology.orchestra.run.smithi135.stderr:2024-09-12T03:16:22.454+0000 7f0f8baf9640 1 -- 172.21.15.135:0/1205926262 wait complete. 2024-09-12T03:16:23.215 DEBUG:teuthology.orchestra.run.smithi019:mds.a> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T03:16:23.218 DEBUG:teuthology.orchestra.run.smithi019:mds.d> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T03:16:23.262 DEBUG:teuthology.orchestra.run.smithi019:mds.g> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T03:16:23.306 DEBUG:teuthology.orchestra.run.smithi019:mds.j> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T03:16:23.349 DEBUG:teuthology.orchestra.run.smithi121:mds.b> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T03:16:23.351 DEBUG:teuthology.orchestra.run.smithi121:mds.e> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T03:16:23.396 DEBUG:teuthology.orchestra.run.smithi121:mds.h> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T03:16:23.440 DEBUG:teuthology.orchestra.run.smithi121:mds.k> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:23.442 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:23.443 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.458 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:23.459 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.460 DEBUG:teuthology.orchestra.run.smithi135:mds.c> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T03:16:23.463 DEBUG:teuthology.orchestra.run.smithi135:mds.f> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T03:16:23.506 DEBUG:teuthology.orchestra.run.smithi135:mds.i> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T03:16:23.549 DEBUG:teuthology.orchestra.run.smithi135:mds.l> sudo journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T03:16:23.591 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T03:16:23.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:16:23.618 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:23.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:23.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:23.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:23.853 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:24.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:24 smithi019 ceph-mon[30953]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:24.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:24 smithi019 ceph-mon[30953]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-09-12T03:16:24.249 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:24 smithi019 ceph-mon[30953]: Deploying daemon mds.l on smithi135 2024-09-12T03:16:24.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 -- 172.21.15.19:0/2297392393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101de0 msgr2=0x7f9f981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 --2- 172.21.15.19:0/2297392393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101de0 0x7f9f981021c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009900 tx=0x7f9f9402a410 comp rx=0 tx=0).stop 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 -- 172.21.15.19:0/2297392393 shutdown_connections 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 --2- 172.21.15.19:0/2297392393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 --2- 172.21.15.19:0/2297392393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98102790 0x7f9f981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 --2- 172.21.15.19:0/2297392393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98101de0 0x7f9f981021c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 -- 172.21.15.19:0/2297392393 >> 172.21.15.19:0/2297392393 conn(0x7f9f980fd8c0 msgr2=0x7f9f980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 -- 172.21.15.19:0/2297392393 shutdown_connections 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.330+0000 7f9f9f848640 1 -- 172.21.15.19:0/2297392393 wait complete. 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 Processor -- start 2024-09-12T03:16:24.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 -- start start 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98101de0 0x7f9f9819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98114130 con 0x7f9f98102790 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f98113fb0 con 0x7f9f98106f10 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9f848640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f981142b0 con 0x7f9f98101de0 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54208/0 (socket says 172.21.15.19:54208) 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 -- 172.21.15.19:0/68603821 learned_addr learned my addr 172.21.15.19:0/68603821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9d5e4640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98101de0 0x7f9f9819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9dde5640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 -- 172.21.15.19:0/68603821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98101de0 msgr2=0x7f9f9819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98101de0 0x7f9f9819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 -- 172.21.15.19:0/68603821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 msgr2=0x7f9f9819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9cde3640 1 -- 172.21.15.19:0/68603821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f9819f560 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.331+0000 7f9f9dde5640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f9cde3640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800d770 tx=0x7f9f8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88017070 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f9819f850 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f88005500 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f88004e00 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.332+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f981a7130 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.333+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f981021c0 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.334+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9f88004fa0 con 0x7f9f98102790 2024-09-12T03:16:24.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.335+0000 7f9f8e7fc640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 0x7f9f840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.335+0000 7f9f9d5e4640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 0x7f9f840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.335+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9f880cea50 con 0x7f9f98102790 2024-09-12T03:16:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.335+0000 7f9f9d5e4640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 0x7f9f840a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009230 tx=0x7f9f940058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:24.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.338+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f8809be50 con 0x7f9f98102790 2024-09-12T03:16:24.458 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:24 smithi135 ceph-mon[35474]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:24.458 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:24 smithi135 ceph-mon[35474]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-09-12T03:16:24.458 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:24 smithi135 ceph-mon[35474]: Deploying daemon mds.l on smithi135 2024-09-12T03:16:24.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.550+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f9819b910 con 0x7f9f98102790 2024-09-12T03:16:24.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.550+0000 7f9f8e7fc640 1 -- 172.21.15.19:0/68603821 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f9f9819b910 con 0x7f9f98102790 2024-09-12T03:16:24.550 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:24.551 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":1,"btime":"2024-09-12T03:09:21:563894+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T03:16:24.551 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 1 2024-09-12T03:16:24.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 msgr2=0x7f9f840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 0x7f9f840a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009230 tx=0x7f9f940058d0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 msgr2=0x7f9f9819aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f8800d770 tx=0x7f9f8800dc30 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 shutdown_connections 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f98106f10 0x7f9f9819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f840a2b80 0x7f9f840a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f98102790 0x7f9f9819aaf0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 --2- 172.21.15.19:0/68603821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f98101de0 0x7f9f9819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 >> 172.21.15.19:0/68603821 conn(0x7f9f980fd8c0 msgr2=0x7f9f980fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 shutdown_connections 2024-09-12T03:16:24.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:24.553+0000 7f9f9f848640 1 -- 172.21.15.19:0/68603821 wait complete. 2024-09-12T03:16:24.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:24 smithi121 ceph-mon[34963]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:16:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:24 smithi121 ceph-mon[34963]: Saving service mds.all spec with placement smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-09-12T03:16:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:24 smithi121 ceph-mon[34963]: Deploying daemon mds.l on smithi135 2024-09-12T03:16:25.044 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:24 smithi135 systemd[1]: Starting Ceph mds.l for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:25.304 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:25 smithi019 ceph-mon[30953]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:25.304 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:25 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/68603821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:25.429 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:25 smithi135 ceph-mon[35474]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:25.429 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:25 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/68603821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:25.429 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 podman[65081]: 2024-09-12 03:16:25.044045306 +0000 UTC m=+0.015884211 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:25.429 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 podman[65081]: 2024-09-12 03:16:25.154218389 +0000 UTC m=+0.126057285 container create 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:16:25.429 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 podman[65081]: 2024-09-12 03:16:25.429670756 +0000 UTC m=+0.401509652 container init 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:16:25.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:25 smithi121 ceph-mon[34963]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:25.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:25 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/68603821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:25.794 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 podman[65081]: 2024-09-12 03:16:25.432445688 +0000 UTC m=+0.404284587 container start 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T03:16:25.794 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 ceph-mds[65100]: -- 172.21.15.135:0/2734996715 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c5ae4cb860 con 0x55c5ae4a8800 2024-09-12T03:16:25.794 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: starting mds.l at 2024-09-12T03:16:25.794 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 bash[65081]: 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 2024-09-12T03:16:25.794 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:16:25 smithi135 systemd[1]: Started Ceph mds.l for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:boot 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: fsmap 1 up:standby 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:26.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:26.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:26.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:26 smithi121 ceph-mon[34963]: Deploying daemon mds.k on smithi121 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:boot 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: fsmap 1 up:standby 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:26.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:26.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:26.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:26 smithi135 ceph-mon[35474]: Deploying daemon mds.k on smithi121 2024-09-12T03:16:26.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:26.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:boot 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: fsmap 1 up:standby 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:26.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:26 smithi019 ceph-mon[30953]: Deploying daemon mds.k on smithi121 2024-09-12T03:16:27.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:27.604 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:27.819 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:27 smithi121 systemd[1]: Starting Ceph mds.k for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:28.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- 172.21.15.19:0/2500034360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38102570 msgr2=0x7f8a381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/2500034360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38102570 0x7f8a381029b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009900 tx=0x7f8a2402a480 comp rx=0 tx=0).stop 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- 172.21.15.19:0/2500034360 shutdown_connections 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/2500034360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 0x7f8a3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/2500034360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38102570 0x7f8a381029b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/2500034360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38100e20 0x7f8a38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- 172.21.15.19:0/2500034360 >> 172.21.15.19:0/2500034360 conn(0x7f8a380fc7d0 msgr2=0x7f8a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- 172.21.15.19:0/2500034360 shutdown_connections 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- 172.21.15.19:0/2500034360 wait complete. 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 Processor -- start 2024-09-12T03:16:28.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.082+0000 7f8a3efb5640 1 -- start start 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38102570 0x7f8a38197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 0x7f8a3819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38110f50 con 0x7f8a38100e20 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8a38110dd0 con 0x7f8a38102570 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381110d0 con 0x7f8a38102ef0 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54216/0 (socket says 172.21.15.19:54216) 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 -- 172.21.15.19:0/3286509981 learned_addr learned my addr 172.21.15.19:0/3286509981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a37fff640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38102570 0x7f8a38197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3d552640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 0x7f8a3819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 -- 172.21.15.19:0/3286509981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 msgr2=0x7f8a3819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 0x7f8a3819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 -- 172.21.15.19:0/3286509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38102570 msgr2=0x7f8a38197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38102570 0x7f8a38197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 -- 172.21.15.19:0/3286509981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3819c400 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3cd51640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d950 tx=0x7f8a2c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c017070 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a2c009c30 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3819c6f0 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.083+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c00f6b0 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.084+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a381a3fd0 con 0x7f8a38100e20 2024-09-12T03:16:28.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.085+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a38101200 con 0x7f8a38100e20 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.086+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8a2c0109d0 con 0x7f8a38100e20 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.086+0000 7f8a35ffb640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.086+0000 7f8a37fff640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.086+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f8a2c0ce810 con 0x7f8a38100e20 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.087+0000 7f8a37fff640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 0x7f8a280a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009230 tx=0x7f8a240058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:28.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.091+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a2c009de0 con 0x7f8a38100e20 2024-09-12T03:16:28.105 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:27 smithi121 podman[64516]: 2024-09-12 03:16:27.721610872 +0000 UTC m=+0.015579487 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:28.105 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:27 smithi121 podman[64516]: 2024-09-12 03:16:27.887141668 +0000 UTC m=+0.181110328 container create 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-12T03:16:28.106 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 podman[64516]: 2024-09-12 03:16:28.106530768 +0000 UTC m=+0.400499371 container init 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:16:28.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.304+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a38109090 con 0x7f8a38100e20 2024-09-12T03:16:28.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.305+0000 7f8a35ffb640 1 -- 172.21.15.19:0/3286509981 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f8a2c09bc10 con 0x7f8a38100e20 2024-09-12T03:16:28.305 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:28.306 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":2,"btime":"2024-09-12T03:16:25:460326+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:16:28.306 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 2 2024-09-12T03:16:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.308+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 msgr2=0x7f8a280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.308+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 0x7f8a280a5040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009230 tx=0x7f8a240058d0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.308+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 msgr2=0x7f8a38197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:28.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.308+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00d950 tx=0x7f8a2c00de10 comp rx=0 tx=0).stop 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 shutdown_connections 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8a38102ef0 0x7f8a3819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8a38102570 0x7f8a38197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8a280a2b80 0x7f8a280a5040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 --2- 172.21.15.19:0/3286509981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8a38100e20 0x7f8a38197450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:28.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 >> 172.21.15.19:0/3286509981 conn(0x7f8a380fc7d0 msgr2=0x7f8a380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:28.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 shutdown_connections 2024-09-12T03:16:28.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:28.309+0000 7f8a3efb5640 1 -- 172.21.15.19:0/3286509981 wait complete. 2024-09-12T03:16:28.399 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 podman[64516]: 2024-09-12 03:16:28.109571292 +0000 UTC m=+0.403539895 container start 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:16:28.400 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mds[64535]: -- 172.21.15.121:0/2684324385 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c463fd860 con 0x561c463db400 2024-09-12T03:16:28.400 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: starting mds.k at 2024-09-12T03:16:28.400 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 bash[64516]: 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd 2024-09-12T03:16:28.400 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:16:28 smithi121 systemd[1]: Started Ceph mds.k for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:28.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:28 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3286509981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:28.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:28.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:28 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3286509981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:28.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:28 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3286509981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:29.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:29 smithi121 ceph-mon[34963]: Deploying daemon mds.j on smithi019 2024-09-12T03:16:29.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:29 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:boot 2024-09-12T03:16:29.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:29 smithi121 ceph-mon[34963]: fsmap 2 up:standby 2024-09-12T03:16:29.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:29 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:16:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:29 smithi019 ceph-mon[30953]: Deploying daemon mds.j on smithi019 2024-09-12T03:16:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:29 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:boot 2024-09-12T03:16:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:29 smithi019 ceph-mon[30953]: fsmap 2 up:standby 2024-09-12T03:16:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:29 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:16:29.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:29 smithi135 ceph-mon[35474]: Deploying daemon mds.j on smithi019 2024-09-12T03:16:29.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:29 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:boot 2024-09-12T03:16:29.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:29 smithi135 ceph-mon[35474]: fsmap 2 up:standby 2024-09-12T03:16:29.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:29 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:16:30.611 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:30 smithi019 ceph-mon[30953]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:30.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:30 smithi135 ceph-mon[35474]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:30.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:30 smithi121 ceph-mon[34963]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:30.914 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:30 smithi019 systemd[1]: Starting Ceph mds.j for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:31.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:31.286 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 podman[69864]: 2024-09-12 03:16:30.915223225 +0000 UTC m=+0.016247245 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:31.286 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 podman[69864]: 2024-09-12 03:16:31.016548442 +0000 UTC m=+0.117572454 container create 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-12T03:16:31.286 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 podman[69864]: 2024-09-12 03:16:31.286184165 +0000 UTC m=+0.387208201 container init 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True) 2024-09-12T03:16:31.473 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:31.596 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 podman[69864]: 2024-09-12 03:16:31.290539997 +0000 UTC m=+0.391564014 container start 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:16:31.597 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 ceph-mds[69912]: -- 172.21.15.19:0/1092933810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564127903860 con 0x5641278e1400 2024-09-12T03:16:31.597 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j[69898]: starting mds.j at 2024-09-12T03:16:31.597 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 bash[69864]: 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 2024-09-12T03:16:31.597 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:16:31 smithi019 systemd[1]: Started Ceph mds.j for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 -- 172.21.15.19:0/3432284841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 msgr2=0x7fcaa81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 --2- 172.21.15.19:0/3432284841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa81021c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca98009900 tx=0x7fca9802a410 comp rx=0 tx=0).stop 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 -- 172.21.15.19:0/3432284841 shutdown_connections 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 --2- 172.21.15.19:0/3432284841 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8106f10 0x7fcaa810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 --2- 172.21.15.19:0/3432284841 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8102790 0x7fcaa81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 --2- 172.21.15.19:0/3432284841 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa81021c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 -- 172.21.15.19:0/3432284841 >> 172.21.15.19:0/3432284841 conn(0x7fcaa80fd8c0 msgr2=0x7fcaa80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:31.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 -- 172.21.15.19:0/3432284841 shutdown_connections 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.964+0000 7fcaacdac640 1 -- 172.21.15.19:0/3432284841 wait complete. 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 Processor -- start 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 -- start start 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8102790 0x7fcaa8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 0x7fcaa819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8111f30 con 0x7fcaa8101de0 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa8111db0 con 0x7fcaa8102790 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaacdac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa81120b0 con 0x7fcaa8106f10 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaa659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaa659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55976/0 (socket says 172.21.15.19:55976) 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fcaa659c640 1 -- 172.21.15.19:0/223533329 learned_addr learned my addr 172.21.15.19:0/223533329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:31.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.965+0000 7fca9dd9b640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8102790 0x7fcaa8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa6d9d640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 0x7fcaa819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 -- 172.21.15.19:0/223533329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 msgr2=0x7fcaa819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 0x7fcaa819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 -- 172.21.15.19:0/223533329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8102790 msgr2=0x7fcaa8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8102790 0x7fcaa8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 -- 172.21.15.19:0/223533329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa819d3e0 con 0x7fcaa8101de0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa6d9d640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 0x7fcaa819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaa659c640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca98005ab0 tx=0x7fca980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9803f030 con 0x7fcaa8101de0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa819d670 con 0x7fcaa8101de0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca98003ce0 con 0x7fcaa8101de0 2024-09-12T03:16:31.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98035070 con 0x7fcaa8101de0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.966+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa81a4fb0 con 0x7fcaa8101de0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.967+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa81021c0 con 0x7fcaa8101de0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.968+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fca98042030 con 0x7fcaa8101de0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.969+0000 7fca9ffff640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.969+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fca980ef210 con 0x7fcaa8101de0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.969+0000 7fca9dd9b640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.970+0000 7fca9dd9b640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 0x7fca940a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80fdd00 tx=0x7fca9000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:31.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:31.973+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca980bc570 con 0x7fcaa8101de0 2024-09-12T03:16:32.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.188+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa810a0f0 con 0x7fcaa8101de0 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.189+0000 7fca9ffff640 1 -- 172.21.15.19:0/223533329 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7fca980bbe70 con 0x7fcaa8101de0 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":4,"btime":"2024-09-12T03:16:31:480257+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:16:32.189 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 4 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 msgr2=0x7fca940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 0x7fca940a5040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80fdd00 tx=0x7fca9000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 msgr2=0x7fcaa8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fca98005ab0 tx=0x7fca980042f0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 shutdown_connections 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcaa8106f10 0x7fcaa819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcaa8102790 0x7fcaa8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fca940a2b80 0x7fca940a5040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 --2- 172.21.15.19:0/223533329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcaa8101de0 0x7fcaa8198430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 >> 172.21.15.19:0/223533329 conn(0x7fcaa80fd8c0 msgr2=0x7fcaa80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 shutdown_connections 2024-09-12T03:16:32.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:32.191+0000 7fcaacdac640 1 -- 172.21.15.19:0/223533329 wait complete. 2024-09-12T03:16:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: Deploying daemon mds.d on smithi019 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6834/1760992520,v1:172.21.15.19:6835/1760992520] up:boot 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: fsmap 3 up:standby 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:16:32.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:32 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/223533329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: Deploying daemon mds.d on smithi019 2024-09-12T03:16:32.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6834/1760992520,v1:172.21.15.19:6835/1760992520] up:boot 2024-09-12T03:16:32.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: fsmap 3 up:standby 2024-09-12T03:16:32.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:16:32.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:32 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/223533329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: Deploying daemon mds.d on smithi019 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6834/1760992520,v1:172.21.15.19:6835/1760992520] up:boot 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: fsmap 3 up:standby 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T03:16:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:32 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/223533329' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:34 smithi019 ceph-mon[30953]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:34.665 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:34 smithi019 systemd[1]: Starting Ceph mds.d for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:34 smithi135 ceph-mon[35474]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:34.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:34 smithi121 ceph-mon[34963]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:34.975 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:34 smithi019 podman[70275]: 2024-09-12 03:16:34.625526505 +0000 UTC m=+0.016420977 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:34.975 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:34 smithi019 podman[70275]: 2024-09-12 03:16:34.731984282 +0000 UTC m=+0.122878756 container create 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T03:16:34.976 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:34 smithi019 podman[70275]: 2024-09-12 03:16:34.976191808 +0000 UTC m=+0.367086280 container init 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:16:35.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:35.384 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:35.414 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:34 smithi019 podman[70275]: 2024-09-12 03:16:34.979231699 +0000 UTC m=+0.370126171 container start 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:16:35.414 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:35 smithi019 ceph-mds[70294]: -- 172.21.15.19:0/4270688804 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562c1e419860 con 0x562c1e3f6800 2024-09-12T03:16:35.414 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d[70290]: starting mds.d at 2024-09-12T03:16:35.414 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:35 smithi019 bash[70275]: 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 2024-09-12T03:16:35.415 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:16:35 smithi019 systemd[1]: Started Ceph mds.d for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:35.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- 172.21.15.19:0/701280901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8881036e0 msgr2=0x7fa888107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/701280901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8881036e0 0x7fa888107b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa87c009930 tx=0x7fa87c02a420 comp rx=0 tx=0).stop 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- 172.21.15.19:0/701280901 shutdown_connections 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/701280901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa88810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/701280901 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa8881036e0 0x7fa888107b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/701280901 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa888102d30 0x7fa888103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- 172.21.15.19:0/701280901 >> 172.21.15.19:0/701280901 conn(0x7fa8880fc7d0 msgr2=0x7fa8880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- 172.21.15.19:0/701280901 shutdown_connections 2024-09-12T03:16:35.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- 172.21.15.19:0/701280901 wait complete. 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 Processor -- start 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.873+0000 7fa88fe2b640 1 -- start start 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8881036e0 0x7fa888100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa888101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa8881120c0 con 0x7fa888102d30 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa888111f40 con 0x7fa8881082a0 2024-09-12T03:16:35.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88fe2b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa888112240 con 0x7fa8881036e0 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56008/0 (socket says 172.21.15.19:56008) 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88e3c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa888101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 -- 172.21.15.19:0/3373382623 learned_addr learned my addr 172.21.15.19:0/3373382623 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88d3c6640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8881036e0 0x7fa888100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 -- 172.21.15.19:0/3373382623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8881036e0 msgr2=0x7fa888100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8881036e0 0x7fa888100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 -- 172.21.15.19:0/3373382623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 msgr2=0x7fa888101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:35.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa888101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 -- 172.21.15.19:0/3373382623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8881a6a30 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88e3c8640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa888101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.874+0000 7fa88dbc7640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa87800ba50 tx=0x7fa87800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.875+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878019070 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.875+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa87800cbe0 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.875+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8881a6d20 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.875+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8780076e0 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.875+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8881a71d0 con 0x7fa888102d30 2024-09-12T03:16:35.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.876+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88806a850 con 0x7fa888102d30 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.877+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa87801c070 con 0x7fa888102d30 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.877+0000 7fa886ffd640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 0x7fa8700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.878+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa8780ce630 con 0x7fa888102d30 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.878+0000 7fa88d3c6640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 0x7fa8700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.878+0000 7fa88d3c6640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 0x7fa8700a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005940 tx=0x7fa87c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:35.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:35.881+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa87809b3e0 con 0x7fa888102d30 2024-09-12T03:16:36.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.091+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa888103110 con 0x7fa888102d30 2024-09-12T03:16:36.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.092+0000 7fa886ffd640 1 -- 172.21.15.19:0/3373382623 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7fa888103110 con 0x7fa888102d30 2024-09-12T03:16:36.092 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:36.092 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"btime":"2024-09-12T03:16:35:458563+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T03:16:36.093 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 5 2024-09-12T03:16:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 msgr2=0x7fa8700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:36.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 0x7fa8700a5040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa87c005940 tx=0x7fa87c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 msgr2=0x7fa888100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa87800ba50 tx=0x7fa87800bf10 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 shutdown_connections 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa8881082a0 0x7fa888101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa8881036e0 0x7fa888100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa8700a2b80 0x7fa8700a5040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 --2- 172.21.15.19:0/3373382623 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa888102d30 0x7fa888100990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 >> 172.21.15.19:0/3373382623 conn(0x7fa8880fc7d0 msgr2=0x7fa888105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 shutdown_connections 2024-09-12T03:16:36.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:36.094+0000 7fa88fe2b640 1 -- 172.21.15.19:0/3373382623 wait complete. 2024-09-12T03:16:36.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: Deploying daemon mds.e on smithi121 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6836/2911685711,v1:172.21.15.19:6837/2911685711] up:boot 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: fsmap 4 up:standby 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:16:36.393 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:36 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3373382623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: Deploying daemon mds.e on smithi121 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6836/2911685711,v1:172.21.15.19:6837/2911685711] up:boot 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: fsmap 4 up:standby 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:16:36.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:36 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3373382623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: Deploying daemon mds.e on smithi121 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6836/2911685711,v1:172.21.15.19:6837/2911685711] up:boot 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: fsmap 4 up:standby 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T03:16:36.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:36 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3373382623' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:37.365 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:37 smithi121 ceph-mon[34963]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:37.365 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 systemd[1]: Starting Ceph mds.e for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:37 smithi019 ceph-mon[30953]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:37 smithi135 ceph-mon[35474]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:37.650 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 podman[64832]: 2024-09-12 03:16:37.267779819 +0000 UTC m=+0.015708383 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:37.650 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 podman[64832]: 2024-09-12 03:16:37.411567824 +0000 UTC m=+0.159496379 container create e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:16:38.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 podman[64832]: 2024-09-12 03:16:37.687105753 +0000 UTC m=+0.435034310 container init e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:16:38.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 podman[64832]: 2024-09-12 03:16:37.690224772 +0000 UTC m=+0.438153326 container start e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:16:38.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 ceph-mds[64851]: -- 172.21.15.121:0/1304215176 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c192267860 con 0x55c192245400 2024-09-12T03:16:38.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: starting mds.e at 2024-09-12T03:16:38.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 bash[64832]: e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c 2024-09-12T03:16:38.151 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:16:37 smithi121 systemd[1]: Started Ceph mds.e for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:38.709 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:38.987 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:39.108 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: Deploying daemon mds.f on smithi135 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6834/3114415604,v1:172.21.15.121:6835/3114415604] up:boot 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: fsmap 5 up:standby 2024-09-12T03:16:39.109 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:39.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:39.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:39.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: Deploying daemon mds.f on smithi135 2024-09-12T03:16:39.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6834/3114415604,v1:172.21.15.121:6835/3114415604] up:boot 2024-09-12T03:16:39.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: fsmap 5 up:standby 2024-09-12T03:16:39.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:16:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: Deploying daemon mds.f on smithi135 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6834/3114415604,v1:172.21.15.121:6835/3114415604] up:boot 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: fsmap 5 up:standby 2024-09-12T03:16:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 -- 172.21.15.19:0/1961298106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770101380 msgr2=0x7f177010dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 --2- 172.21.15.19:0/1961298106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770101380 0x7f177010dfb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1764009900 tx=0x7f176402a410 comp rx=0 tx=0).stop 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 -- 172.21.15.19:0/1961298106 shutdown_connections 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 --2- 172.21.15.19:0/1961298106 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 0x7f1770110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 --2- 172.21.15.19:0/1961298106 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1770101380 0x7f177010dfb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 --2- 172.21.15.19:0/1961298106 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17701009d0 0x7f1770100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 -- 172.21.15.19:0/1961298106 >> 172.21.15.19:0/1961298106 conn(0x7f17700fc7d0 msgr2=0x7f17700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 -- 172.21.15.19:0/1961298106 shutdown_connections 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.500+0000 7f1774dc3640 1 -- 172.21.15.19:0/1961298106 wait complete. 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.501+0000 7f1774dc3640 1 Processor -- start 2024-09-12T03:16:39.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.501+0000 7f1774dc3640 1 -- start start 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.501+0000 7f1774dc3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f1774dc3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1770101380 0x7f17701a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f1774dc3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 0x7f177019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f1774dc3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1770114240 con 0x7f17701009d0 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f1774dc3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17701140c0 con 0x7f1770101380 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f1774dc3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17701143c0 con 0x7f177010e4f0 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56016/0 (socket says 172.21.15.19:56016) 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 -- 172.21.15.19:0/2084258731 learned_addr learned my addr 172.21.15.19:0/2084258731 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176dd9b640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1770101380 0x7f17701a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176ed9d640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 0x7f177019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 -- 172.21.15.19:0/2084258731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 msgr2=0x7f177019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 0x7f177019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 -- 172.21.15.19:0/2084258731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1770101380 msgr2=0x7f17701a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1770101380 0x7f17701a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 -- 172.21.15.19:0/2084258731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f177019ad00 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f176e59c640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f176000c890 tx=0x7f176000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.502+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760004220 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.503+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17600043c0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.503+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f177019aff0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.503+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17600049e0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.503+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701a72e0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.504+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1770100db0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.506+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f17600079b0 con 0x7f17701009d0 2024-09-12T03:16:39.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.506+0000 7f175f7fe640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 0x7f17580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.507+0000 7f176dd9b640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 0x7f17580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.507+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f17600ceb00 con 0x7f17701009d0 2024-09-12T03:16:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.507+0000 7f176dd9b640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 0x7f17580a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1764009230 tx=0x7f17640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:39.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.509+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1760014030 con 0x7f17701009d0 2024-09-12T03:16:39.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.726+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f177019bb00 con 0x7f17701009d0 2024-09-12T03:16:39.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.726+0000 7f175f7fe640 1 -- 172.21.15.19:0/2084258731 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4741 (secure 0 0 0) 0x7f176009b930 con 0x7f17701009d0 2024-09-12T03:16:39.727 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:39.728 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":6,"btime":"2024-09-12T03:16:38:140710+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T03:16:39.728 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 6 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 msgr2=0x7f17580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 0x7f17580a50c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1764009230 tx=0x7f17640058d0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 msgr2=0x7f17701a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f176000c890 tx=0x7f176000cd50 comp rx=0 tx=0).stop 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 shutdown_connections 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f177010e4f0 0x7f177019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1770101380 0x7f17701a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17580a2c00 0x7f17580a50c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 --2- 172.21.15.19:0/2084258731 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17701009d0 0x7f17701a0410 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 >> 172.21.15.19:0/2084258731 conn(0x7f17700fc7d0 msgr2=0x7f177010f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 shutdown_connections 2024-09-12T03:16:39.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:39.729+0000 7f1774dc3640 1 -- 172.21.15.19:0/2084258731 wait complete. 2024-09-12T03:16:40.017 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:39 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2084258731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:40.058 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:39 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2084258731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:40.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:39 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2084258731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:40.794 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:16:40 smithi135 systemd[1]: Started Ceph mds.f for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:41.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:41.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.177 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:41.177 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.177 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.177 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:41.178 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:41.178 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:41.178 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:41.973 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:41 smithi135 ceph-mon[35474]: Deploying daemon mds.c on smithi135 2024-09-12T03:16:41.974 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:41 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6834/3494935988,v1:172.21.15.135:6835/3494935988] up:boot 2024-09-12T03:16:41.974 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:41 smithi135 ceph-mon[35474]: fsmap 6 up:standby 2024-09-12T03:16:41.974 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:16:42.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:41 smithi121 ceph-mon[34963]: Deploying daemon mds.c on smithi135 2024-09-12T03:16:42.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:41 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6834/3494935988,v1:172.21.15.135:6835/3494935988] up:boot 2024-09-12T03:16:42.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:41 smithi121 ceph-mon[34963]: fsmap 6 up:standby 2024-09-12T03:16:42.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:16:42.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:41 smithi019 ceph-mon[30953]: Deploying daemon mds.c on smithi135 2024-09-12T03:16:42.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:41 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6834/3494935988,v1:172.21.15.135:6835/3494935988] up:boot 2024-09-12T03:16:42.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:41 smithi019 ceph-mon[30953]: fsmap 6 up:standby 2024-09-12T03:16:42.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:16:42.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:42.544 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:42 smithi135 systemd[1]: Starting Ceph mds.c for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:42.674 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:42.979 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:42 smithi135 ceph-mon[35474]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:42.980 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:42 smithi135 podman[65682]: 2024-09-12 03:16:42.579345854 +0000 UTC m=+0.015661480 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:42.980 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:42 smithi135 podman[65682]: 2024-09-12 03:16:42.701757355 +0000 UTC m=+0.138072975 container create 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:16:42.980 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:42 smithi135 podman[65682]: 2024-09-12 03:16:42.980026421 +0000 UTC m=+0.416342045 container init 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-12T03:16:43.114 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:42 smithi019 ceph-mon[30953]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:43.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:42 smithi121 ceph-mon[34963]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 -- 172.21.15.19:0/1725366288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100480 msgr2=0x7f00e8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/1725366288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100480 0x7f00e8100860 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009900 tx=0x7f00d802a410 comp rx=0 tx=0).stop 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 -- 172.21.15.19:0/1725366288 shutdown_connections 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/1725366288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e81055b0 0x7f00e810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/1725366288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e8100e30 0x7f00e8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/1725366288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100480 0x7f00e8100860 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 -- 172.21.15.19:0/1725366288 >> 172.21.15.19:0/1725366288 conn(0x7f00e8077210 msgr2=0x7f00e80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 -- 172.21.15.19:0/1725366288 shutdown_connections 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 -- 172.21.15.19:0/1725366288 wait complete. 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.202+0000 7f00ee0a2640 1 Processor -- start 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 -- start start 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e8100480 0x7f00e8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e81055b0 0x7f00e819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8110e60 con 0x7f00e8100e30 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8110ce0 con 0x7f00e8100480 2024-09-12T03:16:43.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00ee0a2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00e8110fe0 con 0x7f00e81055b0 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34386/0 (socket says 172.21.15.19:34386) 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 -- 172.21.15.19:0/693868753 learned_addr learned my addr 172.21.15.19:0/693868753 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e8100480 0x7f00e8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e7fff640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e81055b0 0x7f00e819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 -- 172.21.15.19:0/693868753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e81055b0 msgr2=0x7f00e819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e81055b0 0x7f00e819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 -- 172.21.15.19:0/693868753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e8100480 msgr2=0x7f00e8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e8100480 0x7f00e8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 -- 172.21.15.19:0/693868753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e819d3f0 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.203+0000 7f00e6ffd640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9f0 tx=0x7f00dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.204+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00cd80 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.204+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00dc010910 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.204+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e819d6e0 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.204+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc018990 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.204+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a4fc0 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.205+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8100860 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.206+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f00dc010ac0 con 0x7f00e8100e30 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.206+0000 7f00e4ff9640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.207+0000 7f00e77fe640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:43.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.207+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f00dc0cee10 con 0x7f00e8100e30 2024-09-12T03:16:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.207+0000 7f00e77fe640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 0x7f00d00a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009230 tx=0x7f00d80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:43.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.211+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f00dc09bbc0 con 0x7f00e8100e30 2024-09-12T03:16:43.294 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:42 smithi135 podman[65682]: 2024-09-12 03:16:42.983029694 +0000 UTC m=+0.419345325 container start 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:16:43.294 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:43 smithi135 ceph-mds[65702]: -- 172.21.15.135:0/612695714 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a38dbeb860 con 0x55a38dbc9400 2024-09-12T03:16:43.294 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: starting mds.c at 2024-09-12T03:16:43.294 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:43 smithi135 bash[65682]: 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb 2024-09-12T03:16:43.294 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:16:43 smithi135 systemd[1]: Started Ceph mds.c for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:43.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.422+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00e810a800 con 0x7f00e8100e30 2024-09-12T03:16:43.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.423+0000 7f00e4ff9640 1 -- 172.21.15.19:0/693868753 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5569 (secure 0 0 0) 0x7f00dc09fec0 con 0x7f00e8100e30 2024-09-12T03:16:43.423 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:43.423 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":7,"btime":"2024-09-12T03:16:40:842132+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-12T03:16:43.424 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 7 2024-09-12T03:16:43.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 msgr2=0x7f00d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 0x7f00d00a50c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009230 tx=0x7f00d80058d0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 msgr2=0x7f00e8198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00e9f0 tx=0x7f00dc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 shutdown_connections 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00e81055b0 0x7f00e819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00d00a2c00 0x7f00d00a50c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00e8100e30 0x7f00e8198980 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 --2- 172.21.15.19:0/693868753 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00e8100480 0x7f00e8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 >> 172.21.15.19:0/693868753 conn(0x7f00e8077210 msgr2=0x7f00e80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 shutdown_connections 2024-09-12T03:16:43.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:43.425+0000 7f00ee0a2640 1 -- 172.21.15.19:0/693868753 wait complete. 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: Deploying daemon mds.h on smithi121 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/693868753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:boot 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: fsmap 7 up:standby 2024-09-12T03:16:44.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:44.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: Deploying daemon mds.h on smithi121 2024-09-12T03:16:44.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/693868753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:44.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:boot 2024-09-12T03:16:44.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: fsmap 7 up:standby 2024-09-12T03:16:44.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:16:44.623 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.623 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: Deploying daemon mds.h on smithi121 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/693868753' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:boot 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: fsmap 7 up:standby 2024-09-12T03:16:44.624 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:16:45.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:45 smithi135 ceph-mon[35474]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:45.568 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:45 smithi121 ceph-mon[34963]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:45.568 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 systemd[1]: Starting Ceph mds.h for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:45 smithi019 ceph-mon[30953]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:45.858 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 podman[65130]: 2024-09-12 03:16:45.470659493 +0000 UTC m=+0.016018020 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:45.859 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 podman[65130]: 2024-09-12 03:16:45.590417633 +0000 UTC m=+0.135776156 container create a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:16:45.859 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 podman[65130]: 2024-09-12 03:16:45.859209373 +0000 UTC m=+0.404567894 container init a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:16:46.069 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:46.150 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 podman[65130]: 2024-09-12 03:16:45.86225125 +0000 UTC m=+0.407609778 container start a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T03:16:46.150 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 ceph-mds[65149]: -- 172.21.15.121:0/3542424051 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569630df860 con 0x5569630bc800 2024-09-12T03:16:46.151 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: starting mds.h at 2024-09-12T03:16:46.151 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 bash[65130]: a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 2024-09-12T03:16:46.151 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:16:45 smithi121 systemd[1]: Started Ceph mds.h for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:46.338 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 -- 172.21.15.19:0/620229107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 msgr2=0x7f3ca010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/620229107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca010e610 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009ca0 tx=0x7f3c9802a950 comp rx=0 tx=0).stop 2024-09-12T03:16:46.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 -- 172.21.15.19:0/620229107 shutdown_connections 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/620229107 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca010e610 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/620229107 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/620229107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 0x7f3ca0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 -- 172.21.15.19:0/620229107 >> 172.21.15.19:0/620229107 conn(0x7f3ca00fc7a0 msgr2=0x7f3ca00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 -- 172.21.15.19:0/620229107 shutdown_connections 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.921+0000 7f3ca75bb640 1 -- 172.21.15.19:0/620229107 wait complete. 2024-09-12T03:16:46.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 Processor -- start 2024-09-12T03:16:46.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 -- start start 2024-09-12T03:16:46.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 0x7f3ca01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca0111f50 con 0x7f3ca010c170 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca0111dd0 con 0x7f3ca0102d40 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.922+0000 7f3ca75bb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca01120d0 con 0x7f3ca0103660 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca5357640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 0x7f3ca01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca5b58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45450/0 (socket says 172.21.15.19:45450) 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 -- 172.21.15.19:0/3116956838 learned_addr learned my addr 172.21.15.19:0/3116956838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 -- 172.21.15.19:0/3116956838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 msgr2=0x7f3ca01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 0x7f3ca01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 -- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 msgr2=0x7f3ca0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 -- 172.21.15.19:0/3116956838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca019c4e0 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca5b58640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3ca4b56640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d750 tx=0x7f3c9000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90017070 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.923+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c90009820 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.924+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca019c7d0 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.924+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90004c60 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.924+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca019ccc0 con 0x7f3ca0103660 2024-09-12T03:16:46.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.924+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca006b070 con 0x7f3ca0103660 2024-09-12T03:16:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.926+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3c90005220 con 0x7f3ca0103660 2024-09-12T03:16:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.926+0000 7f3c967fc640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 0x7f3c8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.926+0000 7f3ca5357640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 0x7f3c8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:46.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.927+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3c900ce4a0 con 0x7f3ca0103660 2024-09-12T03:16:46.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.927+0000 7f3ca5357640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 0x7f3c8c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00abe0 tx=0x7f3c9c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:46.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:46.929+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c9009b800 con 0x7f3ca0103660 2024-09-12T03:16:47.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.144+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ca019cf20 con 0x7f3ca0103660 2024-09-12T03:16:47.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.145+0000 7f3c967fc640 1 -- 172.21.15.19:0/3116956838 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7222 (secure 0 0 0) 0x7f3c9009b100 con 0x7f3ca0103660 2024-09-12T03:16:47.145 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:47.146 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":9,"btime":"2024-09-12T03:16:46:170205+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T03:16:47.146 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 9 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 msgr2=0x7f3c8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 0x7f3c8c0a5040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c00abe0 tx=0x7f3c9c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 msgr2=0x7f3ca0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000d750 tx=0x7f3c9000db50 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 shutdown_connections 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c8c0a2b80 0x7f3c8c0a5040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ca010c170 0x7f3ca0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ca0103660 0x7f3ca0100ee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 --2- 172.21.15.19:0/3116956838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ca0102d40 0x7f3ca01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:47.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 >> 172.21.15.19:0/3116956838 conn(0x7f3ca00fc7a0 msgr2=0x7f3ca00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 shutdown_connections 2024-09-12T03:16:47.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:47.148+0000 7f3ca75bb640 1 -- 172.21.15.19:0/3116956838 wait complete. 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: Deploying daemon mds.i on smithi135 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6836/826498871,v1:172.21.15.121:6837/826498871] up:boot 2024-09-12T03:16:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: fsmap 8 up:standby 2024-09-12T03:16:47.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:47 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: Deploying daemon mds.i on smithi135 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6836/826498871,v1:172.21.15.121:6837/826498871] up:boot 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: fsmap 8 up:standby 2024-09-12T03:16:47.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:47 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:16:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: Deploying daemon mds.i on smithi135 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6836/826498871,v1:172.21.15.121:6837/826498871] up:boot 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: fsmap 8 up:standby 2024-09-12T03:16:47.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:47 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:16:48.025 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:47 smithi135 systemd[1]: Starting Ceph mds.i for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:48.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:48 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3116956838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:48.294 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 podman[65997]: 2024-09-12 03:16:48.165770868 +0000 UTC m=+0.109147453 container create 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:16:48.294 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 podman[65997]: 2024-09-12 03:16:48.072876908 +0000 UTC m=+0.016253505 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:48.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:48 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3116956838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:48.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:48 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3116956838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 podman[65997]: 2024-09-12 03:16:48.435098577 +0000 UTC m=+0.378475162 container init 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 podman[65997]: 2024-09-12 03:16:48.438081597 +0000 UTC m=+0.381458194 container start 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 ceph-mds[66016]: -- 172.21.15.135:0/323106396 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df2d1d1860 con 0x55df2d1ae800 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: starting mds.i at 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 bash[65997]: 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 2024-09-12T03:16:48.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:16:48 smithi135 systemd[1]: Started Ceph mds.i for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:49.280 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:49.280 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.280 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:49.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:49.281 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:49 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:49 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:49.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:49 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:49.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:50 smithi135 ceph-mon[35474]: Deploying daemon mds.g on smithi019 2024-09-12T03:16:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:50 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:boot 2024-09-12T03:16:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:50 smithi135 ceph-mon[35474]: fsmap 9 up:standby 2024-09-12T03:16:50.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:50 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:16:50.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:50 smithi019 ceph-mon[30953]: Deploying daemon mds.g on smithi019 2024-09-12T03:16:50.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:50 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:boot 2024-09-12T03:16:50.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:50 smithi019 ceph-mon[30953]: fsmap 9 up:standby 2024-09-12T03:16:50.346 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:50 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:16:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:50 smithi121 ceph-mon[34963]: Deploying daemon mds.g on smithi019 2024-09-12T03:16:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:50 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:boot 2024-09-12T03:16:50.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:50 smithi121 ceph-mon[34963]: fsmap 9 up:standby 2024-09-12T03:16:50.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:50 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/2339950603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 msgr2=0x7fa4f8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/2339950603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8101200 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8009900 tx=0x7fa4e802a480 comp rx=0 tx=0).stop 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/2339950603 shutdown_connections 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/2339950603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102ef0 0x7fa4f810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/2339950603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102570 0x7fa4f81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.783+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/2339950603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8101200 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.784+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/2339950603 >> 172.21.15.19:0/2339950603 conn(0x7fa4f80fc7d0 msgr2=0x7fa4f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.784+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/2339950603 shutdown_connections 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.784+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/2339950603 wait complete. 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.784+0000 7fa4fe9a8640 1 Processor -- start 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.784+0000 7fa4fe9a8640 1 -- start start 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102570 0x7fa4f81998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 0x7fa4f81a4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8110f80 con 0x7fa4f8100e20 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8110e00 con 0x7fa4f8102570 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fe9a8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f8111100 con 0x7fa4f8102ef0 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49728/0 (socket says 172.21.15.19:49728) 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 -- 172.21.15.19:0/1694507707 learned_addr learned my addr 172.21.15.19:0/1694507707 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f77fe640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102570 0x7fa4f81998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fcf45640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 0x7fa4f81a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 -- 172.21.15.19:0/1694507707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 msgr2=0x7fa4f81a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 0x7fa4f81a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 -- 172.21.15.19:0/1694507707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102570 msgr2=0x7fa4f81998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102570 0x7fa4f81998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4f7fff640 1 -- 172.21.15.19:0/1694507707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f81a4940 con 0x7fa4f8100e20 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.785+0000 7fa4fcf45640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 0x7fa4f81a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4f7fff640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006010 tx=0x7fa4e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e803f070 con 0x7fa4f8100e20 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f81a4b10 con 0x7fa4f8100e20 2024-09-12T03:16:50.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4e8031c30 con 0x7fa4f8100e20 2024-09-12T03:16:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8035030 con 0x7fa4f8100e20 2024-09-12T03:16:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.786+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f81a4fc0 con 0x7fa4f8100e20 2024-09-12T03:16:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.787+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f8101200 con 0x7fa4f8100e20 2024-09-12T03:16:50.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.788+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa4e8042070 con 0x7fa4f8100e20 2024-09-12T03:16:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.789+0000 7fa4f57fa640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.789+0000 7fa4f77fe640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.789+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fa4e80ef2b0 con 0x7fa4f8100e20 2024-09-12T03:16:50.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.789+0000 7fa4f77fe640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 0x7fa4e00a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f806aac0 tx=0x7fa4ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:50.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:50.794+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa4e8031de0 con 0x7fa4f8100e20 2024-09-12T03:16:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.020+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4f8109090 con 0x7fa4f8100e20 2024-09-12T03:16:51.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.020+0000 7fa4f57fa640 1 -- 172.21.15.19:0/1694507707 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8052 (secure 0 0 0) 0x7fa4e80bc6b0 con 0x7fa4f8100e20 2024-09-12T03:16:51.021 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:51.021 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"btime":"2024-09-12T03:16:49:026201+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:16:51.021 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 10 2024-09-12T03:16:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.022+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 msgr2=0x7fa4e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.022+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 0x7fa4e00a5040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa4f806aac0 tx=0x7fa4ec005f90 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.022+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 msgr2=0x7fa4f8199380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.022+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8006010 tx=0x7fa4e8003c10 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 shutdown_connections 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa4f8102ef0 0x7fa4f81a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa4f8102570 0x7fa4f81998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa4e00a2b80 0x7fa4e00a5040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 --2- 172.21.15.19:0/1694507707 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa4f8100e20 0x7fa4f8199380 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 >> 172.21.15.19:0/1694507707 conn(0x7fa4f80fc7d0 msgr2=0x7fa4f80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 shutdown_connections 2024-09-12T03:16:51.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:51.023+0000 7fa4fe9a8640 1 -- 172.21.15.19:0/1694507707 wait complete. 2024-09-12T03:16:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:51 smithi135 ceph-mon[35474]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:51 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1694507707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:51.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:51 smithi121 ceph-mon[34963]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:51.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:51 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1694507707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:51 smithi019 ceph-mon[30953]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:51 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1694507707' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:52.091 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:16:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: starting mds.g at 2024-09-12T03:16:52.414 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:16:52 smithi019 bash[71150]: 32c08a4540d59e56536ba4218c83ca78ab664ecb00173b8a1b71b59b6c12ee69 2024-09-12T03:16:52.414 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:16:52 smithi019 systemd[1]: Started Ceph mds.g for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:53.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:53.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6838/2689902455,v1:172.21.15.19:6839/2689902455] up:boot 2024-09-12T03:16:53.161 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: fsmap 10 up:standby 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:53.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:53 smithi019 ceph-mon[30953]: Deploying daemon mds.a on smithi019 2024-09-12T03:16:53.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6838/2689902455,v1:172.21.15.19:6839/2689902455] up:boot 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: fsmap 10 up:standby 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:53.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:53 smithi121 ceph-mon[34963]: Deploying daemon mds.a on smithi019 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6838/2689902455,v1:172.21.15.19:6839/2689902455] up:boot 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: fsmap 10 up:standby 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:53.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:53 smithi135 ceph-mon[35474]: Deploying daemon mds.a on smithi019 2024-09-12T03:16:53.840 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:54.115 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:54.164 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:54 smithi019 systemd[1]: Starting Ceph mds.a for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 -- 172.21.15.19:0/797685465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 msgr2=0x7f980410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 --2- 172.21.15.19:0/797685465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980410ec40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f97fc009cd0 tx=0x7f97fc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 -- 172.21.15.19:0/797685465 shutdown_connections 2024-09-12T03:16:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 --2- 172.21.15.19:0/797685465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980410ec40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 --2- 172.21.15.19:0/797685465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 0x7f9804107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 --2- 172.21.15.19:0/797685465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 -- 172.21.15.19:0/797685465 >> 172.21.15.19:0/797685465 conn(0x7f98040fc7d0 msgr2=0x7f98040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 -- 172.21.15.19:0/797685465 shutdown_connections 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 -- 172.21.15.19:0/797685465 wait complete. 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.604+0000 7f980968e640 1 Processor -- start 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 -- start start 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 0x7f9804198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112100 con 0x7f98041082a0 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9804111f80 con 0x7f98041036e0 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f980968e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9804112280 con 0x7f9804102d30 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49760/0 (socket says 172.21.15.19:49760) 2024-09-12T03:16:54.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 -- 172.21.15.19:0/3130043463 learned_addr learned my addr 172.21.15.19:0/3130043463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98027fc640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 0x7f9804198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f9802ffd640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 -- 172.21.15.19:0/3130043463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 msgr2=0x7f9804198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 -- 172.21.15.19:0/3130043463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 msgr2=0x7f9804198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 0x7f9804198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f98037fe640 1 -- 172.21.15.19:0/3130043463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980419d500 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.605+0000 7f9802ffd640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f98037fe640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc004450 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97fc003e10 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980419d790 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97fc046480 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.606+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98041a50d0 con 0x7f98041082a0 2024-09-12T03:16:54.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.607+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9804103110 con 0x7f98041082a0 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.608+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f97fc0045f0 con 0x7f98041082a0 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.608+0000 7f97fbfff640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 0x7f97f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.609+0000 7f9802ffd640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 0x7f97f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.609+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f97fc02e030 con 0x7f98041082a0 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.609+0000 7f9802ffd640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 0x7f97f00a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009930 tx=0x7f97f4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:54.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.613+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97fc04b060 con 0x7f98041082a0 2024-09-12T03:16:54.789 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:54 smithi019 podman[71541]: 2024-09-12 03:16:54.662286842 +0000 UTC m=+0.250308561 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:54.789 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:54 smithi019 podman[71541]: 2024-09-12 03:16:54.790004415 +0000 UTC m=+0.378026122 container create e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:16:54.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.826+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f980410a2b0 con 0x7f98041082a0 2024-09-12T03:16:54.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.827+0000 7f97fbfff640 1 -- 172.21.15.19:0/3130043463 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8878 (secure 0 0 0) 0x7f97fc0bd950 con 0x7f98041082a0 2024-09-12T03:16:54.828 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:54.828 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":11,"btime":"2024-09-12T03:16:52:040363+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:16:54.829 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 11 2024-09-12T03:16:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 msgr2=0x7f97f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 0x7f97f00a5040 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009930 tx=0x7f97f4008040 comp rx=0 tx=0).stop 2024-09-12T03:16:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 msgr2=0x7f980419ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97fc006010 tx=0x7f97fc003c20 comp rx=0 tx=0).stop 2024-09-12T03:16:54.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 shutdown_connections 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97f00a2b80 0x7f97f00a5040 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98041082a0 0x7f980419ce20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98041036e0 0x7f9804198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 --2- 172.21.15.19:0/3130043463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9804102d30 0x7f9804198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 >> 172.21.15.19:0/3130043463 conn(0x7f98040fc7d0 msgr2=0x7f9804106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 shutdown_connections 2024-09-12T03:16:54.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:54.829+0000 7f980968e640 1 -- 172.21.15.19:0/3130043463 wait complete. 2024-09-12T03:16:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:55 smithi121 ceph-mon[34963]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:55 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3130043463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:55 smithi019 ceph-mon[30953]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:55 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3130043463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:55 smithi135 ceph-mon[35474]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:55 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3130043463' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 podman[71541]: 2024-09-12 03:16:55.889840331 +0000 UTC m=+1.477862039 container init e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 podman[71541]: 2024-09-12 03:16:55.893335481 +0000 UTC m=+1.481357191 container start e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 ceph-mds[71596]: -- 172.21.15.19:0/2418386779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d983a8f860 con 0x55d983a6d000 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: starting mds.a at 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 bash[71541]: e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 2024-09-12T03:16:56.165 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:16:55 smithi019 systemd[1]: Started Ceph mds.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6840/9500015,v1:172.21.15.19:6841/9500015] up:boot 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: fsmap 11 up:standby 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:57.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:57.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:57 smithi121 ceph-mon[34963]: Deploying daemon mds.b on smithi121 2024-09-12T03:16:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6840/9500015,v1:172.21.15.19:6841/9500015] up:boot 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: fsmap 11 up:standby 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:57.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:57 smithi019 ceph-mon[30953]: Deploying daemon mds.b on smithi121 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6840/9500015,v1:172.21.15.19:6841/9500015] up:boot 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: fsmap 11 up:standby 2024-09-12T03:16:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:16:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T03:16:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T03:16:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:16:57.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:57 smithi135 ceph-mon[35474]: Deploying daemon mds.b on smithi121 2024-09-12T03:16:57.612 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:16:57.886 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:16:58.149 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 systemd[1]: Starting Ceph mds.b for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:16:58.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 -- 172.21.15.19:0/1717669975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc410c170 msgr2=0x7f5fc410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 --2- 172.21.15.19:0/1717669975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc410c170 0x7f5fc410e610 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009ca0 tx=0x7f5fbc02a950 comp rx=0 tx=0).stop 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 -- 172.21.15.19:0/1717669975 shutdown_connections 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 --2- 172.21.15.19:0/1717669975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc410c170 0x7f5fc410e610 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 --2- 172.21.15.19:0/1717669975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4103660 0x7f5fc410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 --2- 172.21.15.19:0/1717669975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc4102d40 0x7f5fc4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 -- 172.21.15.19:0/1717669975 >> 172.21.15.19:0/1717669975 conn(0x7f5fc40fc7a0 msgr2=0x7f5fc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 -- 172.21.15.19:0/1717669975 shutdown_connections 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.350+0000 7f5fca07a640 1 -- 172.21.15.19:0/1717669975 wait complete. 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 Processor -- start 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 -- start start 2024-09-12T03:16:58.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4102d40 0x7f5fc4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc410c170 0x7f5fc419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4111f70 con 0x7f5fc4103660 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc4111df0 con 0x7f5fc4102d40 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fca07a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41120f0 con 0x7f5fc410c170 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fc2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49808/0 (socket says 172.21.15.19:49808) 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fc2ffd640 1 -- 172.21.15.19:0/2853053253 learned_addr learned my addr 172.21.15.19:0/2853053253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fc37fe640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4102d40 0x7f5fc4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.351+0000 7f5fc3fff640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc410c170 0x7f5fc419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 -- 172.21.15.19:0/2853053253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc410c170 msgr2=0x7f5fc419cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc410c170 0x7f5fc419cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 -- 172.21.15.19:0/2853053253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4102d40 msgr2=0x7f5fc4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4102d40 0x7f5fc4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 -- 172.21.15.19:0/2853053253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc419d3c0 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc2ffd640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c940 tx=0x7f5fb800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004220 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fb80043c0 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc419d6b0 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8004a00 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.352+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41a4f90 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.353+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc4103120 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.354+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5fb80079b0 con 0x7f5fc4103660 2024-09-12T03:16:58.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.355+0000 7f5fc0ff9640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 0x7f5fac0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:16:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.355+0000 7f5fc37fe640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 0x7f5fac0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:16:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.355+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f5fb80ce9a0 con 0x7f5fc4103660 2024-09-12T03:16:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.355+0000 7f5fc37fe640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 0x7f5fac0a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005fd0 tx=0x7f5fb400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:16:58.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.359+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5fb8014030 con 0x7f5fc4103660 2024-09-12T03:16:58.569 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 podman[65445]: 2024-09-12 03:16:58.199426423 +0000 UTC m=+0.015735352 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:16:58.569 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 podman[65445]: 2024-09-12 03:16:58.320241895 +0000 UTC m=+0.136550823 container create 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:16:58.569 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 podman[65445]: 2024-09-12 03:16:58.569169618 +0000 UTC m=+0.385478547 container init 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-12T03:16:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.569+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fc41a52d0 con 0x7f5fc4103660 2024-09-12T03:16:58.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.570+0000 7f5fc0ff9640 1 -- 172.21.15.19:0/2853053253 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7f5fb809b750 con 0x7f5fc4103660 2024-09-12T03:16:58.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:16:58.571 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":12,"btime":"2024-09-12T03:16:56:049361+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:16:58.571 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 12 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 msgr2=0x7f5fac0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 0x7f5fac0a50c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4005fd0 tx=0x7f5fb400b390 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 msgr2=0x7f5fc4198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800c940 tx=0x7f5fb800ce00 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 shutdown_connections 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5fc410c170 0x7f5fc419cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5fac0a2c00 0x7f5fac0a50c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5fc4103660 0x7f5fc4198950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 --2- 172.21.15.19:0/2853053253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc4102d40 0x7f5fc4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:16:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 >> 172.21.15.19:0/2853053253 conn(0x7f5fc40fc7a0 msgr2=0x7f5fc40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:16:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 shutdown_connections 2024-09-12T03:16:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:16:58.572+0000 7f5fca07a640 1 -- 172.21.15.19:0/2853053253 wait complete. 2024-09-12T03:16:58.899 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 podman[65445]: 2024-09-12 03:16:58.572463744 +0000 UTC m=+0.388772673 container start 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:16:58.900 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 ceph-mds[65464]: -- 172.21.15.121:0/3037292292 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a3a9f1860 con 0x564a3a9ce800 2024-09-12T03:16:58.900 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: starting mds.b at 2024-09-12T03:16:58.900 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 bash[65445]: 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 2024-09-12T03:16:58.900 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:16:58 smithi121 systemd[1]: Started Ceph mds.b for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:16:59.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2853053253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:16:59 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2853053253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:16:59 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2853053253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:16:59.490 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:16:59 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:00.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:00 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:boot 2024-09-12T03:17:00.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:00 smithi121 ceph-mon[34963]: fsmap 12 up:standby 2024-09-12T03:17:00.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:00 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:00 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:boot 2024-09-12T03:17:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:00 smithi019 ceph-mon[30953]: fsmap 12 up:standby 2024-09-12T03:17:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:00 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:00 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:boot 2024-09-12T03:17:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:00 smithi135 ceph-mon[35474]: fsmap 12 up:standby 2024-09-12T03:17:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:00 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:17:01.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:01.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:01 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.407 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.408 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.408 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.408 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.408 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:01 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:01.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:01 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:17:01.889 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:02.286 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:02 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 -- 172.21.15.19:0/2511315465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c106a80 msgr2=0x7f9c1c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/2511315465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c106a80 0x7f9c1c106e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009930 tx=0x7f9c0c02a420 comp rx=0 tx=0).stop 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 -- 172.21.15.19:0/2511315465 shutdown_connections 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/2511315465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c101420 0x7f9c1c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/2511315465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 0x7f9c1c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/2511315465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c106a80 0x7f9c1c106e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 -- 172.21.15.19:0/2511315465 >> 172.21.15.19:0/2511315465 conn(0x7f9c1c0fc7a0 msgr2=0x7f9c1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 -- 172.21.15.19:0/2511315465 shutdown_connections 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.369+0000 7f9c22b0f640 1 -- 172.21.15.19:0/2511315465 wait complete. 2024-09-12T03:17:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 Processor -- start 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 -- start start 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 0x7f9c1c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 0x7f9c1c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c110e80 con 0x7f9c1c101420 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c110d00 con 0x7f9c1c106a80 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c22b0f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c1c111000 con 0x7f9c1c100a80 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59220/0 (socket says 172.21.15.19:59220) 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 -- 172.21.15.19:0/1665282160 learned_addr learned my addr 172.21.15.19:0/1665282160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c210ac640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 0x7f9c1c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c208ab640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 0x7f9c1c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 -- 172.21.15.19:0/1665282160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 msgr2=0x7f9c1c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 0x7f9c1c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 -- 172.21.15.19:0/1665282160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 msgr2=0x7f9c1c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.370+0000 7f9c1bfff640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 0x7f9c1c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c1bfff640 1 -- 172.21.15.19:0/1665282160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c1c19d420 con 0x7f9c1c101420 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c210ac640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 0x7f9c1c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c1bfff640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d770 tx=0x7f9c0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04017070 con 0x7f9c1c101420 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c04005500 con 0x7f9c1c101420 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c1c19d710 con 0x7f9c1c101420 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04004e00 con 0x7f9c1c101420 2024-09-12T03:17:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.371+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c1c1a4ff0 con 0x7f9c1c101420 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.372+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1c100ee0 con 0x7f9c1c101420 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.373+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9c04004fa0 con 0x7f9c1c101420 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.374+0000 7f9c19ffb640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 0x7f9c080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.374+0000 7f9c208ab640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 0x7f9c080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.374+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f9c040ce6c0 con 0x7f9c1c101420 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.374+0000 7f9c208ab640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 0x7f9c080a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:02.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.378+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c0409ba20 con 0x7f9c1c101420 2024-09-12T03:17:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:02 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:02.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.625+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c1c10a060 con 0x7f9c1c101420 2024-09-12T03:17:02.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.626+0000 7f9c19ffb640 1 -- 172.21.15.19:0/1665282160 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7f9c0409b320 con 0x7f9c1c101420 2024-09-12T03:17:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:02.627 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":13,"btime":"2024-09-12T03:16:59:047650+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14577,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14589,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24457,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24463,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24467,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T03:17:02.627 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 13 2024-09-12T03:17:02.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 msgr2=0x7f9c080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 0x7f9c080a50c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c005a20 comp rx=0 tx=0).stop 2024-09-12T03:17:02.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 msgr2=0x7f9c1c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400d770 tx=0x7f9c0400dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 shutdown_connections 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c1c106a80 0x7f9c1c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c080a2c00 0x7f9c080a50c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c1c101420 0x7f9c1c1989b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 --2- 172.21.15.19:0/1665282160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c1c100a80 0x7f9c1c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 >> 172.21.15.19:0/1665282160 conn(0x7f9c1c0fc7a0 msgr2=0x7f9c1c06aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 shutdown_connections 2024-09-12T03:17:02.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:02.629+0000 7f9c22b0f640 1 -- 172.21.15.19:0/1665282160 wait complete. 2024-09-12T03:17:02.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:02 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1665282160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:03.556 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1665282160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:03.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1665282160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:04.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 -- 172.21.15.19:0/396203120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0100e20 msgr2=0x7f98a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 --2- 172.21.15.19:0/396203120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0100e20 0x7f98a0101200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9890009900 tx=0x7f989002a480 comp rx=0 tx=0).stop 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 -- 172.21.15.19:0/396203120 shutdown_connections 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 --2- 172.21.15.19:0/396203120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 --2- 172.21.15.19:0/396203120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0102570 0x7f98a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 --2- 172.21.15.19:0/396203120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0100e20 0x7f98a0101200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 -- 172.21.15.19:0/396203120 >> 172.21.15.19:0/396203120 conn(0x7f98a00fc7d0 msgr2=0x7f98a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 -- 172.21.15.19:0/396203120 shutdown_connections 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.060+0000 7f98a4c78640 1 -- 172.21.15.19:0/396203120 wait complete. 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 Processor -- start 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 -- start start 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 0x7f98a0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0111020 con 0x7f98a0102570 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98a0110ea0 con 0x7f98a0102ef0 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f98a4c78640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f98a01111a0 con 0x7f98a0100e20 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59232/0 (socket says 172.21.15.19:59232) 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 -- 172.21.15.19:0/2833407589 learned_addr learned my addr 172.21.15.19:0/2833407589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f989e59c640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 0x7f98a0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f989ed9d640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 -- 172.21.15.19:0/2833407589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 msgr2=0x7f98a0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 0x7f98a0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 -- 172.21.15.19:0/2833407589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 msgr2=0x7f98a019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f9897fff640 1 -- 172.21.15.19:0/2833407589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a019d620 con 0x7f98a0102570 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f989e59c640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 0x7f98a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.061+0000 7f989ed9d640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f9897fff640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f988800b4a0 tx=0x7f988800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888014070 con 0x7f98a0102570 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9888010af0 con 0x7f98a0102570 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9888017d10 con 0x7f98a0102570 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a019d910 con 0x7f98a0102570 2024-09-12T03:17:04.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.062+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a01a51f0 con 0x7f98a0102570 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.063+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a0101280 con 0x7f98a0102570 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.064+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9888017070 con 0x7f98a0102570 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.065+0000 7f98977fe640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 0x7f988c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.065+0000 7f989e59c640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 0x7f988c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.065+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f98880cdfb0 con 0x7f98a0102570 2024-09-12T03:17:04.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.065+0000 7f989e59c640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 0x7f988c0a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9890000c00 tx=0x7f98900041e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:04.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.069+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f988809b3b0 con 0x7f98a0102570 2024-09-12T03:17:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.245+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f98a010a0a0 con 0x7f98a0102570 2024-09-12T03:17:04.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.246+0000 7f98977fe640 1 -- 172.21.15.19:0/2833407589 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) ==== 84+0+65 (secure 0 0 0) 0x7f988809acb0 con 0x7f98a0102570 2024-09-12T03:17:04.246 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:04.246 INFO:teuthology.orchestra.run.smithi019.stdout:[ 2024-09-12T03:17:04.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:17:04.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolnum": 1, 2024-09-12T03:17:04.247 INFO:teuthology.orchestra.run.smithi019.stdout: "poolname": ".mgr" 2024-09-12T03:17:04.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:17:04.247 INFO:teuthology.orchestra.run.smithi019.stdout:] 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 msgr2=0x7f988c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 0x7f988c0a50c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9890000c00 tx=0x7f98900041e0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 msgr2=0x7f98a0198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f988800b4a0 tx=0x7f988800b960 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 shutdown_connections 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98a0102ef0 0x7f98a019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f988c0a2c00 0x7f988c0a50c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f98a0102570 0x7f98a0198bb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 --2- 172.21.15.19:0/2833407589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f98a0100e20 0x7f98a0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 >> 172.21.15.19:0/2833407589 conn(0x7f98a00fc7d0 msgr2=0x7f98a006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 shutdown_connections 2024-09-12T03:17:04.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:04.248+0000 7f98a4c78640 1 -- 172.21.15.19:0/2833407589 wait complete. 2024-09-12T03:17:04.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:04.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:04.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:04.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.599 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:04 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2833407589' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:04.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:04.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:04 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2833407589' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:17:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:17:04.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:17:04.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:04 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2833407589' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T03:17:04.964 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T03:17:04.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T03:17:05.226 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:05.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/4138452964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 msgr2=0x7fe3f010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/4138452964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f010ec40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8009930 tx=0x7fe3e802a420 comp rx=0 tx=0).stop 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/4138452964 shutdown_connections 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/4138452964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f010ec40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/4138452964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f01036e0 0x7fe3f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/4138452964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f0102d30 0x7fe3f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/4138452964 >> 172.21.15.19:0/4138452964 conn(0x7fe3f00fc7d0 msgr2=0x7fe3f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/4138452964 shutdown_connections 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.682+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/4138452964 wait complete. 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 Processor -- start 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 -- start start 2024-09-12T03:17:05.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f0102d30 0x7fe3f01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f01036e0 0x7fe3f0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01120d0 con 0x7fe3f01082a0 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0111f50 con 0x7fe3f01036e0 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3f4b9d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0112250 con 0x7fe3f0102d30 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59244/0 (socket says 172.21.15.19:59244) 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 -- 172.21.15.19:0/2544652977 learned_addr learned my addr 172.21.15.19:0/2544652977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3ee59c640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f0102d30 0x7fe3f01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3edd9b640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f01036e0 0x7fe3f0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 -- 172.21.15.19:0/2544652977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f0102d30 msgr2=0x7fe3f01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f0102d30 0x7fe3f01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 -- 172.21.15.19:0/2544652977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f01036e0 msgr2=0x7fe3f0198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f01036e0 0x7fe3f0198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.683+0000 7fe3eed9d640 1 -- 172.21.15.19:0/2544652977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f019d470 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3eed9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe3e802aa10 tx=0x7fe3e8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e803f070 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe3e8031c90 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8035030 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f019d700 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.684+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f01a5040 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.685+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f006a850 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.686+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe3e8042070 con 0x7fe3f01082a0 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.687+0000 7fe3e77fe640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 0x7fe3dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:05.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.687+0000 7fe3ee59c640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 0x7fe3dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.687+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7fe3e80efa30 con 0x7fe3f01082a0 2024-09-12T03:17:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.688+0000 7fe3ee59c640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 0x7fe3dc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0007350 tx=0x7fe3e00072e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:05.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.690+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3e80bcd90 con 0x7fe3f01082a0 2024-09-12T03:17:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:05.867+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe3f01a5210 con 0x7fe3f01082a0 2024-09-12T03:17:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.811+0000 7fe3e77fe640 1 -- 172.21.15.19:0/2544652977 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) ==== 138+0+0 (secure 0 0 0) 0x7fe3e80bc690 con 0x7fe3f01082a0 2024-09-12T03:17:06.812 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' created 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 msgr2=0x7fe3dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 0x7fe3dc0a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0007350 tx=0x7fe3e00072e0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 msgr2=0x7fe3f019cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe3e802aa10 tx=0x7fe3e8003b90 comp rx=0 tx=0).stop 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 shutdown_connections 2024-09-12T03:17:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe3dc0a2b80 0x7fe3dc0a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe3f01082a0 0x7fe3f019cd90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3f01036e0 0x7fe3f0198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 --2- 172.21.15.19:0/2544652977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe3f0102d30 0x7fe3f01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 >> 172.21.15.19:0/2544652977 conn(0x7fe3f00fc7d0 msgr2=0x7fe3f00fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 shutdown_connections 2024-09-12T03:17:06.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:06.816+0000 7fe3f4b9d640 1 -- 172.21.15.19:0/2544652977 wait complete. 2024-09-12T03:17:07.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:06 smithi121 ceph-mon[34963]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:07.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:06 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:17:07.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:06 smithi019 ceph-mon[30953]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:07.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:06 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:17:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:06 smithi135 ceph-mon[35474]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:06 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T03:17:07.519 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T03:17:07.784 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:08.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:07 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:17:08.070 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:07 smithi019 ceph-mon[30953]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:17:08.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:07 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:17:08.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:07 smithi121 ceph-mon[34963]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:17:08.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- 172.21.15.19:0/169093594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20102790 msgr2=0x7fab201067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 --2- 172.21.15.19:0/169093594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20102790 0x7fab201067e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fab14009900 tx=0x7fab1402a410 comp rx=0 tx=0).stop 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- 172.21.15.19:0/169093594 shutdown_connections 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 --2- 172.21.15.19:0/169093594 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 --2- 172.21.15.19:0/169093594 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20102790 0x7fab201067e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 --2- 172.21.15.19:0/169093594 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20101de0 0x7fab201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- 172.21.15.19:0/169093594 >> 172.21.15.19:0/169093594 conn(0x7fab200fd8c0 msgr2=0x7fab200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- 172.21.15.19:0/169093594 shutdown_connections 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- 172.21.15.19:0/169093594 wait complete. 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 Processor -- start 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.277+0000 7fab2539f640 1 -- start start 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 0x7fab201989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fab20111fb0 con 0x7fab20101de0 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fab20111e30 con 0x7fab20106f10 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fab20112130 con 0x7fab20102790 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59260/0 (socket says 172.21.15.19:59260) 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 -- 172.21.15.19:0/4288549826 learned_addr learned my addr 172.21.15.19:0/4288549826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1e7fc640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 0x7fab201989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1f7fe640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 -- 172.21.15.19:0/4288549826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 msgr2=0x7fab201989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 0x7fab201989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 -- 172.21.15.19:0/4288549826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 msgr2=0x7fab2019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 -- 172.21.15.19:0/4288549826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2019d440 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1e7fc640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 0x7fab201989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1f7fe640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab1effd640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d770 tx=0x7fab1000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10017070 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab10005500 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.278+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2019d730 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.279+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10004e00 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.279+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab201a5010 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.279+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab201021c0 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.281+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fab10004fa0 con 0x7fab20101de0 2024-09-12T03:17:08.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.281+0000 7fab07fff640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 0x7fab080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.281+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7497+0+0 (secure 0 0 0) 0x7fab100cec70 con 0x7fab20101de0 2024-09-12T03:17:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.281+0000 7fab1e7fc640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 0x7fab080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.282+0000 7fab1e7fc640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 0x7fab080a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab14000c00 tx=0x7fab14032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:08.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.285+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fab1009bf00 con 0x7fab20101de0 2024-09-12T03:17:08.293 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:07 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2544652977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T03:17:08.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:07 smithi135 ceph-mon[35474]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T03:17:08.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.460+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fab2010a0f0 con 0x7fab20101de0 2024-09-12T03:17:08.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.899+0000 7fab07fff640 1 -- 172.21.15.19:0/4288549826 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7fab1009b800 con 0x7fab20101de0 2024-09-12T03:17:08.900 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' created 2024-09-12T03:17:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 msgr2=0x7fab080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 0x7fab080a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fab14000c00 tx=0x7fab14032c70 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 msgr2=0x7fab20198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fab1000d770 tx=0x7fab1000dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 shutdown_connections 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fab20106f10 0x7fab2019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fab20102790 0x7fab201989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fab080a2b80 0x7fab080a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 --2- 172.21.15.19:0/4288549826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fab20101de0 0x7fab20198490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 >> 172.21.15.19:0/4288549826 conn(0x7fab200fd8c0 msgr2=0x7fab200fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 shutdown_connections 2024-09-12T03:17:08.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:08.902+0000 7fab2539f640 1 -- 172.21.15.19:0/4288549826 wait complete. 2024-09-12T03:17:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:08 smithi121 ceph-mon[34963]: pgmap v274: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:08 smithi121 ceph-mon[34963]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:17:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:08 smithi121 ceph-mon[34963]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:17:09.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:08 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:17:09.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:08 smithi019 ceph-mon[30953]: pgmap v274: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:09.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:08 smithi019 ceph-mon[30953]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:17:09.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:08 smithi019 ceph-mon[30953]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:17:09.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:08 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:17:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:08 smithi135 ceph-mon[35474]: pgmap v274: 33 pgs: 7 creating+peering, 1 active+clean, 25 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:08 smithi135 ceph-mon[35474]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T03:17:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:08 smithi135 ceph-mon[35474]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T03:17:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:08 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T03:17:09.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T03:17:09.821 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:10.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:09 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:17:10.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:09 smithi019 ceph-mon[30953]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:17:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:09 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:17:10.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:09 smithi135 ceph-mon[35474]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 -- 172.21.15.19:0/918306916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102570 msgr2=0x7fef5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 --2- 172.21.15.19:0/918306916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102570 0x7fef5c1029b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fef4c009900 tx=0x7fef4c02a480 comp rx=0 tx=0).stop 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 -- 172.21.15.19:0/918306916 shutdown_connections 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 --2- 172.21.15.19:0/918306916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c102ef0 0x7fef5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 --2- 172.21.15.19:0/918306916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102570 0x7fef5c1029b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 --2- 172.21.15.19:0/918306916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c100e20 0x7fef5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 -- 172.21.15.19:0/918306916 >> 172.21.15.19:0/918306916 conn(0x7fef5c0fc7d0 msgr2=0x7fef5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 -- 172.21.15.19:0/918306916 shutdown_connections 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.300+0000 7fef63eb7640 1 -- 172.21.15.19:0/918306916 wait complete. 2024-09-12T03:17:10.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 Processor -- start 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 -- start start 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c100e20 0x7fef5c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c102570 0x7fef5c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c110f80 con 0x7fef5c102ef0 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c110e00 con 0x7fef5c102570 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef63eb7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c111100 con 0x7fef5c100e20 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59268/0 (socket says 172.21.15.19:59268) 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 -- 172.21.15.19:0/3970418219 learned_addr learned my addr 172.21.15.19:0/3970418219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef61452640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c102570 0x7fef5c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef61c53640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c100e20 0x7fef5c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 -- 172.21.15.19:0/3970418219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c100e20 msgr2=0x7fef5c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c100e20 0x7fef5c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 -- 172.21.15.19:0/3970418219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c102570 msgr2=0x7fef5c198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c102570 0x7fef5c198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.301+0000 7fef62454640 1 -- 172.21.15.19:0/3970418219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c19d4a0 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef62454640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef54009fd0 tx=0x7fef5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef54017070 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c19d790 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef54003e50 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef540106c0 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.302+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1a5070 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.303+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c1021b0 con 0x7fef5c102ef0 2024-09-12T03:17:10.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.304+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef54003970 con 0x7fef5c102ef0 2024-09-12T03:17:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.305+0000 7fef52ffd640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 0x7fef480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.305+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7fef540cec40 con 0x7fef5c102ef0 2024-09-12T03:17:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.305+0000 7fef61c53640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 0x7fef480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.306+0000 7fef61c53640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 0x7fef480a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef58005d00 tx=0x7fef58005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:10.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.313+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fef5409bcb0 con 0x7fef5c102ef0 2024-09-12T03:17:10.329 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:09 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4288549826' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T03:17:10.329 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:09 smithi121 ceph-mon[34963]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T03:17:10.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.525+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fef5c1a52a0 con 0x7fef5c102ef0 2024-09-12T03:17:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.906+0000 7fef52ffd640 1 -- 172.21.15.19:0/3970418219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fef5409b5b0 con 0x7fef5c102ef0 2024-09-12T03:17:10.907 INFO:teuthology.orchestra.run.smithi019.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 msgr2=0x7fef480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 0x7fef480a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef58005d00 tx=0x7fef58005c50 comp rx=0 tx=0).stop 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 msgr2=0x7fef5c19cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef54009fd0 tx=0x7fef5400b260 comp rx=0 tx=0).stop 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 shutdown_connections 2024-09-12T03:17:10.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fef480a2b80 0x7fef480a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef5c102ef0 0x7fef5c19cdc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef5c102570 0x7fef5c198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 --2- 172.21.15.19:0/3970418219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef5c100e20 0x7fef5c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 >> 172.21.15.19:0/3970418219 conn(0x7fef5c0fc7d0 msgr2=0x7fef5c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 shutdown_connections 2024-09-12T03:17:10.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:10.909+0000 7fef63eb7640 1 -- 172.21.15.19:0/3970418219 wait complete. 2024-09-12T03:17:11.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:10 smithi019 ceph-mon[30953]: pgmap v277: 97 pgs: 11 creating+peering, 15 active+clean, 71 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:11.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:10 smithi019 ceph-mon[30953]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:17:11.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:10 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:17:11.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:10 smithi019 ceph-mon[30953]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T03:17:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:10 smithi135 ceph-mon[35474]: pgmap v277: 97 pgs: 11 creating+peering, 15 active+clean, 71 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:10 smithi135 ceph-mon[35474]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:17:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:10 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:17:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:10 smithi135 ceph-mon[35474]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T03:17:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:10 smithi121 ceph-mon[34963]: pgmap v277: 97 pgs: 11 creating+peering, 15 active+clean, 71 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:11.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:10 smithi121 ceph-mon[34963]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T03:17:11.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:10 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T03:17:11.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:10 smithi121 ceph-mon[34963]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T03:17:11.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:11.872 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:12.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:11 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:17:12.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:11 smithi019 ceph-mon[30953]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:17:12.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:11 smithi019 ceph-mon[30953]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T03:17:12.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:11 smithi019 ceph-mon[30953]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T03:17:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:11 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:17:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:11 smithi135 ceph-mon[35474]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:17:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:11 smithi135 ceph-mon[35474]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T03:17:12.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:11 smithi135 ceph-mon[35474]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T03:17:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 -- 172.21.15.19:0/3141676491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618101de0 msgr2=0x7fd6181021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 --2- 172.21.15.19:0/3141676491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618101de0 0x7fd6181021c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd614009900 tx=0x7fd61402a410 comp rx=0 tx=0).stop 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 -- 172.21.15.19:0/3141676491 shutdown_connections 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 --2- 172.21.15.19:0/3141676491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618106f10 0x7fd61810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 --2- 172.21.15.19:0/3141676491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618102790 0x7fd6181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 --2- 172.21.15.19:0/3141676491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618101de0 0x7fd6181021c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 -- 172.21.15.19:0/3141676491 >> 172.21.15.19:0/3141676491 conn(0x7fd6180fd8c0 msgr2=0x7fd6180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 -- 172.21.15.19:0/3141676491 shutdown_connections 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.355+0000 7fd6200ca640 1 -- 172.21.15.19:0/3141676491 wait complete. 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 Processor -- start 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 -- start start 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618101de0 0x7fd6180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 0x7fd618074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd618111f90 con 0x7fd618102790 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd618111e10 con 0x7fd618106f10 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd6200ca640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd618112110 con 0x7fd618101de0 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd61d665640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd61d665640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54108/0 (socket says 172.21.15.19:54108) 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd61d665640 1 -- 172.21.15.19:0/1510552385 learned_addr learned my addr 172.21.15.19:0/1510552385 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.356+0000 7fd61de66640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618101de0 0x7fd6180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61e667640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 0x7fd618074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 -- 172.21.15.19:0/1510552385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618101de0 msgr2=0x7fd6180744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618101de0 0x7fd6180744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 -- 172.21.15.19:0/1510552385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 msgr2=0x7fd618074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 0x7fd618074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 -- 172.21.15.19:0/1510552385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6181aae80 con 0x7fd618102790 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61e667640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 0x7fd618074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd61d665640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd60800b4a0 tx=0x7fd60800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608014070 con 0x7fd618102790 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6181ab0b0 con 0x7fd618102790 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd608010af0 con 0x7fd618102790 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608017d10 con 0x7fd618102790 2024-09-12T03:17:12.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.357+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6181ab5f0 con 0x7fd618102790 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.358+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6181032c0 con 0x7fd618102790 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.359+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd608017070 con 0x7fd618102790 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.360+0000 7fd60effd640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.360+0000 7fd61de66640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.360+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd6080ce3b0 con 0x7fd618102790 2024-09-12T03:17:12.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.361+0000 7fd61de66640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 0x7fd6040a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd6140058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:12.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.364+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd60809b3c0 con 0x7fd618102790 2024-09-12T03:17:12.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:11 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3970418219' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T03:17:12.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:11 smithi121 ceph-mon[34963]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T03:17:12.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:11 smithi121 ceph-mon[34963]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T03:17:12.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:11 smithi121 ceph-mon[34963]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T03:17:12.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.539+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd61810a0f0 con 0x7fd618102790 2024-09-12T03:17:12.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.541+0000 7fd60effd640 1 -- 172.21.15.19:0/1510552385 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22303 (secure 0 0 0) 0x7fd60809acc0 con 0x7fd618102790 2024-09-12T03:17:12.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:12.541 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:17:10.527091+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 msgr2=0x7fd6040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 0x7fd6040a5040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd614000c00 tx=0x7fd6140058d0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 msgr2=0x7fd618074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd60800b4a0 tx=0x7fd60800b960 comp rx=0 tx=0).stop 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 shutdown_connections 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd618106f10 0x7fd618074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd6040a2b80 0x7fd6040a5040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd618102790 0x7fd618074a30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 --2- 172.21.15.19:0/1510552385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd618101de0 0x7fd6180744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 >> 172.21.15.19:0/1510552385 conn(0x7fd6180fd8c0 msgr2=0x7fd6180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 shutdown_connections 2024-09-12T03:17:12.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:12.543+0000 7fd6200ca640 1 -- 172.21.15.19:0/1510552385 wait complete. 2024-09-12T03:17:13.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: pgmap v280: 97 pgs: 19 creating+peering, 51 active+clean, 27 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:13.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:17:13.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: Cluster is now healthy 2024-09-12T03:17:13.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:active 2024-09-12T03:17:13.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:12 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1510552385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:13.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: pgmap v280: 97 pgs: 19 creating+peering, 51 active+clean, 27 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: Cluster is now healthy 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:active 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:12 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1510552385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: pgmap v280: 97 pgs: 19 creating+peering, 51 active+clean, 27 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:17:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T03:17:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: Cluster is now healthy 2024-09-12T03:17:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] up:active 2024-09-12T03:17:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:13.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:12 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1510552385' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:13.506 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 -- 172.21.15.19:0/1823595073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4106a80 msgr2=0x7f26a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 --2- 172.21.15.19:0/1823595073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4106a80 0x7f26a4106e60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2694009930 tx=0x7f269402a420 comp rx=0 tx=0).stop 2024-09-12T03:17:14.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 -- 172.21.15.19:0/1823595073 shutdown_connections 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 --2- 172.21.15.19:0/1823595073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4101420 0x7f26a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 --2- 172.21.15.19:0/1823595073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 --2- 172.21.15.19:0/1823595073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4106a80 0x7f26a4106e60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 -- 172.21.15.19:0/1823595073 >> 172.21.15.19:0/1823595073 conn(0x7f26a40fc7a0 msgr2=0x7f26a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 -- 172.21.15.19:0/1823595073 shutdown_connections 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.047+0000 7f26a890a640 1 -- 172.21.15.19:0/1823595073 wait complete. 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 Processor -- start 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 -- start start 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 0x7f26a41a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4110e80 con 0x7f26a4101420 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4110d00 con 0x7f26a4106a80 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a890a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4111000 con 0x7f26a4100a80 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54118/0 (socket says 172.21.15.19:54118) 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a1d9b640 1 -- 172.21.15.19:0/4004135627 learned_addr learned my addr 172.21.15.19:0/4004135627 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a259c640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.048+0000 7f26a2d9d640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 0x7f26a41a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 -- 172.21.15.19:0/4004135627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 msgr2=0x7f26a4199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 -- 172.21.15.19:0/4004135627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 msgr2=0x7f26a41a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 0x7f26a41a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 -- 172.21.15.19:0/4004135627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a41a48d0 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a2d9d640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 0x7f26a41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a259c640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a1d9b640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f269800d770 tx=0x7f269800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698017070 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a41a4aa0 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2698005500 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2698004e00 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.049+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a41a4f60 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.050+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a4100ee0 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.051+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2698004fa0 con 0x7f26a4101420 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.052+0000 7f26937fe640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 0x7f268c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.052+0000 7f26a259c640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 0x7f268c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:14.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.052+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f26980cea20 con 0x7f26a4101420 2024-09-12T03:17:14.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.052+0000 7f26a259c640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 0x7f268c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2694000c00 tx=0x7f2694032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:14.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.056+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f269809bad0 con 0x7f26a4101420 2024-09-12T03:17:14.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.233+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26a4108ff0 con 0x7f26a4101420 2024-09-12T03:17:14.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.234+0000 7f26937fe640 1 -- 172.21.15.19:0/4004135627 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22303 (secure 0 0 0) 0x7f269809b3d0 con 0x7f26a4101420 2024-09-12T03:17:14.235 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:14.235 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:17:10.527091+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 msgr2=0x7f268c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 0x7f268c0a5040 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2694000c00 tx=0x7f2694032c70 comp rx=0 tx=0).stop 2024-09-12T03:17:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 msgr2=0x7f26a4199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f269800d770 tx=0x7f269800dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:14.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 shutdown_connections 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26a4106a80 0x7f26a41a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f268c0a2b80 0x7f268c0a5040 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26a4101420 0x7f26a4199850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 --2- 172.21.15.19:0/4004135627 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26a4100a80 0x7f26a4199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 >> 172.21.15.19:0/4004135627 conn(0x7f26a40fc7a0 msgr2=0x7f26a40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 shutdown_connections 2024-09-12T03:17:14.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:14.237+0000 7f26a890a640 1 -- 172.21.15.19:0/4004135627 wait complete. 2024-09-12T03:17:14.297 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:17:14 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:17:14.040+0000 7f7545979640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T03:17:14.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T03:17:15.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:14 smithi019 ceph-mon[30953]: pgmap v281: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 2 op/s 2024-09-12T03:17:15.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4004135627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:15.195 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:14 smithi135 ceph-mon[35474]: pgmap v281: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 2 op/s 2024-09-12T03:17:15.195 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4004135627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:15.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:14 smithi121 ceph-mon[34963]: pgmap v281: 97 pgs: 12 creating+peering, 85 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 376 B/s wr, 2 op/s 2024-09-12T03:17:15.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4004135627' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:15.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.667+0000 7f437ade5640 1 -- 172.21.15.19:0/3578786503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374102790 msgr2=0x7f43741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.667+0000 7f437ade5640 1 --2- 172.21.15.19:0/3578786503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374102790 0x7f43741067e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f436002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- 172.21.15.19:0/3578786503 shutdown_connections 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- 172.21.15.19:0/3578786503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374106f10 0x7f437410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- 172.21.15.19:0/3578786503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374102790 0x7f43741067e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- 172.21.15.19:0/3578786503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374101de0 0x7f43741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- 172.21.15.19:0/3578786503 >> 172.21.15.19:0/3578786503 conn(0x7f43740fd8c0 msgr2=0x7f43740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- 172.21.15.19:0/3578786503 shutdown_connections 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- 172.21.15.19:0/3578786503 wait complete. 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 Processor -- start 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- start start 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374101de0 0x7f43740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374102790 0x7f4374074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4374111f70 con 0x7f4374106f10 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4374111df0 con 0x7f4374101de0 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.668+0000 7f437ade5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43741120f0 con 0x7f4374102790 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54138/0 (socket says 172.21.15.19:54138) 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 -- 172.21.15.19:0/2861460576 learned_addr learned my addr 172.21.15.19:0/2861460576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4373fff640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374102790 0x7f4374074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4378b81640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374101de0 0x7f43740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 -- 172.21.15.19:0/2861460576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374102790 msgr2=0x7f4374074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374102790 0x7f4374074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 -- 172.21.15.19:0/2861460576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374101de0 msgr2=0x7f43740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374101de0 0x7f43740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 -- 172.21.15.19:0/2861460576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43741a6970 con 0x7f4374106f10 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4379382640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f436c00eaa0 tx=0x7f436c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00cdb0 con 0x7f4374106f10 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f436c005020 con 0x7f4374106f10 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741a6c60 con 0x7f4374106f10 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.669+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c0116b0 con 0x7f4374106f10 2024-09-12T03:17:15.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.670+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741a7120 con 0x7f4374106f10 2024-09-12T03:17:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.671+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43741021c0 con 0x7f4374106f10 2024-09-12T03:17:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.672+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f436c005560 con 0x7f4374106f10 2024-09-12T03:17:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.672+0000 7f4371ffb640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 0x7f43640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.672+0000 7f4378b81640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 0x7f43640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:15.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.672+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f436c0cf050 con 0x7f4374106f10 2024-09-12T03:17:15.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.673+0000 7f4378b81640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 0x7f43640a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4368009bd0 tx=0x7f4368005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:15.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.677+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f436c016030 con 0x7f4374106f10 2024-09-12T03:17:15.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.891+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f437410a180 con 0x7f4374106f10 2024-09-12T03:17:15.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.935+0000 7f4371ffb640 1 -- 172.21.15.19:0/2861460576 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f436c09bab0 con 0x7f4374106f10 2024-09-12T03:17:15.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 msgr2=0x7f43640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 0x7f43640a5040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4368009bd0 tx=0x7f4368005f90 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 msgr2=0x7f4374074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f436c00eaa0 tx=0x7f436c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 shutdown_connections 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43640a2b80 0x7f43640a5040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4374106f10 0x7f4374074f70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4374102790 0x7f4374074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 --2- 172.21.15.19:0/2861460576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4374101de0 0x7f43740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.937+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 >> 172.21.15.19:0/2861460576 conn(0x7f43740fd8c0 msgr2=0x7f43740fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.938+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 shutdown_connections 2024-09-12T03:17:15.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:15.938+0000 7f437ade5640 1 -- 172.21.15.19:0/2861460576 wait complete. 2024-09-12T03:17:16.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:17:16.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:17:16.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T03:17:16.619 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-12T03:17:16.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-12T03:17:16.878 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:17.188 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:16 smithi019 ceph-mon[30953]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:17.188 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:16 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:17:17.188 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:16 smithi019 ceph-mon[30953]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:16 smithi135 ceph-mon[35474]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:16 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:17:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:16 smithi135 ceph-mon[35474]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:17.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:16 smithi121 ceph-mon[34963]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 6 op/s 2024-09-12T03:17:17.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:16 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2861460576' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T03:17:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:16 smithi121 ceph-mon[34963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 -- 172.21.15.19:0/4157618583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 msgr2=0x7f182010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 --2- 172.21.15.19:0/4157618583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f182010bb50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1810009900 tx=0x7f181002a480 comp rx=0 tx=0).stop 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 -- 172.21.15.19:0/4157618583 shutdown_connections 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 --2- 172.21.15.19:0/4157618583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f182010c170 0x7f182010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 --2- 172.21.15.19:0/4157618583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f182010bb50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 --2- 172.21.15.19:0/4157618583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1820102d40 0x7f1820103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 -- 172.21.15.19:0/4157618583 >> 172.21.15.19:0/4157618583 conn(0x7f18200fc7a0 msgr2=0x7f18200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 -- 172.21.15.19:0/4157618583 shutdown_connections 2024-09-12T03:17:17.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.459+0000 7f1827b5b640 1 -- 172.21.15.19:0/4157618583 wait complete. 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 Processor -- start 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 -- start start 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1820102d40 0x7f18201009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 0x7f1820101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1820111fb0 con 0x7f1820103660 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1820111e30 con 0x7f1820102d40 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f1827b5b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1820112130 con 0x7f182010c170 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f18250f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f18250f6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54160/0 (socket says 172.21.15.19:54160) 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.460+0000 7f18250f6640 1 -- 172.21.15.19:0/1894268149 learned_addr learned my addr 172.21.15.19:0/1894268149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:17.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18258f7640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1820102d40 0x7f18201009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18260f8640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 0x7f1820101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 -- 172.21.15.19:0/1894268149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 msgr2=0x7f1820101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 0x7f1820101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 -- 172.21.15.19:0/1894268149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1820102d40 msgr2=0x7f18201009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1820102d40 0x7f18201009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 -- 172.21.15.19:0/1894268149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18201a68b0 con 0x7f1820103660 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18260f8640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 0x7f1820101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f18250f6640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1810005ab0 tx=0x7f18100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181003f030 con 0x7f1820103660 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1810003ce0 con 0x7f1820103660 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18201a6b40 con 0x7f1820103660 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810035070 con 0x7f1820103660 2024-09-12T03:17:17.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.461+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18201a7070 con 0x7f1820103660 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.462+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1820103120 con 0x7f1820103660 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.463+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1810042030 con 0x7f1820103660 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.464+0000 7f1816ffd640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 0x7f180c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.464+0000 7f18258f7640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 0x7f180c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.464+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f18100ef6b0 con 0x7f1820103660 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.464+0000 7f18258f7640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 0x7f180c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f181c00abe0 tx=0x7f181c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:17.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.468+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f18100bc760 con 0x7f1820103660 2024-09-12T03:17:17.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.681+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f18201071e0 con 0x7f1820103660 2024-09-12T03:17:17.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.941+0000 7f1816ffd640 1 -- 172.21.15.19:0/1894268149 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f18100bc060 con 0x7f1820103660 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 msgr2=0x7f180c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 0x7f180c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f181c00abe0 tx=0x7f181c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 msgr2=0x7f1820100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1810005ab0 tx=0x7f18100042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 shutdown_connections 2024-09-12T03:17:17.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f182010c170 0x7f1820101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f180c0a2c00 0x7f180c0a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1820103660 0x7f1820100ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 --2- 172.21.15.19:0/1894268149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1820102d40 0x7f18201009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 >> 172.21.15.19:0/1894268149 conn(0x7f18200fc7a0 msgr2=0x7f18200fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 shutdown_connections 2024-09-12T03:17:17.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:17.944+0000 7f1827b5b640 1 -- 172.21.15.19:0/1894268149 wait complete. 2024-09-12T03:17:18.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:17 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T03:17:18.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:17 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T03:17:18.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:17 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T03:17:18.644 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T03:17:18.905 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:19.294 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:17:19.041+0000 7f7545979640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-12T03:17:19.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:17:19.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:17:19.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T03:17:19.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:17:19.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:19.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T03:17:19.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:18 smithi135 ceph-mon[35474]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T03:17:19.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:17:19.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:17:19.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T03:17:19.309 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:17:19.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:19.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T03:17:19.310 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:18 smithi019 ceph-mon[30953]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T03:17:19.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1894268149' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-09-12T03:17:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:18 smithi121 ceph-mon[34963]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 -- 172.21.15.19:0/274392462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c100a80 msgr2=0x7f077c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 --2- 172.21.15.19:0/274392462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c100a80 0x7f077c100ee0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0764009930 tx=0x7f076402a420 comp rx=0 tx=0).stop 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 -- 172.21.15.19:0/274392462 shutdown_connections 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 --2- 172.21.15.19:0/274392462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c101420 0x7f077c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 --2- 172.21.15.19:0/274392462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c100a80 0x7f077c100ee0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 --2- 172.21.15.19:0/274392462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c106a80 0x7f077c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 -- 172.21.15.19:0/274392462 >> 172.21.15.19:0/274392462 conn(0x7f077c0fc7a0 msgr2=0x7f077c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 -- 172.21.15.19:0/274392462 shutdown_connections 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.395+0000 7f07828ae640 1 -- 172.21.15.19:0/274392462 wait complete. 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 Processor -- start 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 -- start start 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c100a80 0x7f077c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c101420 0x7f077c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f077c110f20 con 0x7f077c106a80 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f077c110da0 con 0x7f077c100a80 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f07828ae640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f077c1110a0 con 0x7f077c101420 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54174/0 (socket says 172.21.15.19:54174) 2024-09-12T03:17:19.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 -- 172.21.15.19:0/1792137870 learned_addr learned my addr 172.21.15.19:0/1792137870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0773fff640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c101420 0x7f077c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f077bfff640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c100a80 0x7f077c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 -- 172.21.15.19:0/1792137870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c101420 msgr2=0x7f077c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c101420 0x7f077c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 -- 172.21.15.19:0/1792137870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c100a80 msgr2=0x7f077c1985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c100a80 0x7f077c1985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.396+0000 7f0780e4b640 1 -- 172.21.15.19:0/1792137870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c19d570 con 0x7f077c106a80 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f0780e4b640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f077400b970 tx=0x7f077400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774019070 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f077400cb40 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c19d860 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0774007650 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.397+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1a5140 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.398+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c100f60 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.399+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f077401c070 con 0x7f077c106a80 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.400+0000 7f0779ffb640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 0x7f07680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:19.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.400+0000 7f077bfff640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 0x7f07680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.400+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f07740ce520 con 0x7f077c106a80 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.400+0000 7f077bfff640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 0x7f07680a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f076c009890 tx=0x7f076c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:19.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.404+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f077409b5d0 con 0x7f077c106a80 2024-09-12T03:17:19.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.620+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f077c1a53a0 con 0x7f077c106a80 2024-09-12T03:17:19.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.807+0000 7f0779ffb640 1 -- 172.21.15.19:0/1792137870 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f077409aed0 con 0x7f077c106a80 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 msgr2=0x7f07680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 0x7f07680a50c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f076c009890 tx=0x7f076c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 msgr2=0x7f077c19ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f077400b970 tx=0x7f077400be30 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 shutdown_connections 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07680a2c00 0x7f07680a50c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f077c106a80 0x7f077c19ce90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f077c101420 0x7f077c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 --2- 172.21.15.19:0/1792137870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f077c100a80 0x7f077c1985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:19.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 >> 172.21.15.19:0/1792137870 conn(0x7f077c0fc7a0 msgr2=0x7f077c104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 shutdown_connections 2024-09-12T03:17:19.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:19.809+0000 7f07828ae640 1 -- 172.21.15.19:0/1792137870 wait complete. 2024-09-12T03:17:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:active 2024-09-12T03:17:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:17:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T03:17:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:17:20.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:active 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:20.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:19 smithi135 ceph-mon[35474]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-09-12T03:17:20.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:active 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:active 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:20.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:20.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:19 smithi121 ceph-mon[34963]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] up:active 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 10 up:standby 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:creating} 9 up:standby 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1792137870' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] up:active 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: daemon mds.l assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:creating} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:20.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:19 smithi019 ceph-mon[30953]: daemon mds.l is now active in filesystem cephfs as rank 3 2024-09-12T03:17:20.437 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T03:17:20.699 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:active 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: Cluster is now healthy 2024-09-12T03:17:21.092 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:21.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-12T03:17:21.093 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:20 smithi019 ceph-mon[30953]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-12T03:17:21.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 -- 172.21.15.19:0/222344964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0101de0 msgr2=0x7fd5a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/222344964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0101de0 0x7fd5a01021c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009900 tx=0x7fd59c02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 -- 172.21.15.19:0/222344964 shutdown_connections 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/222344964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0106f10 0x7fd5a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/222344964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0102790 0x7fd5a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/222344964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0101de0 0x7fd5a01021c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 -- 172.21.15.19:0/222344964 >> 172.21.15.19:0/222344964 conn(0x7fd5a00fd8c0 msgr2=0x7fd5a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 -- 172.21.15.19:0/222344964 shutdown_connections 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.176+0000 7fd5a7cab640 1 -- 172.21.15.19:0/222344964 wait complete. 2024-09-12T03:17:21.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 Processor -- start 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 -- start start 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0101de0 0x7fd5a00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0106f10 0x7fd5a0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111f70 con 0x7fd5a0102790 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0111df0 con 0x7fd5a0101de0 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a7cab640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a01120f0 con 0x7fd5a0106f10 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a5246640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a5246640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55562/0 (socket says 172.21.15.19:55562) 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.177+0000 7fd5a5246640 1 -- 172.21.15.19:0/1300537242 learned_addr learned my addr 172.21.15.19:0/1300537242 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:21.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a6248640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0106f10 0x7fd5a0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5a47640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0101de0 0x7fd5a00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 -- 172.21.15.19:0/1300537242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0106f10 msgr2=0x7fd5a0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0106f10 0x7fd5a0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 -- 172.21.15.19:0/1300537242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0101de0 msgr2=0x7fd5a00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0101de0 0x7fd5a00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 -- 172.21.15.19:0/1300537242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a01a6930 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a5246640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d770 tx=0x7fd59000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd590005500 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a01a6c20 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd590004e00 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.178+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01a7110 con 0x7fd5a0102790 2024-09-12T03:17:21.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.179+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a01021c0 con 0x7fd5a0102790 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.180+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd590004fa0 con 0x7fd5a0102790 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.181+0000 7fd596ffd640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 0x7fd58c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.181+0000 7fd5a5a47640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 0x7fd58c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.181+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fd5900cea20 con 0x7fd5a0102790 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.181+0000 7fd5a5a47640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 0x7fd58c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005ab0 tx=0x7fd59c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:21.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.185+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd59009bad0 con 0x7fd5a0102790 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:active 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: Cluster is now healthy 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-12T03:17:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:20 smithi135 ceph-mon[35474]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: pgmap v284: 97 pgs: 97 active+clean; 580 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s wr, 6 op/s 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] up:active 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: daemon mds.k assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: Cluster is now healthy 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=c=up:active,3=l=up:active} 3 up:standby-replay 5 up:standby 2024-09-12T03:17:21.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-09-12T03:17:21.302 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:20 smithi121 ceph-mon[34963]: daemon mds.k is now active in filesystem cephfs as rank 4 2024-09-12T03:17:21.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.396+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fd5a010a0f0 con 0x7fd5a0102790 2024-09-12T03:17:21.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.821+0000 7fd596ffd640 1 -- 172.21.15.19:0/1300537242 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fd59009b3d0 con 0x7fd5a0102790 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.823+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 msgr2=0x7fd58c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.823+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 0x7fd58c0a5040 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd59c005ab0 tx=0x7fd59c005a20 comp rx=0 tx=0).stop 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 msgr2=0x7fd5a0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd59000d770 tx=0x7fd59000dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 shutdown_connections 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd5a0106f10 0x7fd5a0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd58c0a2b80 0x7fd58c0a5040 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd5a0102790 0x7fd5a0074a30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 --2- 172.21.15.19:0/1300537242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd5a0101de0 0x7fd5a00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 >> 172.21.15.19:0/1300537242 conn(0x7fd5a00fd8c0 msgr2=0x7fd5a00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 shutdown_connections 2024-09-12T03:17:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:21.824+0000 7fd5a7cab640 1 -- 172.21.15.19:0/1300537242 wait complete. 2024-09-12T03:17:22.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:17:22.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:17:22.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:21 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:active 2024-09-12T03:17:22.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:21 smithi019 ceph-mon[30953]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-12T03:17:22.148 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:21 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:17:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:17:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:17:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:21 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:active 2024-09-12T03:17:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:21 smithi135 ceph-mon[35474]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-12T03:17:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:21 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:17:22.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T03:17:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1300537242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T03:17:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:21 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] up:active 2024-09-12T03:17:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:21 smithi121 ceph-mon[34963]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-09-12T03:17:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:21 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:17:22.601 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-12T03:17:22.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T03:17:22.862 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:23.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:22 smithi019 ceph-mon[30953]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 692 B/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T03:17:23.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:22 smithi135 ceph-mon[35474]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 692 B/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T03:17:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 -- 172.21.15.19:0/3567896385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c1036c0 msgr2=0x7f659c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:23.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 --2- 172.21.15.19:0/3567896385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c1036c0 0x7f659c10bbb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f658c009900 tx=0x7f658c02a480 comp rx=0 tx=0).stop 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 -- 172.21.15.19:0/3567896385 shutdown_connections 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 --2- 172.21.15.19:0/3567896385 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 0x7f659c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 --2- 172.21.15.19:0/3567896385 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c1036c0 0x7f659c10bbb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 --2- 172.21.15.19:0/3567896385 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c102da0 0x7f659c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 -- 172.21.15.19:0/3567896385 >> 172.21.15.19:0/3567896385 conn(0x7f659c0fc7a0 msgr2=0x7f659c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 -- 172.21.15.19:0/3567896385 shutdown_connections 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.355+0000 7f65a4125640 1 -- 172.21.15.19:0/3567896385 wait complete. 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 Processor -- start 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 -- start start 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:23.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 0x7f659c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 0x7f659c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111f60 con 0x7f659c102da0 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f659c111de0 con 0x7f659c1036c0 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a4125640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f659c1120e0 con 0x7f659c10c180 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55582/0 (socket says 172.21.15.19:55582) 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 -- 172.21.15.19:0/2182076334 learned_addr learned my addr 172.21.15.19:0/2182076334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a16c0640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 0x7f659c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 -- 172.21.15.19:0/2182076334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 msgr2=0x7f659c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a26c2640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 0x7f659c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 0x7f659c19cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 -- 172.21.15.19:0/2182076334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 msgr2=0x7f659c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 0x7f659c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a1ec1640 1 -- 172.21.15.19:0/2182076334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c19d360 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.356+0000 7f65a16c0640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 0x7f659c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f65a1ec1640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f659800c920 tx=0x7f659800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598007c70 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c19d650 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6598007e10 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598018890 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.357+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1a4f30 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.358+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c103180 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.359+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6598018a70 con 0x7f659c102da0 2024-09-12T03:17:23.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.360+0000 7f6592ffd640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 0x7f65880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.360+0000 7f65a16c0640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 0x7f65880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.360+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f65980cf0f0 con 0x7f659c102da0 2024-09-12T03:17:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.360+0000 7f65a16c0640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 0x7f65880a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f659c0fedc0 tx=0x7f658c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:23.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.364+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6598014030 con 0x7f659c102da0 2024-09-12T03:17:23.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:22 smithi121 ceph-mon[34963]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 692 B/s rd, 3.1 KiB/s wr, 12 op/s 2024-09-12T03:17:23.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.567+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f659c062630 con 0x7f65880a2b80 2024-09-12T03:17:23.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.594+0000 7f6592ffd640 1 -- 172.21.15.19:0/2182076334 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f659c062630 con 0x7f65880a2b80 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 msgr2=0x7f65880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 0x7f65880a5040 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f659c0fedc0 tx=0x7f658c005a20 comp rx=0 tx=0).stop 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 msgr2=0x7f659c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f659800c920 tx=0x7f659800cde0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 shutdown_connections 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f659c10c180 0x7f659c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f659c1036c0 0x7f659c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f65880a2b80 0x7f65880a5040 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 --2- 172.21.15.19:0/2182076334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f659c102da0 0x7f659c1983b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 >> 172.21.15.19:0/2182076334 conn(0x7f659c0fc7a0 msgr2=0x7f659c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 shutdown_connections 2024-09-12T03:17:23.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:23.596+0000 7f65a4125640 1 -- 172.21.15.19:0/2182076334 wait complete. 2024-09-12T03:17:23.914 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:17:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:17:23.572+0000 7fa9e303e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:23.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:17:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:17:23.572+0000 7fa9e303e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:23.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:17:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:17:23.572+0000 7fa9e303e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:23.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:17:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:17:23.572+0000 7fa9e303e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:23.915 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:17:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:17:23.572+0000 7fa9e303e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T03:17:24.370 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T03:17:24.627 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:25.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:24 smithi019 ceph-mon[30953]: pgmap v286: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-12T03:17:25.048 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:24 smithi019 ceph-mon[30953]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.135+0000 7f141840e640 1 -- 172.21.15.19:0/1325608142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410102570 msgr2=0x7f14101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.135+0000 7f141840e640 1 --2- 172.21.15.19:0/1325608142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410102570 0x7f14101029b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1404009900 tx=0x7f140402a480 comp rx=0 tx=0).stop 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 -- 172.21.15.19:0/1325608142 shutdown_connections 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 --2- 172.21.15.19:0/1325608142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102ef0 0x7f141010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 --2- 172.21.15.19:0/1325608142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410102570 0x7f14101029b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 --2- 172.21.15.19:0/1325608142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410100e20 0x7f1410101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 -- 172.21.15.19:0/1325608142 >> 172.21.15.19:0/1325608142 conn(0x7f14100fc7d0 msgr2=0x7f14100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 -- 172.21.15.19:0/1325608142 shutdown_connections 2024-09-12T03:17:25.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 -- 172.21.15.19:0/1325608142 wait complete. 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.136+0000 7f141840e640 1 Processor -- start 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 -- start start 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 0x7f1410198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410102ef0 0x7f141019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1410112060 con 0x7f1410100e20 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1410111ee0 con 0x7f1410102570 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f141840e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14101121e0 con 0x7f1410102ef0 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55620/0 (socket says 172.21.15.19:55620) 2024-09-12T03:17:25.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 -- 172.21.15.19:0/3616826616 learned_addr learned my addr 172.21.15.19:0/3616826616 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14169ab640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410102ef0 0x7f141019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14159a9640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 0x7f1410198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 -- 172.21.15.19:0/3616826616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410102ef0 msgr2=0x7f141019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410102ef0 0x7f141019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 -- 172.21.15.19:0/3616826616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 msgr2=0x7f1410198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 0x7f1410198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 -- 172.21.15.19:0/3616826616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141019d4a0 con 0x7f1410100e20 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14159a9640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 0x7f1410198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.137+0000 7f14161aa640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f140000d950 tx=0x7f140000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.138+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400017070 con 0x7f1410100e20 2024-09-12T03:17:25.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.138+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1400009c30 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.138+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141019d790 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.138+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140000f6b0 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.138+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14101a5070 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.139+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1410101200 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.140+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f14000109d0 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.141+0000 7f140f7fe640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 0x7f13f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.141+0000 7f14159a9640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 0x7f13f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.141+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f14000cec20 con 0x7f1410100e20 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.141+0000 7f14159a9640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 0x7f13f80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c00 tx=0x7f1404004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:25.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.145+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1400009de0 con 0x7f1410100e20 2024-09-12T03:17:25.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:24 smithi135 ceph-mon[35474]: pgmap v286: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-12T03:17:25.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:24 smithi135 ceph-mon[35474]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:25.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.346+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1410062630 con 0x7f13f80a2b80 2024-09-12T03:17:25.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.370+0000 7f140f7fe640 1 -- 172.21.15.19:0/3616826616 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1410062630 con 0x7f13f80a2b80 2024-09-12T03:17:25.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 msgr2=0x7f13f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 0x7f13f80a5040 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1404000c00 tx=0x7f1404004100 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 msgr2=0x7f14101984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f140000d950 tx=0x7f140000de10 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 shutdown_connections 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1410102ef0 0x7f141019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1410102570 0x7f1410198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f13f80a2b80 0x7f13f80a5040 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 --2- 172.21.15.19:0/3616826616 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1410100e20 0x7f14101984f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 >> 172.21.15.19:0/3616826616 conn(0x7f14100fc7d0 msgr2=0x7f14100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 shutdown_connections 2024-09-12T03:17:25.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:25.372+0000 7f141840e640 1 -- 172.21.15.19:0/3616826616 wait complete. 2024-09-12T03:17:25.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:24 smithi121 ceph-mon[34963]: pgmap v286: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-09-12T03:17:25.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:24 smithi121 ceph-mon[34963]: from='client.14673 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:26.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T03:17:26.468 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 -- 172.21.15.19:0/2153372470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc71810c180 msgr2=0x7fc71810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/2153372470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc71810c180 0x7fc71810e600 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc710009ca0 tx=0x7fc71002a950 comp rx=0 tx=0).stop 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 -- 172.21.15.19:0/2153372470 shutdown_connections 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/2153372470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc71810c180 0x7fc71810e600 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/2153372470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc71810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/2153372470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc718102da0 0x7fc718103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 -- 172.21.15.19:0/2153372470 >> 172.21.15.19:0/2153372470 conn(0x7fc7180fc7a0 msgr2=0x7fc7180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 -- 172.21.15.19:0/2153372470 shutdown_connections 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.944+0000 7fc71cc9e640 1 -- 172.21.15.19:0/2153372470 wait complete. 2024-09-12T03:17:26.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 Processor -- start 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 -- start start 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc718198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc71810c180 0x7fc71819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc718111fb0 con 0x7fc718102da0 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc718111e30 con 0x7fc7181036c0 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71cc9e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc718112130 con 0x7fc71810c180 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55640/0 (socket says 172.21.15.19:55640) 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.945+0000 7fc71659c640 1 -- 172.21.15.19:0/1002085865 learned_addr learned my addr 172.21.15.19:0/1002085865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc716d9d640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc71810c180 0x7fc71819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc70ffff640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc718198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 -- 172.21.15.19:0/1002085865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc71810c180 msgr2=0x7fc71819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc71810c180 0x7fc71819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 -- 172.21.15.19:0/1002085865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 msgr2=0x7fc718198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc718198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 -- 172.21.15.19:0/1002085865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71819d2f0 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc70ffff640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc718198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71659c640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d770 tx=0x7fc70800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708017070 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc708005500 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71819d5e0 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708004e00 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.946+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7181a4ec0 con 0x7fc718102da0 2024-09-12T03:17:26.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.947+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc718103180 con 0x7fc718102da0 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.948+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc708004fa0 con 0x7fc718102da0 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.949+0000 7fc70f7fe640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 0x7fc7040a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.949+0000 7fc70ffff640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 0x7fc7040a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.949+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc7080cef10 con 0x7fc718102da0 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.950+0000 7fc70ffff640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 0x7fc7040a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc7180fecc0 tx=0x7fc700005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:26.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:26.952+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc70809bf40 con 0x7fc718102da0 2024-09-12T03:17:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.154+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fc7180fe460 con 0x7fc7040a2c90 2024-09-12T03:17:27.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:26 smithi019 ceph-mon[30953]: pgmap v287: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-12T03:17:27.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:26 smithi019 ceph-mon[30953]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:27.155 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:26 smithi019 ceph-mon[30953]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:17:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.159+0000 7fc70f7fe640 1 -- 172.21.15.19:0/1002085865 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc7180fe460 con 0x7fc7040a2c90 2024-09-12T03:17:27.159 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_0/c2c030d6-190f-480c-837c-7d02a7065565 2024-09-12T03:17:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 msgr2=0x7fc7040a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 0x7fc7040a5150 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc7180fecc0 tx=0x7fc700005d20 comp rx=0 tx=0).stop 2024-09-12T03:17:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 msgr2=0x7fc718198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc70800d770 tx=0x7fc70800dc30 comp rx=0 tx=0).stop 2024-09-12T03:17:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 shutdown_connections 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc71810c180 0x7fc71819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7181036c0 0x7fc718198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc7040a2c90 0x7fc7040a5150 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 --2- 172.21.15.19:0/1002085865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc718102da0 0x7fc718198340 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 >> 172.21.15.19:0/1002085865 conn(0x7fc7180fc7a0 msgr2=0x7fc7180fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 shutdown_connections 2024-09-12T03:17:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:27.161+0000 7fc71cc9e640 1 -- 172.21.15.19:0/1002085865 wait complete. 2024-09-12T03:17:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:26 smithi135 ceph-mon[35474]: pgmap v287: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-12T03:17:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:26 smithi135 ceph-mon[35474]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:26 smithi135 ceph-mon[35474]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:17:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:26 smithi121 ceph-mon[34963]: pgmap v287: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 5.7 KiB/s wr, 21 op/s 2024-09-12T03:17:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:26 smithi121 ceph-mon[34963]: from='client.14685 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:26 smithi121 ceph-mon[34963]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T03:17:27.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T03:17:28.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:28.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:27 smithi019 ceph-mon[30953]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:28.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:27 smithi135 ceph-mon[35474]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:28.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:27 smithi121 ceph-mon[34963]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 -- 172.21.15.19:0/3845064850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0101420 msgr2=0x7efcb010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 --2- 172.21.15.19:0/3845064850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0101420 0x7efcb010d950 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efca8009cd0 tx=0x7efca802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 -- 172.21.15.19:0/3845064850 shutdown_connections 2024-09-12T03:17:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 --2- 172.21.15.19:0/3845064850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0101420 0x7efcb010d950 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 --2- 172.21.15.19:0/3845064850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0100a80 0x7efcb0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 --2- 172.21.15.19:0/3845064850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0106a80 0x7efcb0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 -- 172.21.15.19:0/3845064850 >> 172.21.15.19:0/3845064850 conn(0x7efcb00fc7a0 msgr2=0x7efcb00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 -- 172.21.15.19:0/3845064850 shutdown_connections 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.569+0000 7efcb508b640 1 -- 172.21.15.19:0/3845064850 wait complete. 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 Processor -- start 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 -- start start 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0101420 0x7efcb01988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0106a80 0x7efcb019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0111fa0 con 0x7efcb0100a80 2024-09-12T03:17:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0111e20 con 0x7efcb0101420 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcb508b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112120 con 0x7efcb0106a80 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55652/0 (socket says 172.21.15.19:55652) 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 -- 172.21.15.19:0/1622127444 learned_addr learned my addr 172.21.15.19:0/1622127444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efca7fff640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0101420 0x7efcb01988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaf59e640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0106a80 0x7efcb019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 -- 172.21.15.19:0/1622127444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0106a80 msgr2=0x7efcb019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0106a80 0x7efcb019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 -- 172.21.15.19:0/1622127444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0101420 msgr2=0x7efcb01988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0101420 0x7efcb01988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.570+0000 7efcaed9d640 1 -- 172.21.15.19:0/1622127444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb019d340 con 0x7efcb0100a80 2024-09-12T03:17:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcaed9d640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0007c70 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca0007e10 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb019d630 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca0018890 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.571+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb01a4f10 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.572+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb0100ee0 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.573+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7efca0018a70 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.574+0000 7efcacd99640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.574+0000 7efca7fff640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.574+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7efca00cf1a0 con 0x7efcb0100a80 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.574+0000 7efca7fff640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc98005fd0 tx=0x7efc98005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.578+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efca0014030 con 0x7efcb0100a80 2024-09-12T03:17:28.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.779+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efcb00fd4e0 con 0x7efc9c0a2c10 2024-09-12T03:17:28.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.804+0000 7efcacd99640 1 -- 172.21.15.19:0/1622127444 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efcb00fd4e0 con 0x7efc9c0a2c10 2024-09-12T03:17:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 msgr2=0x7efc9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efc98005fd0 tx=0x7efc98005e20 comp rx=0 tx=0).stop 2024-09-12T03:17:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 msgr2=0x7efcb0198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efca000c920 tx=0x7efca000cde0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 shutdown_connections 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcb0106a80 0x7efcb019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcb0101420 0x7efcb01988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc9c0a2c10 0x7efc9c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 --2- 172.21.15.19:0/1622127444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcb0100a80 0x7efcb0198390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 >> 172.21.15.19:0/1622127444 conn(0x7efcb00fc7a0 msgr2=0x7efcb00fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 shutdown_connections 2024-09-12T03:17:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:28.806+0000 7efcb508b640 1 -- 172.21.15.19:0/1622127444 wait complete. 2024-09-12T03:17:29.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:28 smithi019 ceph-mon[30953]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-09-12T03:17:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:28 smithi135 ceph-mon[35474]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-09-12T03:17:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:28 smithi121 ceph-mon[34963]: pgmap v288: 97 pgs: 97 active+clean; 591 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 5.4 KiB/s wr, 21 op/s 2024-09-12T03:17:29.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T03:17:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:30.101 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:30 smithi019 ceph-mon[30953]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:30.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 -- 172.21.15.19:0/2625458369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f237410c170 msgr2=0x7f237410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 --2- 172.21.15.19:0/2625458369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f237410c170 0x7f237410e610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f236c009ca0 tx=0x7f236c02a950 comp rx=0 tx=0).stop 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 -- 172.21.15.19:0/2625458369 shutdown_connections 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 --2- 172.21.15.19:0/2625458369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f237410c170 0x7f237410e610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 --2- 172.21.15.19:0/2625458369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374103660 0x7f237410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 --2- 172.21.15.19:0/2625458369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2374102d40 0x7f2374103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 -- 172.21.15.19:0/2625458369 >> 172.21.15.19:0/2625458369 conn(0x7f23740fc7a0 msgr2=0x7f23740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 -- 172.21.15.19:0/2625458369 shutdown_connections 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.187+0000 7f2378f62640 1 -- 172.21.15.19:0/2625458369 wait complete. 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 Processor -- start 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 -- start start 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374102d40 0x7f2374198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 0x7f237419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2374111fb0 con 0x7f2374103660 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2374111e30 con 0x7f2374102d40 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2378f62640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2374112130 con 0x7f237410c170 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.188+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55658/0 (socket says 172.21.15.19:55658) 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 -- 172.21.15.19:0/2806075639 learned_addr learned my addr 172.21.15.19:0/2806075639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2372d9d640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 0x7f237419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f237259c640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374102d40 0x7f2374198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 -- 172.21.15.19:0/2806075639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 msgr2=0x7f237419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 0x7f237419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 -- 172.21.15.19:0/2806075639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374102d40 msgr2=0x7f2374198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374102d40 0x7f2374198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 -- 172.21.15.19:0/2806075639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237419d420 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2372d9d640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 0x7f237419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2371d9b640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f236800c940 tx=0x7f236800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004220 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f23680043c0 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237419d710 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368004a00 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.189+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23741a4ff0 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.190+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2374103120 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.192+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23680079b0 con 0x7f2374103660 2024-09-12T03:17:30.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.192+0000 7f23637fe640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 0x7f235c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.192+0000 7f237259c640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 0x7f235c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.192+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f23680cf0d0 con 0x7f2374103660 2024-09-12T03:17:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.193+0000 7f237259c640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 0x7f235c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2364009890 tx=0x7f2364008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:30.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.197+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2368014030 con 0x7f2374103660 2024-09-12T03:17:30.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:30 smithi135 ceph-mon[35474]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:30.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.399+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f23740fe480 con 0x7f235c0a2c10 2024-09-12T03:17:30.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:30 smithi121 ceph-mon[34963]: from='client.14697 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:30.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.404+0000 7f23637fe640 1 -- 172.21.15.19:0/2806075639 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f23740fe480 con 0x7f235c0a2c10 2024-09-12T03:17:30.404 INFO:teuthology.orchestra.run.smithi019.stdout:/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6 2024-09-12T03:17:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 msgr2=0x7f235c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 0x7f235c0a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2364009890 tx=0x7f2364008040 comp rx=0 tx=0).stop 2024-09-12T03:17:30.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 msgr2=0x7f23741989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f236800c940 tx=0x7f236800ce00 comp rx=0 tx=0).stop 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 shutdown_connections 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f237410c170 0x7f237419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f235c0a2c10 0x7f235c0a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2374103660 0x7f23741989b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 --2- 172.21.15.19:0/2806075639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2374102d40 0x7f2374198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 >> 172.21.15.19:0/2806075639 conn(0x7f23740fc7a0 msgr2=0x7f23740fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 shutdown_connections 2024-09-12T03:17:30.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:30.406+0000 7f2378f62640 1 -- 172.21.15.19:0/2806075639 wait complete. 2024-09-12T03:17:31.105 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:31 smithi019 ceph-mon[30953]: pgmap v289: 97 pgs: 97 active+clean; 592 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-09-12T03:17:31.137 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:17:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:31 smithi135 ceph-mon[35474]: pgmap v289: 97 pgs: 97 active+clean; 592 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-09-12T03:17:31.395 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:31 smithi121 ceph-mon[34963]: pgmap v289: 97 pgs: 97 active+clean; 592 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.9 KiB/s wr, 21 op/s 2024-09-12T03:17:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 -- 172.21.15.19:0/954358656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680101420 msgr2=0x7fa68010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 --2- 172.21.15.19:0/954358656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680101420 0x7fa68010d950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa678009cd0 tx=0x7fa67802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 -- 172.21.15.19:0/954358656 shutdown_connections 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 --2- 172.21.15.19:0/954358656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680101420 0x7fa68010d950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 --2- 172.21.15.19:0/954358656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680100a80 0x7fa680100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 --2- 172.21.15.19:0/954358656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680106a80 0x7fa680106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 -- 172.21.15.19:0/954358656 >> 172.21.15.19:0/954358656 conn(0x7fa6800fc7a0 msgr2=0x7fa6800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 -- 172.21.15.19:0/954358656 shutdown_connections 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.875+0000 7fa686b83640 1 -- 172.21.15.19:0/954358656 wait complete. 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 Processor -- start 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 -- start start 2024-09-12T03:17:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680101420 0x7fa6801988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 0x7fa68019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa680110e80 con 0x7fa680100a80 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa680110d00 con 0x7fa680101420 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa686b83640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa680111000 con 0x7fa680106a80 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa68491f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa68491f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54764/0 (socket says 172.21.15.19:54764) 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa68491f640 1 -- 172.21.15.19:0/2101253212 learned_addr learned my addr 172.21.15.19:0/2101253212 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.876+0000 7fa685120640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 0x7fa68019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa67ffff640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680101420 0x7fa6801988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 -- 172.21.15.19:0/2101253212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 msgr2=0x7fa68019cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 0x7fa68019cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 -- 172.21.15.19:0/2101253212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680101420 msgr2=0x7fa6801988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680101420 0x7fa6801988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 -- 172.21.15.19:0/2101253212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68019d340 con 0x7fa680100a80 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa685120640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 0x7fa68019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa68491f640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa67400c920 tx=0x7fa67400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674007c70 con 0x7fa680100a80 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68019d630 con 0x7fa680100a80 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa674007e10 con 0x7fa680100a80 2024-09-12T03:17:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa674018890 con 0x7fa680100a80 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.877+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a4f10 con 0x7fa680100a80 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.878+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680100ee0 con 0x7fa680100a80 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.879+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa674018a70 con 0x7fa680100a80 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.880+0000 7fa67dffb640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 0x7fa6700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.880+0000 7fa67ffff640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 0x7fa6700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.880+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa6740cf2a0 con 0x7fa680100a80 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.880+0000 7fa67ffff640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 0x7fa6700a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6801050f0 tx=0x7fa66c005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:31.884+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa674014030 con 0x7fa680100a80 2024-09-12T03:17:32.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.097+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa68010a080 con 0x7fa680100a80 2024-09-12T03:17:32.098 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:32 smithi019 ceph-mon[30953]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:32.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.098+0000 7fa67dffb640 1 -- 172.21.15.19:0/2101253212 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7fa67409c150 con 0x7fa680100a80 2024-09-12T03:17:32.098 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:32.099 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:17:32.099 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:17:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 msgr2=0x7fa6700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 0x7fa6700a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa6801050f0 tx=0x7fa66c005c60 comp rx=0 tx=0).stop 2024-09-12T03:17:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 msgr2=0x7fa680198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa67400c920 tx=0x7fa67400cde0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 shutdown_connections 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa680106a80 0x7fa68019cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa680101420 0x7fa6801988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa6700a2c10 0x7fa6700a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 --2- 172.21.15.19:0/2101253212 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa680100a80 0x7fa680198390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 >> 172.21.15.19:0/2101253212 conn(0x7fa6800fc7a0 msgr2=0x7fa68006a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 shutdown_connections 2024-09-12T03:17:32.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:32.100+0000 7fa686b83640 1 -- 172.21.15.19:0/2101253212 wait complete. 2024-09-12T03:17:32.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:32 smithi135 ceph-mon[35474]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:32.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:32 smithi121 ceph-mon[34963]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:17:33.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:33.265 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:33.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:33 smithi135 ceph-mon[35474]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 24 op/s 2024-09-12T03:17:33.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:33 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2101253212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:33.386 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:33 smithi019 ceph-mon[30953]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 24 op/s 2024-09-12T03:17:33.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:33 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2101253212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:33.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:33 smithi121 ceph-mon[34963]: pgmap v290: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.0 KiB/s wr, 24 op/s 2024-09-12T03:17:33.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:33 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2101253212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 -- 172.21.15.19:0/526309608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210102ef0 msgr2=0x7fc21010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 --2- 172.21.15.19:0/526309608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210102ef0 0x7fc21010da40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc208009ca0 tx=0x7fc20802a950 comp rx=0 tx=0).stop 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 -- 172.21.15.19:0/526309608 shutdown_connections 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 --2- 172.21.15.19:0/526309608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210102ef0 0x7fc21010da40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 --2- 172.21.15.19:0/526309608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102570 0x7fc2101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 --2- 172.21.15.19:0/526309608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210100e20 0x7fc210101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 -- 172.21.15.19:0/526309608 >> 172.21.15.19:0/526309608 conn(0x7fc2100fc7d0 msgr2=0x7fc2100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 -- 172.21.15.19:0/526309608 shutdown_connections 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 -- 172.21.15.19:0/526309608 wait complete. 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.733+0000 7fc216121640 1 Processor -- start 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 -- start start 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 0x7fc2101998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102ef0 0x7fc2101a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc210110fa0 con 0x7fc210100e20 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc210110e20 con 0x7fc210102570 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc216121640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc210111120 con 0x7fc210102ef0 2024-09-12T03:17:33.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54786/0 (socket says 172.21.15.19:54786) 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 -- 172.21.15.19:0/748842862 learned_addr learned my addr 172.21.15.19:0/748842862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20ffff640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102ef0 0x7fc2101a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20effd640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 0x7fc2101998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 -- 172.21.15.19:0/748842862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102ef0 msgr2=0x7fc2101a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102ef0 0x7fc2101a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 -- 172.21.15.19:0/748842862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 msgr2=0x7fc2101998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 0x7fc2101998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 -- 172.21.15.19:0/748842862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2101a4970 con 0x7fc210100e20 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20effd640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 0x7fc2101998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.734+0000 7fc20f7fe640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b4a0 tx=0x7fc20000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.735+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200014070 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.735+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc200010af0 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.735+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2101a4b40 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.735+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc200017d10 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.735+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2101a5080 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.736+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc210101200 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.737+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc200017070 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.738+0000 7fc20cff9640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 0x7fc1fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.738+0000 7fc20effd640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 0x7fc1fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.738+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc2000ce790 con 0x7fc210100e20 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.738+0000 7fc20effd640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 0x7fc1fc0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc21006aac0 tx=0x7fc1f8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:33.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.741+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc20009b750 con 0x7fc210100e20 2024-09-12T03:17:33.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.917+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc21006a560 con 0x7fc210100e20 2024-09-12T03:17:33.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.919+0000 7fc20cff9640 1 -- 172.21.15.19:0/748842862 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22303 (secure 0 0 0) 0x7fc21006a560 con 0x7fc210100e20 2024-09-12T03:17:33.919 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:33.919 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:17:10.527091+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.921+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 msgr2=0x7fc1fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.921+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 0x7fc1fc0a5150 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc21006aac0 tx=0x7fc1f8005d30 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.921+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 msgr2=0x7fc2101993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.921+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b4a0 tx=0x7fc20000b960 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 shutdown_connections 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc210102ef0 0x7fc2101a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc210102570 0x7fc2101998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc1fc0a2c90 0x7fc1fc0a5150 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 --2- 172.21.15.19:0/748842862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc210100e20 0x7fc2101993b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 >> 172.21.15.19:0/748842862 conn(0x7fc2100fc7d0 msgr2=0x7fc210106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 shutdown_connections 2024-09-12T03:17:33.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:33.922+0000 7fc216121640 1 -- 172.21.15.19:0/748842862 wait complete. 2024-09-12T03:17:34.239 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:34 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/748842862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:34.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:34 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/748842862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:34 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/748842862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:34.728 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:17:34.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:17:34.991 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:35.395 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:35 smithi019 ceph-mon[30953]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 20 op/s 2024-09-12T03:17:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:35 smithi121 ceph-mon[34963]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 20 op/s 2024-09-12T03:17:35.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 -- 172.21.15.19:0/2665307600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae81049a0 msgr2=0x7ffae81089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 --2- 172.21.15.19:0/2665307600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae81049a0 0x7ffae81089f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffad0009900 tx=0x7ffad002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 -- 172.21.15.19:0/2665307600 shutdown_connections 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 --2- 172.21.15.19:0/2665307600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae8109120 0x7ffae8110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 --2- 172.21.15.19:0/2665307600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae81049a0 0x7ffae81089f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 --2- 172.21.15.19:0/2665307600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 -- 172.21.15.19:0/2665307600 >> 172.21.15.19:0/2665307600 conn(0x7ffae80ffab0 msgr2=0x7ffae8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 -- 172.21.15.19:0/2665307600 shutdown_connections 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.476+0000 7ffaed86e640 1 -- 172.21.15.19:0/2665307600 wait complete. 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 Processor -- start 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 -- start start 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae81049a0 0x7ffae819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8114120 con 0x7ffae8109120 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8113fa0 con 0x7ffae8103ff0 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffaed86e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffae81142a0 con 0x7ffae81049a0 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54806/0 (socket says 172.21.15.19:54806) 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffae77fe640 1 -- 172.21.15.19:0/378576230 learned_addr learned my addr 172.21.15.19:0/378576230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffadffff640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae81049a0 0x7ffae819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.477+0000 7ffae6ffd640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 -- 172.21.15.19:0/378576230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae81049a0 msgr2=0x7ffae819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae81049a0 0x7ffae819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 -- 172.21.15.19:0/378576230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 msgr2=0x7ffae819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 -- 172.21.15.19:0/378576230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae819f650 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae6ffd640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffae77fe640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffae000eaa0 tx=0x7ffae000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae000cdb0 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae0005020 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae819f940 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae00116b0 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.478+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81a7220 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.479+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae81043d0 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.480+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffae0005560 con 0x7ffae8109120 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.481+0000 7ffaec86c640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 0x7ffad40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:35.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.481+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ffae00cf1a0 con 0x7ffae8109120 2024-09-12T03:17:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.481+0000 7ffae6ffd640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 0x7ffad40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.482+0000 7ffae6ffd640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 0x7ffad40a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad80058c0 tx=0x7ffad8005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:35.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.485+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffae0016030 con 0x7ffae8109120 2024-09-12T03:17:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:35 smithi135 ceph-mon[35474]: pgmap v291: 97 pgs: 97 active+clean; 624 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 20 op/s 2024-09-12T03:17:35.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.655+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffae8062630 con 0x7ffad40a2c90 2024-09-12T03:17:35.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.663+0000 7ffaec86c640 1 -- 172.21.15.19:0/378576230 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258932 (secure 0 0 0) 0x7ffae8062630 con 0x7ffad40a2c90 2024-09-12T03:17:35.666 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:35.667 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:17:35.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 msgr2=0x7ffad40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 0x7ffad40a5150 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad80058c0 tx=0x7ffad8005850 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 msgr2=0x7ffae819ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ffae000eaa0 tx=0x7ffae000ef60 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 shutdown_connections 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffad40a2c90 0x7ffad40a5150 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffae8109120 0x7ffae819ef70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffae81049a0 0x7ffae819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 --2- 172.21.15.19:0/378576230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffae8103ff0 0x7ffae819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:35.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 >> 172.21.15.19:0/378576230 conn(0x7ffae80ffab0 msgr2=0x7ffae8100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 shutdown_connections 2024-09-12T03:17:35.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:35.666+0000 7ffaed86e640 1 -- 172.21.15.19:0/378576230 wait complete. 2024-09-12T03:17:36.459 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-12T03:17:35.283771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652179,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":301,"num_read_kb":270,"num_write":421,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355888,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585936,"statfs":{"total":1151940427776,"available":1151575998464,"internally_reserved":0,"allocated":11059200,"data_stored":6965549,"data_compressed":109431,"data_compressed_allocated":139264,"data_compressed_original":1777664,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":54539,"num_objects":17,"num_object_clones":0,"num_object_copies":51,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":138,"num_read_kb":133,"num_write":66,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.018878"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910936+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:17:10.910936+0000","last_peered":"2024-09-12T03:17:10.910936+0000","last_clean":"2024-09-12T03:17:10.910936+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:17:10.910936+0000","last_undegraded":"2024-09-12T03:17:10.910936+0000","last_fullsized":"2024-09-12T03:17:10.910936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919352+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:17:10.919352+0000","last_peered":"2024-09-12T03:17:10.919352+0000","last_clean":"2024-09-12T03:17:10.919352+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:17:10.919352+0000","last_undegraded":"2024-09-12T03:17:10.919352+0000","last_fullsized":"2024-09-12T03:17:10.919352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917721+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:17:10.917721+0000","last_peered":"2024-09-12T03:17:10.917721+0000","last_clean":"2024-09-12T03:17:10.917721+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:17:10.917721+0000","last_undegraded":"2024-09-12T03:17:10.917721+0000","last_fullsized":"2024-09-12T03:17:10.917721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916429+0000","last_change":"2024-09-12T03:17:09.925311+0000","last_active":"2024-09-12T03:17:10.916429+0000","last_peered":"2024-09-12T03:17:10.916429+0000","last_clean":"2024-09-12T03:17:10.916429+0000","last_became_active":"2024-09-12T03:17:09.924761+0000","last_became_peered":"2024-09-12T03:17:09.924761+0000","last_unstale":"2024-09-12T03:17:10.916429+0000","last_undegraded":"2024-09-12T03:17:10.916429+0000","last_fullsized":"2024-09-12T03:17:10.916429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:40.297483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916678+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:17:10.916678+0000","last_peered":"2024-09-12T03:17:10.916678+0000","last_clean":"2024-09-12T03:17:10.916678+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:17:10.916678+0000","last_undegraded":"2024-09-12T03:17:10.916678+0000","last_fullsized":"2024-09-12T03:17:10.916678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.914951+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:17:10.914951+0000","last_peered":"2024-09-12T03:17:10.914951+0000","last_clean":"2024-09-12T03:17:10.914951+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:17:10.914951+0000","last_undegraded":"2024-09-12T03:17:10.914951+0000","last_fullsized":"2024-09-12T03:17:10.914951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919834+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:17:10.919834+0000","last_peered":"2024-09-12T03:17:10.919834+0000","last_clean":"2024-09-12T03:17:10.919834+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:17:10.919834+0000","last_undegraded":"2024-09-12T03:17:10.919834+0000","last_fullsized":"2024-09-12T03:17:10.919834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911699+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:17:10.911699+0000","last_peered":"2024-09-12T03:17:10.911699+0000","last_clean":"2024-09-12T03:17:10.911699+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:17:10.911699+0000","last_undegraded":"2024-09-12T03:17:10.911699+0000","last_fullsized":"2024-09-12T03:17:10.911699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910435+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:17:10.910435+0000","last_peered":"2024-09-12T03:17:10.910435+0000","last_clean":"2024-09-12T03:17:10.910435+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:17:10.910435+0000","last_undegraded":"2024-09-12T03:17:10.910435+0000","last_fullsized":"2024-09-12T03:17:10.910435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915448+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:17:10.915448+0000","last_peered":"2024-09-12T03:17:10.915448+0000","last_clean":"2024-09-12T03:17:10.915448+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:17:10.915448+0000","last_undegraded":"2024-09-12T03:17:10.915448+0000","last_fullsized":"2024-09-12T03:17:10.915448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920686+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:17:10.920686+0000","last_peered":"2024-09-12T03:17:10.920686+0000","last_clean":"2024-09-12T03:17:10.920686+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:17:10.920686+0000","last_undegraded":"2024-09-12T03:17:10.920686+0000","last_fullsized":"2024-09-12T03:17:10.920686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916574+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:17:10.916574+0000","last_peered":"2024-09-12T03:17:10.916574+0000","last_clean":"2024-09-12T03:17:10.916574+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:17:10.916574+0000","last_undegraded":"2024-09-12T03:17:10.916574+0000","last_fullsized":"2024-09-12T03:17:10.916574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915920+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:17:10.915920+0000","last_peered":"2024-09-12T03:17:10.915920+0000","last_clean":"2024-09-12T03:17:10.915920+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:17:10.915920+0000","last_undegraded":"2024-09-12T03:17:10.915920+0000","last_fullsized":"2024-09-12T03:17:10.915920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918076+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:17:10.918076+0000","last_peered":"2024-09-12T03:17:10.918076+0000","last_clean":"2024-09-12T03:17:10.918076+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:17:10.918076+0000","last_undegraded":"2024-09-12T03:17:10.918076+0000","last_fullsized":"2024-09-12T03:17:10.918076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920627+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:17:10.920627+0000","last_peered":"2024-09-12T03:17:10.920627+0000","last_clean":"2024-09-12T03:17:10.920627+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:17:10.920627+0000","last_undegraded":"2024-09-12T03:17:10.920627+0000","last_fullsized":"2024-09-12T03:17:10.920627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.909906+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:17:10.909906+0000","last_peered":"2024-09-12T03:17:10.909906+0000","last_clean":"2024-09-12T03:17:10.909906+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:17:10.909906+0000","last_undegraded":"2024-09-12T03:17:10.909906+0000","last_fullsized":"2024-09-12T03:17:10.909906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911086+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:17:10.911086+0000","last_peered":"2024-09-12T03:17:10.911086+0000","last_clean":"2024-09-12T03:17:10.911086+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:17:10.911086+0000","last_undegraded":"2024-09-12T03:17:10.911086+0000","last_fullsized":"2024-09-12T03:17:10.911086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917358+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:17:10.917358+0000","last_peered":"2024-09-12T03:17:10.917358+0000","last_clean":"2024-09-12T03:17:10.917358+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:17:10.917358+0000","last_undegraded":"2024-09-12T03:17:10.917358+0000","last_fullsized":"2024-09-12T03:17:10.917358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916130+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:17:10.916130+0000","last_peered":"2024-09-12T03:17:10.916130+0000","last_clean":"2024-09-12T03:17:10.916130+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:17:10.916130+0000","last_undegraded":"2024-09-12T03:17:10.916130+0000","last_fullsized":"2024-09-12T03:17:10.916130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910819+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:17:10.910819+0000","last_peered":"2024-09-12T03:17:10.910819+0000","last_clean":"2024-09-12T03:17:10.910819+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:17:10.910819+0000","last_undegraded":"2024-09-12T03:17:10.910819+0000","last_fullsized":"2024-09-12T03:17:10.910819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915960+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:17:10.915960+0000","last_peered":"2024-09-12T03:17:10.915960+0000","last_clean":"2024-09-12T03:17:10.915960+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:17:10.915960+0000","last_undegraded":"2024-09-12T03:17:10.915960+0000","last_fullsized":"2024-09-12T03:17:10.915960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910084+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:17:10.910084+0000","last_peered":"2024-09-12T03:17:10.910084+0000","last_clean":"2024-09-12T03:17:10.910084+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:17:10.910084+0000","last_undegraded":"2024-09-12T03:17:10.910084+0000","last_fullsized":"2024-09-12T03:17:10.910084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911735+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:17:10.911735+0000","last_peered":"2024-09-12T03:17:10.911735+0000","last_clean":"2024-09-12T03:17:10.911735+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:17:10.911735+0000","last_undegraded":"2024-09-12T03:17:10.911735+0000","last_fullsized":"2024-09-12T03:17:10.911735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917646+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:17:10.917646+0000","last_peered":"2024-09-12T03:17:10.917646+0000","last_clean":"2024-09-12T03:17:10.917646+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:17:10.917646+0000","last_undegraded":"2024-09-12T03:17:10.917646+0000","last_fullsized":"2024-09-12T03:17:10.917646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912354+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:17:10.912354+0000","last_peered":"2024-09-12T03:17:10.912354+0000","last_clean":"2024-09-12T03:17:10.912354+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:17:10.912354+0000","last_undegraded":"2024-09-12T03:17:10.912354+0000","last_fullsized":"2024-09-12T03:17:10.912354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916458+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:17:10.916458+0000","last_peered":"2024-09-12T03:17:10.916458+0000","last_clean":"2024-09-12T03:17:10.916458+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:17:10.916458+0000","last_undegraded":"2024-09-12T03:17:10.916458+0000","last_fullsized":"2024-09-12T03:17:10.916458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910708+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:17:10.910708+0000","last_peered":"2024-09-12T03:17:10.910708+0000","last_clean":"2024-09-12T03:17:10.910708+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:17:10.910708+0000","last_undegraded":"2024-09-12T03:17:10.910708+0000","last_fullsized":"2024-09-12T03:17:10.910708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915969+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:17:10.915969+0000","last_peered":"2024-09-12T03:17:10.915969+0000","last_clean":"2024-09-12T03:17:10.915969+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:17:10.915969+0000","last_undegraded":"2024-09-12T03:17:10.915969+0000","last_fullsized":"2024-09-12T03:17:10.915969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917962+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:17:10.917962+0000","last_peered":"2024-09-12T03:17:10.917962+0000","last_clean":"2024-09-12T03:17:10.917962+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:17:10.917962+0000","last_undegraded":"2024-09-12T03:17:10.917962+0000","last_fullsized":"2024-09-12T03:17:10.917962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915844+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:17:10.915844+0000","last_peered":"2024-09-12T03:17:10.915844+0000","last_clean":"2024-09-12T03:17:10.915844+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:17:10.915844+0000","last_undegraded":"2024-09-12T03:17:10.915844+0000","last_fullsized":"2024-09-12T03:17:10.915844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915971+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:17:10.915971+0000","last_peered":"2024-09-12T03:17:10.915971+0000","last_clean":"2024-09-12T03:17:10.915971+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:17:10.915971+0000","last_undegraded":"2024-09-12T03:17:10.915971+0000","last_fullsized":"2024-09-12T03:17:10.915971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910616+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:17:10.910616+0000","last_peered":"2024-09-12T03:17:10.910616+0000","last_clean":"2024-09-12T03:17:10.910616+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:17:10.910616+0000","last_undegraded":"2024-09-12T03:17:10.910616+0000","last_fullsized":"2024-09-12T03:17:10.910616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918973+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:17:10.918973+0000","last_peered":"2024-09-12T03:17:10.918973+0000","last_clean":"2024-09-12T03:17:10.918973+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:17:10.918973+0000","last_undegraded":"2024-09-12T03:17:10.918973+0000","last_fullsized":"2024-09-12T03:17:10.918973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:31.834475+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:17:31.834475+0000","last_peered":"2024-09-12T03:17:31.834475+0000","last_clean":"2024-09-12T03:17:31.834475+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:17:31.834475+0000","last_undegraded":"2024-09-12T03:17:31.834475+0000","last_fullsized":"2024-09-12T03:17:31.834475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918302+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:17:10.918302+0000","last_peered":"2024-09-12T03:17:10.918302+0000","last_clean":"2024-09-12T03:17:10.918302+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:17:10.918302+0000","last_undegraded":"2024-09-12T03:17:10.918302+0000","last_fullsized":"2024-09-12T03:17:10.918302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.825682+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:17:19.825682+0000","last_peered":"2024-09-12T03:17:19.825682+0000","last_clean":"2024-09-12T03:17:19.825682+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:17:19.825682+0000","last_undegraded":"2024-09-12T03:17:19.825682+0000","last_fullsized":"2024-09-12T03:17:19.825682+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:18.964662+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:17:18.964662+0000","last_peered":"2024-09-12T03:17:18.964662+0000","last_clean":"2024-09-12T03:17:18.964662+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:17:18.964662+0000","last_undegraded":"2024-09-12T03:17:18.964662+0000","last_fullsized":"2024-09-12T03:17:18.964662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911763+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:17:10.911763+0000","last_peered":"2024-09-12T03:17:10.911763+0000","last_clean":"2024-09-12T03:17:10.911763+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:17:10.911763+0000","last_undegraded":"2024-09-12T03:17:10.911763+0000","last_fullsized":"2024-09-12T03:17:10.911763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831556+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:17:20.831556+0000","last_peered":"2024-09-12T03:17:20.831556+0000","last_clean":"2024-09-12T03:17:20.831556+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:17:20.831556+0000","last_undegraded":"2024-09-12T03:17:20.831556+0000","last_fullsized":"2024-09-12T03:17:20.831556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917972+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:17:10.917972+0000","last_peered":"2024-09-12T03:17:10.917972+0000","last_clean":"2024-09-12T03:17:10.917972+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:17:10.917972+0000","last_undegraded":"2024-09-12T03:17:10.917972+0000","last_fullsized":"2024-09-12T03:17:10.917972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:34.838084+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:17:34.838084+0000","last_peered":"2024-09-12T03:17:34.838084+0000","last_clean":"2024-09-12T03:17:34.838084+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:17:34.838084+0000","last_undegraded":"2024-09-12T03:17:34.838084+0000","last_fullsized":"2024-09-12T03:17:34.838084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910386+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:17:10.910386+0000","last_peered":"2024-09-12T03:17:10.910386+0000","last_clean":"2024-09-12T03:17:10.910386+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:17:10.910386+0000","last_undegraded":"2024-09-12T03:17:10.910386+0000","last_fullsized":"2024-09-12T03:17:10.910386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830705+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:17:20.830705+0000","last_peered":"2024-09-12T03:17:20.830705+0000","last_clean":"2024-09-12T03:17:20.830705+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:17:20.830705+0000","last_undegraded":"2024-09-12T03:17:20.830705+0000","last_fullsized":"2024-09-12T03:17:20.830705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920111+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:17:10.920111+0000","last_peered":"2024-09-12T03:17:10.920111+0000","last_clean":"2024-09-12T03:17:10.920111+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:17:10.920111+0000","last_undegraded":"2024-09-12T03:17:10.920111+0000","last_fullsized":"2024-09-12T03:17:10.920111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.817101+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:17:19.817101+0000","last_peered":"2024-09-12T03:17:19.817101+0000","last_clean":"2024-09-12T03:17:19.817101+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:17:19.817101+0000","last_undegraded":"2024-09-12T03:17:19.817101+0000","last_fullsized":"2024-09-12T03:17:19.817101+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910273+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:17:10.910273+0000","last_peered":"2024-09-12T03:17:10.910273+0000","last_clean":"2024-09-12T03:17:10.910273+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:17:10.910273+0000","last_undegraded":"2024-09-12T03:17:10.910273+0000","last_fullsized":"2024-09-12T03:17:10.910273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'27","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:34.839650+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:17:34.839650+0000","last_peered":"2024-09-12T03:17:34.839650+0000","last_clean":"2024-09-12T03:17:34.839650+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:17:34.839650+0000","last_undegraded":"2024-09-12T03:17:34.839650+0000","last_fullsized":"2024-09-12T03:17:34.839650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54369,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":71,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916262+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:17:10.916262+0000","last_peered":"2024-09-12T03:17:10.916262+0000","last_clean":"2024-09-12T03:17:10.916262+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:17:10.916262+0000","last_undegraded":"2024-09-12T03:17:10.916262+0000","last_fullsized":"2024-09-12T03:17:10.916262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830417+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:17:20.830417+0000","last_peered":"2024-09-12T03:17:20.830417+0000","last_clean":"2024-09-12T03:17:20.830417+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:17:20.830417+0000","last_undegraded":"2024-09-12T03:17:20.830417+0000","last_fullsized":"2024-09-12T03:17:20.830417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915552+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:17:10.915552+0000","last_peered":"2024-09-12T03:17:10.915552+0000","last_clean":"2024-09-12T03:17:10.915552+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:17:10.915552+0000","last_undegraded":"2024-09-12T03:17:10.915552+0000","last_fullsized":"2024-09-12T03:17:10.915552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832531+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:17:20.832531+0000","last_peered":"2024-09-12T03:17:20.832531+0000","last_clean":"2024-09-12T03:17:20.832531+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:17:20.832531+0000","last_undegraded":"2024-09-12T03:17:20.832531+0000","last_fullsized":"2024-09-12T03:17:20.832531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920453+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:17:10.920453+0000","last_peered":"2024-09-12T03:17:10.920453+0000","last_clean":"2024-09-12T03:17:10.920453+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:17:10.920453+0000","last_undegraded":"2024-09-12T03:17:10.920453+0000","last_fullsized":"2024-09-12T03:17:10.920453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:30.832893+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:17:30.832893+0000","last_peered":"2024-09-12T03:17:30.832893+0000","last_clean":"2024-09-12T03:17:30.832893+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:17:30.832893+0000","last_undegraded":"2024-09-12T03:17:30.832893+0000","last_fullsized":"2024-09-12T03:17:30.832893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919772+0000","last_change":"2024-09-12T03:17:09.928028+0000","last_active":"2024-09-12T03:17:10.919772+0000","last_peered":"2024-09-12T03:17:10.919772+0000","last_clean":"2024-09-12T03:17:10.919772+0000","last_became_active":"2024-09-12T03:17:09.927706+0000","last_became_peered":"2024-09-12T03:17:09.927706+0000","last_unstale":"2024-09-12T03:17:10.919772+0000","last_undegraded":"2024-09-12T03:17:10.919772+0000","last_fullsized":"2024-09-12T03:17:10.919772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:39.496602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.824423+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:17:19.824423+0000","last_peered":"2024-09-12T03:17:19.824423+0000","last_clean":"2024-09-12T03:17:19.824423+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:17:19.824423+0000","last_undegraded":"2024-09-12T03:17:19.824423+0000","last_fullsized":"2024-09-12T03:17:19.824423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916185+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:17:10.916185+0000","last_peered":"2024-09-12T03:17:10.916185+0000","last_clean":"2024-09-12T03:17:10.916185+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:17:10.916185+0000","last_undegraded":"2024-09-12T03:17:10.916185+0000","last_fullsized":"2024-09-12T03:17:10.916185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.814808+0000","last_change":"2024-09-12T03:17:07.831968+0000","last_active":"2024-09-12T03:17:19.814808+0000","last_peered":"2024-09-12T03:17:19.814808+0000","last_clean":"2024-09-12T03:17:19.814808+0000","last_became_active":"2024-09-12T03:17:07.828479+0000","last_became_peered":"2024-09-12T03:17:07.828479+0000","last_unstale":"2024-09-12T03:17:19.814808+0000","last_undegraded":"2024-09-12T03:17:19.814808+0000","last_fullsized":"2024-09-12T03:17:19.814808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:55.104152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910685+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:17:10.910685+0000","last_peered":"2024-09-12T03:17:10.910685+0000","last_clean":"2024-09-12T03:17:10.910685+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:17:10.910685+0000","last_undegraded":"2024-09-12T03:17:10.910685+0000","last_fullsized":"2024-09-12T03:17:10.910685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.829356+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:17:20.829356+0000","last_peered":"2024-09-12T03:17:20.829356+0000","last_clean":"2024-09-12T03:17:20.829356+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:17:20.829356+0000","last_undegraded":"2024-09-12T03:17:20.829356+0000","last_fullsized":"2024-09-12T03:17:20.829356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920448+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:17:10.920448+0000","last_peered":"2024-09-12T03:17:10.920448+0000","last_clean":"2024-09-12T03:17:10.920448+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:17:10.920448+0000","last_undegraded":"2024-09-12T03:17:10.920448+0000","last_fullsized":"2024-09-12T03:17:10.920448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.825142+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:17:19.825142+0000","last_peered":"2024-09-12T03:17:19.825142+0000","last_clean":"2024-09-12T03:17:19.825142+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:17:19.825142+0000","last_undegraded":"2024-09-12T03:17:19.825142+0000","last_fullsized":"2024-09-12T03:17:19.825142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.130685+0000","last_change":"2024-09-12T03:17:09.925753+0000","last_active":"2024-09-12T03:17:12.130685+0000","last_peered":"2024-09-12T03:17:12.130685+0000","last_clean":"2024-09-12T03:17:12.130685+0000","last_became_active":"2024-09-12T03:17:09.925426+0000","last_became_peered":"2024-09-12T03:17:09.925426+0000","last_unstale":"2024-09-12T03:17:12.130685+0000","last_undegraded":"2024-09-12T03:17:12.130685+0000","last_fullsized":"2024-09-12T03:17:12.130685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:13.548318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":34,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:28.845833+0000","last_change":"2024-09-12T03:17:07.981943+0000","last_active":"2024-09-12T03:17:28.845833+0000","last_peered":"2024-09-12T03:17:28.845833+0000","last_clean":"2024-09-12T03:17:28.845833+0000","last_became_active":"2024-09-12T03:17:07.981581+0000","last_became_peered":"2024-09-12T03:17:07.981581+0000","last_unstale":"2024-09-12T03:17:28.845833+0000","last_undegraded":"2024-09-12T03:17:28.845833+0000","last_fullsized":"2024-09-12T03:17:28.845833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:49.351106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920766+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:17:10.920766+0000","last_peered":"2024-09-12T03:17:10.920766+0000","last_clean":"2024-09-12T03:17:10.920766+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:17:10.920766+0000","last_undegraded":"2024-09-12T03:17:10.920766+0000","last_fullsized":"2024-09-12T03:17:10.920766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919845+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:17:10.919845+0000","last_peered":"2024-09-12T03:17:10.919845+0000","last_clean":"2024-09-12T03:17:10.919845+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:17:10.919845+0000","last_undegraded":"2024-09-12T03:17:10.919845+0000","last_fullsized":"2024-09-12T03:17:10.919845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832258+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:17:20.832258+0000","last_peered":"2024-09-12T03:17:20.832258+0000","last_clean":"2024-09-12T03:17:20.832258+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:17:20.832258+0000","last_undegraded":"2024-09-12T03:17:20.832258+0000","last_fullsized":"2024-09-12T03:17:20.832258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911998+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:17:10.911998+0000","last_peered":"2024-09-12T03:17:10.911998+0000","last_clean":"2024-09-12T03:17:10.911998+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:17:10.911998+0000","last_undegraded":"2024-09-12T03:17:10.911998+0000","last_fullsized":"2024-09-12T03:17:10.911998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:18.964618+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:17:18.964618+0000","last_peered":"2024-09-12T03:17:18.964618+0000","last_clean":"2024-09-12T03:17:18.964618+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:17:18.964618+0000","last_undegraded":"2024-09-12T03:17:18.964618+0000","last_fullsized":"2024-09-12T03:17:18.964618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915411+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:17:10.915411+0000","last_peered":"2024-09-12T03:17:10.915411+0000","last_clean":"2024-09-12T03:17:10.915411+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:17:10.915411+0000","last_undegraded":"2024-09-12T03:17:10.915411+0000","last_fullsized":"2024-09-12T03:17:10.915411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:31.833568+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:17:31.833568+0000","last_peered":"2024-09-12T03:17:31.833568+0000","last_clean":"2024-09-12T03:17:31.833568+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:17:31.833568+0000","last_undegraded":"2024-09-12T03:17:31.833568+0000","last_fullsized":"2024-09-12T03:17:31.833568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917802+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:17:10.917802+0000","last_peered":"2024-09-12T03:17:10.917802+0000","last_clean":"2024-09-12T03:17:10.917802+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:17:10.917802+0000","last_undegraded":"2024-09-12T03:17:10.917802+0000","last_fullsized":"2024-09-12T03:17:10.917802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":35,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:30.847795+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:17:30.847795+0000","last_peered":"2024-09-12T03:17:30.847795+0000","last_clean":"2024-09-12T03:17:30.847795+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:17:30.847795+0000","last_undegraded":"2024-09-12T03:17:30.847795+0000","last_fullsized":"2024-09-12T03:17:30.847795+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915132+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:17:10.915132+0000","last_peered":"2024-09-12T03:17:10.915132+0000","last_clean":"2024-09-12T03:17:10.915132+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:17:10.915132+0000","last_undegraded":"2024-09-12T03:17:10.915132+0000","last_fullsized":"2024-09-12T03:17:10.915132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917406+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:17:10.917406+0000","last_peered":"2024-09-12T03:17:10.917406+0000","last_clean":"2024-09-12T03:17:10.917406+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:17:10.917406+0000","last_undegraded":"2024-09-12T03:17:10.917406+0000","last_fullsized":"2024-09-12T03:17:10.917406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915742+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:17:10.915742+0000","last_peered":"2024-09-12T03:17:10.915742+0000","last_clean":"2024-09-12T03:17:10.915742+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:17:10.915742+0000","last_undegraded":"2024-09-12T03:17:10.915742+0000","last_fullsized":"2024-09-12T03:17:10.915742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.826570+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:17:19.826570+0000","last_peered":"2024-09-12T03:17:19.826570+0000","last_clean":"2024-09-12T03:17:19.826570+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:17:19.826570+0000","last_undegraded":"2024-09-12T03:17:19.826570+0000","last_fullsized":"2024-09-12T03:17:19.826570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916112+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:17:10.916112+0000","last_peered":"2024-09-12T03:17:10.916112+0000","last_clean":"2024-09-12T03:17:10.916112+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:17:10.916112+0000","last_undegraded":"2024-09-12T03:17:10.916112+0000","last_fullsized":"2024-09-12T03:17:10.916112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:33.917801+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:17:33.917801+0000","last_peered":"2024-09-12T03:17:33.917801+0000","last_clean":"2024-09-12T03:17:33.917801+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:17:33.917801+0000","last_undegraded":"2024-09-12T03:17:33.917801+0000","last_fullsized":"2024-09-12T03:17:33.917801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912504+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:17:10.912504+0000","last_peered":"2024-09-12T03:17:10.912504+0000","last_clean":"2024-09-12T03:17:10.912504+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:17:10.912504+0000","last_undegraded":"2024-09-12T03:17:10.912504+0000","last_fullsized":"2024-09-12T03:17:10.912504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832160+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:17:20.832160+0000","last_peered":"2024-09-12T03:17:20.832160+0000","last_clean":"2024-09-12T03:17:20.832160+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:17:20.832160+0000","last_undegraded":"2024-09-12T03:17:20.832160+0000","last_fullsized":"2024-09-12T03:17:20.832160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919501+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:17:10.919501+0000","last_peered":"2024-09-12T03:17:10.919501+0000","last_clean":"2024-09-12T03:17:10.919501+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:17:10.919501+0000","last_undegraded":"2024-09-12T03:17:10.919501+0000","last_fullsized":"2024-09-12T03:17:10.919501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:17.957416+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:17:17.957416+0000","last_peered":"2024-09-12T03:17:17.957416+0000","last_clean":"2024-09-12T03:17:17.957416+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:17:17.957416+0000","last_undegraded":"2024-09-12T03:17:17.957416+0000","last_fullsized":"2024-09-12T03:17:17.957416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915998+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:17:10.915998+0000","last_peered":"2024-09-12T03:17:10.915998+0000","last_clean":"2024-09-12T03:17:10.915998+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:17:10.915998+0000","last_undegraded":"2024-09-12T03:17:10.915998+0000","last_fullsized":"2024-09-12T03:17:10.915998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831894+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:17:20.831894+0000","last_peered":"2024-09-12T03:17:20.831894+0000","last_clean":"2024-09-12T03:17:20.831894+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:17:20.831894+0000","last_undegraded":"2024-09-12T03:17:20.831894+0000","last_fullsized":"2024-09-12T03:17:20.831894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910510+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:17:10.910510+0000","last_peered":"2024-09-12T03:17:10.910510+0000","last_clean":"2024-09-12T03:17:10.910510+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:17:10.910510+0000","last_undegraded":"2024-09-12T03:17:10.910510+0000","last_fullsized":"2024-09-12T03:17:10.910510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.953207+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:17:19.953207+0000","last_peered":"2024-09-12T03:17:19.953207+0000","last_clean":"2024-09-12T03:17:19.953207+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:17:19.953207+0000","last_undegraded":"2024-09-12T03:17:19.953207+0000","last_fullsized":"2024-09-12T03:17:19.953207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912158+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:17:10.912158+0000","last_peered":"2024-09-12T03:17:10.912158+0000","last_clean":"2024-09-12T03:17:10.912158+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:17:10.912158+0000","last_undegraded":"2024-09-12T03:17:10.912158+0000","last_fullsized":"2024-09-12T03:17:10.912158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:17.956474+0000","last_change":"2024-09-12T03:17:07.833527+0000","last_active":"2024-09-12T03:17:17.956474+0000","last_peered":"2024-09-12T03:17:17.956474+0000","last_clean":"2024-09-12T03:17:17.956474+0000","last_became_active":"2024-09-12T03:17:07.833008+0000","last_became_peered":"2024-09-12T03:17:07.833008+0000","last_unstale":"2024-09-12T03:17:17.956474+0000","last_undegraded":"2024-09-12T03:17:17.956474+0000","last_fullsized":"2024-09-12T03:17:17.956474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:48.886554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920866+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:17:10.920866+0000","last_peered":"2024-09-12T03:17:10.920866+0000","last_clean":"2024-09-12T03:17:10.920866+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:17:10.920866+0000","last_undegraded":"2024-09-12T03:17:10.920866+0000","last_fullsized":"2024-09-12T03:17:10.920866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916823+0000","last_change":"2024-09-12T03:17:09.921800+0000","last_active":"2024-09-12T03:17:10.916823+0000","last_peered":"2024-09-12T03:17:10.916823+0000","last_clean":"2024-09-12T03:17:10.916823+0000","last_became_active":"2024-09-12T03:17:09.921491+0000","last_became_peered":"2024-09-12T03:17:09.921491+0000","last_unstale":"2024-09-12T03:17:10.916823+0000","last_undegraded":"2024-09-12T03:17:10.916823+0000","last_fullsized":"2024-09-12T03:17:10.916823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:52.389297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831969+0000","last_change":"2024-09-12T03:17:07.830656+0000","last_active":"2024-09-12T03:17:20.831969+0000","last_peered":"2024-09-12T03:17:20.831969+0000","last_clean":"2024-09-12T03:17:20.831969+0000","last_became_active":"2024-09-12T03:17:07.828631+0000","last_became_peered":"2024-09-12T03:17:07.828631+0000","last_unstale":"2024-09-12T03:17:20.831969+0000","last_undegraded":"2024-09-12T03:17:20.831969+0000","last_fullsized":"2024-09-12T03:17:20.831969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:44.617551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910111+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:17:10.910111+0000","last_peered":"2024-09-12T03:17:10.910111+0000","last_clean":"2024-09-12T03:17:10.910111+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:17:10.910111+0000","last_undegraded":"2024-09-12T03:17:10.910111+0000","last_fullsized":"2024-09-12T03:17:10.910111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:21.827032+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:17:21.827032+0000","last_peered":"2024-09-12T03:17:21.827032+0000","last_clean":"2024-09-12T03:17:21.827032+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:17:21.827032+0000","last_undegraded":"2024-09-12T03:17:21.827032+0000","last_fullsized":"2024-09-12T03:17:21.827032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919966+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:17:10.919966+0000","last_peered":"2024-09-12T03:17:10.919966+0000","last_clean":"2024-09-12T03:17:10.919966+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:17:10.919966+0000","last_undegraded":"2024-09-12T03:17:10.919966+0000","last_fullsized":"2024-09-12T03:17:10.919966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.921227+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:17:10.921227+0000","last_peered":"2024-09-12T03:17:10.921227+0000","last_clean":"2024-09-12T03:17:10.921227+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:17:10.921227+0000","last_undegraded":"2024-09-12T03:17:10.921227+0000","last_fullsized":"2024-09-12T03:17:10.921227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912309+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:17:10.912309+0000","last_peered":"2024-09-12T03:17:10.912309+0000","last_clean":"2024-09-12T03:17:10.912309+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:17:10.912309+0000","last_undegraded":"2024-09-12T03:17:10.912309+0000","last_fullsized":"2024-09-12T03:17:10.912309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:33.851411+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:17:33.851411+0000","last_peered":"2024-09-12T03:17:33.851411+0000","last_clean":"2024-09-12T03:17:33.851411+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:17:33.851411+0000","last_undegraded":"2024-09-12T03:17:33.851411+0000","last_fullsized":"2024-09-12T03:17:33.851411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61577,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":175,"num_read_kb":161,"num_write":238,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":158123,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579922,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1066618,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234018,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":421135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462950,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1014048,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626476,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":420474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855409,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1013333,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018934,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":420728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247866,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444096,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":463065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":462452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":421813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":43430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":56498,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:36.463 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:17:36.724 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:37.121 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:37 smithi019 ceph-mon[30953]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-12T03:17:37.122 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:37 smithi019 ceph-mon[30953]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:37.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 -- 172.21.15.19:0/148060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 msgr2=0x7f31301067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 --2- 172.21.15.19:0/148060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f31301067e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3124009900 tx=0x7f312402a410 comp rx=0 tx=0).stop 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 -- 172.21.15.19:0/148060425 shutdown_connections 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 --2- 172.21.15.19:0/148060425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 0x7f313010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 --2- 172.21.15.19:0/148060425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f31301067e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 --2- 172.21.15.19:0/148060425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 0x7f31301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 -- 172.21.15.19:0/148060425 >> 172.21.15.19:0/148060425 conn(0x7f31300fd8c0 msgr2=0x7f31300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 -- 172.21.15.19:0/148060425 shutdown_connections 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 -- 172.21.15.19:0/148060425 wait complete. 2024-09-12T03:17:37.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.203+0000 7f3137d71640 1 Processor -- start 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 -- start start 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 0x7f3130198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 0x7f313019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3130111f70 con 0x7f3130102790 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3130111df0 con 0x7f3130106f10 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3137d71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f31301120f0 con 0x7f3130101de0 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54826/0 (socket says 172.21.15.19:54826) 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 -- 172.21.15.19:0/3490193321 learned_addr learned my addr 172.21.15.19:0/3490193321 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:37.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f3135b0d640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 0x7f3130198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313630e640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 0x7f313019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 -- 172.21.15.19:0/3490193321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 msgr2=0x7f3130198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 0x7f3130198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 -- 172.21.15.19:0/3490193321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 msgr2=0x7f313019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 0x7f313019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 -- 172.21.15.19:0/3490193321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313019d3e0 con 0x7f3130102790 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.204+0000 7f313530c640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:37.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.205+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312403f030 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.205+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3124003ce0 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.205+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313019d670 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.205+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124035070 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.205+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31301a4fb0 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.206+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31301021c0 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.207+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3124042030 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.208+0000 7f3122ffd640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 0x7f311c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.208+0000 7f3135b0d640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 0x7f311c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.208+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f31240ef5b0 con 0x7f3130102790 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.208+0000 7f3135b0d640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 0x7f311c0a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f312c005fd0 tx=0x7f312c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:37.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.212+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f31240bc620 con 0x7f3130102790 2024-09-12T03:17:37.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.384+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f313010a120 con 0x7f311c0a2c90 2024-09-12T03:17:37.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.390+0000 7f3122ffd640 1 -- 172.21.15.19:0/3490193321 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258967 (secure 0 0 0) 0x7f313010a120 con 0x7f311c0a2c90 2024-09-12T03:17:37.394 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:37.395 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:17:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 msgr2=0x7f311c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 0x7f311c0a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f312c005fd0 tx=0x7f312c00b390 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 msgr2=0x7f3130198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3124006010 tx=0x7f31240042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 shutdown_connections 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3130106f10 0x7f313019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f311c0a2c90 0x7f311c0a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3130102790 0x7f3130198970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 --2- 172.21.15.19:0/3490193321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3130101de0 0x7f3130198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 >> 172.21.15.19:0/3490193321 conn(0x7f31300fd8c0 msgr2=0x7f31300fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 shutdown_connections 2024-09-12T03:17:37.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:37.393+0000 7f3137d71640 1 -- 172.21.15.19:0/3490193321 wait complete. 2024-09-12T03:17:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:37 smithi121 ceph-mon[34963]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-12T03:17:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:37 smithi121 ceph-mon[34963]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:37 smithi135 ceph-mon[35474]: pgmap v292: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-09-12T03:17:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:37 smithi135 ceph-mon[35474]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:38.144 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-12T03:17:37.284206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652179,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":316,"num_read_kb":270,"num_write":421,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355912,"kb_used_data":10824,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585912,"statfs":{"total":1151940427776,"available":1151575973888,"internally_reserved":0,"allocated":11083776,"data_stored":6992157,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":47175,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":121,"num_read_kb":100,"num_write":34,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.018769"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910936+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:17:10.910936+0000","last_peered":"2024-09-12T03:17:10.910936+0000","last_clean":"2024-09-12T03:17:10.910936+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:17:10.910936+0000","last_undegraded":"2024-09-12T03:17:10.910936+0000","last_fullsized":"2024-09-12T03:17:10.910936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919352+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:17:10.919352+0000","last_peered":"2024-09-12T03:17:10.919352+0000","last_clean":"2024-09-12T03:17:10.919352+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:17:10.919352+0000","last_undegraded":"2024-09-12T03:17:10.919352+0000","last_fullsized":"2024-09-12T03:17:10.919352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917721+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:17:10.917721+0000","last_peered":"2024-09-12T03:17:10.917721+0000","last_clean":"2024-09-12T03:17:10.917721+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:17:10.917721+0000","last_undegraded":"2024-09-12T03:17:10.917721+0000","last_fullsized":"2024-09-12T03:17:10.917721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916429+0000","last_change":"2024-09-12T03:17:09.925311+0000","last_active":"2024-09-12T03:17:10.916429+0000","last_peered":"2024-09-12T03:17:10.916429+0000","last_clean":"2024-09-12T03:17:10.916429+0000","last_became_active":"2024-09-12T03:17:09.924761+0000","last_became_peered":"2024-09-12T03:17:09.924761+0000","last_unstale":"2024-09-12T03:17:10.916429+0000","last_undegraded":"2024-09-12T03:17:10.916429+0000","last_fullsized":"2024-09-12T03:17:10.916429+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:31:40.297483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916678+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:17:10.916678+0000","last_peered":"2024-09-12T03:17:10.916678+0000","last_clean":"2024-09-12T03:17:10.916678+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:17:10.916678+0000","last_undegraded":"2024-09-12T03:17:10.916678+0000","last_fullsized":"2024-09-12T03:17:10.916678+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.914951+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:17:10.914951+0000","last_peered":"2024-09-12T03:17:10.914951+0000","last_clean":"2024-09-12T03:17:10.914951+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:17:10.914951+0000","last_undegraded":"2024-09-12T03:17:10.914951+0000","last_fullsized":"2024-09-12T03:17:10.914951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919834+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:17:10.919834+0000","last_peered":"2024-09-12T03:17:10.919834+0000","last_clean":"2024-09-12T03:17:10.919834+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:17:10.919834+0000","last_undegraded":"2024-09-12T03:17:10.919834+0000","last_fullsized":"2024-09-12T03:17:10.919834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911699+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:17:10.911699+0000","last_peered":"2024-09-12T03:17:10.911699+0000","last_clean":"2024-09-12T03:17:10.911699+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:17:10.911699+0000","last_undegraded":"2024-09-12T03:17:10.911699+0000","last_fullsized":"2024-09-12T03:17:10.911699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910435+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:17:10.910435+0000","last_peered":"2024-09-12T03:17:10.910435+0000","last_clean":"2024-09-12T03:17:10.910435+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:17:10.910435+0000","last_undegraded":"2024-09-12T03:17:10.910435+0000","last_fullsized":"2024-09-12T03:17:10.910435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915448+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:17:10.915448+0000","last_peered":"2024-09-12T03:17:10.915448+0000","last_clean":"2024-09-12T03:17:10.915448+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:17:10.915448+0000","last_undegraded":"2024-09-12T03:17:10.915448+0000","last_fullsized":"2024-09-12T03:17:10.915448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920686+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:17:10.920686+0000","last_peered":"2024-09-12T03:17:10.920686+0000","last_clean":"2024-09-12T03:17:10.920686+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:17:10.920686+0000","last_undegraded":"2024-09-12T03:17:10.920686+0000","last_fullsized":"2024-09-12T03:17:10.920686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916574+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:17:10.916574+0000","last_peered":"2024-09-12T03:17:10.916574+0000","last_clean":"2024-09-12T03:17:10.916574+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:17:10.916574+0000","last_undegraded":"2024-09-12T03:17:10.916574+0000","last_fullsized":"2024-09-12T03:17:10.916574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915920+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:17:10.915920+0000","last_peered":"2024-09-12T03:17:10.915920+0000","last_clean":"2024-09-12T03:17:10.915920+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:17:10.915920+0000","last_undegraded":"2024-09-12T03:17:10.915920+0000","last_fullsized":"2024-09-12T03:17:10.915920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918076+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:17:10.918076+0000","last_peered":"2024-09-12T03:17:10.918076+0000","last_clean":"2024-09-12T03:17:10.918076+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:17:10.918076+0000","last_undegraded":"2024-09-12T03:17:10.918076+0000","last_fullsized":"2024-09-12T03:17:10.918076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920627+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:17:10.920627+0000","last_peered":"2024-09-12T03:17:10.920627+0000","last_clean":"2024-09-12T03:17:10.920627+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:17:10.920627+0000","last_undegraded":"2024-09-12T03:17:10.920627+0000","last_fullsized":"2024-09-12T03:17:10.920627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.909906+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:17:10.909906+0000","last_peered":"2024-09-12T03:17:10.909906+0000","last_clean":"2024-09-12T03:17:10.909906+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:17:10.909906+0000","last_undegraded":"2024-09-12T03:17:10.909906+0000","last_fullsized":"2024-09-12T03:17:10.909906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911086+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:17:10.911086+0000","last_peered":"2024-09-12T03:17:10.911086+0000","last_clean":"2024-09-12T03:17:10.911086+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:17:10.911086+0000","last_undegraded":"2024-09-12T03:17:10.911086+0000","last_fullsized":"2024-09-12T03:17:10.911086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917358+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:17:10.917358+0000","last_peered":"2024-09-12T03:17:10.917358+0000","last_clean":"2024-09-12T03:17:10.917358+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:17:10.917358+0000","last_undegraded":"2024-09-12T03:17:10.917358+0000","last_fullsized":"2024-09-12T03:17:10.917358+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916130+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:17:10.916130+0000","last_peered":"2024-09-12T03:17:10.916130+0000","last_clean":"2024-09-12T03:17:10.916130+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:17:10.916130+0000","last_undegraded":"2024-09-12T03:17:10.916130+0000","last_fullsized":"2024-09-12T03:17:10.916130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910819+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:17:10.910819+0000","last_peered":"2024-09-12T03:17:10.910819+0000","last_clean":"2024-09-12T03:17:10.910819+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:17:10.910819+0000","last_undegraded":"2024-09-12T03:17:10.910819+0000","last_fullsized":"2024-09-12T03:17:10.910819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915960+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:17:10.915960+0000","last_peered":"2024-09-12T03:17:10.915960+0000","last_clean":"2024-09-12T03:17:10.915960+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:17:10.915960+0000","last_undegraded":"2024-09-12T03:17:10.915960+0000","last_fullsized":"2024-09-12T03:17:10.915960+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910084+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:17:10.910084+0000","last_peered":"2024-09-12T03:17:10.910084+0000","last_clean":"2024-09-12T03:17:10.910084+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:17:10.910084+0000","last_undegraded":"2024-09-12T03:17:10.910084+0000","last_fullsized":"2024-09-12T03:17:10.910084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911735+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:17:10.911735+0000","last_peered":"2024-09-12T03:17:10.911735+0000","last_clean":"2024-09-12T03:17:10.911735+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:17:10.911735+0000","last_undegraded":"2024-09-12T03:17:10.911735+0000","last_fullsized":"2024-09-12T03:17:10.911735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917646+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:17:10.917646+0000","last_peered":"2024-09-12T03:17:10.917646+0000","last_clean":"2024-09-12T03:17:10.917646+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:17:10.917646+0000","last_undegraded":"2024-09-12T03:17:10.917646+0000","last_fullsized":"2024-09-12T03:17:10.917646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912354+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:17:10.912354+0000","last_peered":"2024-09-12T03:17:10.912354+0000","last_clean":"2024-09-12T03:17:10.912354+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:17:10.912354+0000","last_undegraded":"2024-09-12T03:17:10.912354+0000","last_fullsized":"2024-09-12T03:17:10.912354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916458+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:17:10.916458+0000","last_peered":"2024-09-12T03:17:10.916458+0000","last_clean":"2024-09-12T03:17:10.916458+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:17:10.916458+0000","last_undegraded":"2024-09-12T03:17:10.916458+0000","last_fullsized":"2024-09-12T03:17:10.916458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910708+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:17:10.910708+0000","last_peered":"2024-09-12T03:17:10.910708+0000","last_clean":"2024-09-12T03:17:10.910708+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:17:10.910708+0000","last_undegraded":"2024-09-12T03:17:10.910708+0000","last_fullsized":"2024-09-12T03:17:10.910708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915969+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:17:10.915969+0000","last_peered":"2024-09-12T03:17:10.915969+0000","last_clean":"2024-09-12T03:17:10.915969+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:17:10.915969+0000","last_undegraded":"2024-09-12T03:17:10.915969+0000","last_fullsized":"2024-09-12T03:17:10.915969+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917962+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:17:10.917962+0000","last_peered":"2024-09-12T03:17:10.917962+0000","last_clean":"2024-09-12T03:17:10.917962+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:17:10.917962+0000","last_undegraded":"2024-09-12T03:17:10.917962+0000","last_fullsized":"2024-09-12T03:17:10.917962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915844+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:17:10.915844+0000","last_peered":"2024-09-12T03:17:10.915844+0000","last_clean":"2024-09-12T03:17:10.915844+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:17:10.915844+0000","last_undegraded":"2024-09-12T03:17:10.915844+0000","last_fullsized":"2024-09-12T03:17:10.915844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915971+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:17:10.915971+0000","last_peered":"2024-09-12T03:17:10.915971+0000","last_clean":"2024-09-12T03:17:10.915971+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:17:10.915971+0000","last_undegraded":"2024-09-12T03:17:10.915971+0000","last_fullsized":"2024-09-12T03:17:10.915971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910616+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:17:10.910616+0000","last_peered":"2024-09-12T03:17:10.910616+0000","last_clean":"2024-09-12T03:17:10.910616+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:17:10.910616+0000","last_undegraded":"2024-09-12T03:17:10.910616+0000","last_fullsized":"2024-09-12T03:17:10.910616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918973+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:17:10.918973+0000","last_peered":"2024-09-12T03:17:10.918973+0000","last_clean":"2024-09-12T03:17:10.918973+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:17:10.918973+0000","last_undegraded":"2024-09-12T03:17:10.918973+0000","last_fullsized":"2024-09-12T03:17:10.918973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":41,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:31.834475+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:17:31.834475+0000","last_peered":"2024-09-12T03:17:31.834475+0000","last_clean":"2024-09-12T03:17:31.834475+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:17:31.834475+0000","last_undegraded":"2024-09-12T03:17:31.834475+0000","last_fullsized":"2024-09-12T03:17:31.834475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918302+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:17:10.918302+0000","last_peered":"2024-09-12T03:17:10.918302+0000","last_clean":"2024-09-12T03:17:10.918302+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:17:10.918302+0000","last_undegraded":"2024-09-12T03:17:10.918302+0000","last_fullsized":"2024-09-12T03:17:10.918302+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.825682+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:17:19.825682+0000","last_peered":"2024-09-12T03:17:19.825682+0000","last_clean":"2024-09-12T03:17:19.825682+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:17:19.825682+0000","last_undegraded":"2024-09-12T03:17:19.825682+0000","last_fullsized":"2024-09-12T03:17:19.825682+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:18.964662+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:17:18.964662+0000","last_peered":"2024-09-12T03:17:18.964662+0000","last_clean":"2024-09-12T03:17:18.964662+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:17:18.964662+0000","last_undegraded":"2024-09-12T03:17:18.964662+0000","last_fullsized":"2024-09-12T03:17:18.964662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911763+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:17:10.911763+0000","last_peered":"2024-09-12T03:17:10.911763+0000","last_clean":"2024-09-12T03:17:10.911763+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:17:10.911763+0000","last_undegraded":"2024-09-12T03:17:10.911763+0000","last_fullsized":"2024-09-12T03:17:10.911763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831556+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:17:20.831556+0000","last_peered":"2024-09-12T03:17:20.831556+0000","last_clean":"2024-09-12T03:17:20.831556+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:17:20.831556+0000","last_undegraded":"2024-09-12T03:17:20.831556+0000","last_fullsized":"2024-09-12T03:17:20.831556+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917972+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:17:10.917972+0000","last_peered":"2024-09-12T03:17:10.917972+0000","last_clean":"2024-09-12T03:17:10.917972+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:17:10.917972+0000","last_undegraded":"2024-09-12T03:17:10.917972+0000","last_fullsized":"2024-09-12T03:17:10.917972+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":52,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:34.838084+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:17:34.838084+0000","last_peered":"2024-09-12T03:17:34.838084+0000","last_clean":"2024-09-12T03:17:34.838084+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:17:34.838084+0000","last_undegraded":"2024-09-12T03:17:34.838084+0000","last_fullsized":"2024-09-12T03:17:34.838084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910386+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:17:10.910386+0000","last_peered":"2024-09-12T03:17:10.910386+0000","last_clean":"2024-09-12T03:17:10.910386+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:17:10.910386+0000","last_undegraded":"2024-09-12T03:17:10.910386+0000","last_fullsized":"2024-09-12T03:17:10.910386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830705+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:17:20.830705+0000","last_peered":"2024-09-12T03:17:20.830705+0000","last_clean":"2024-09-12T03:17:20.830705+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:17:20.830705+0000","last_undegraded":"2024-09-12T03:17:20.830705+0000","last_fullsized":"2024-09-12T03:17:20.830705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920111+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:17:10.920111+0000","last_peered":"2024-09-12T03:17:10.920111+0000","last_clean":"2024-09-12T03:17:10.920111+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:17:10.920111+0000","last_undegraded":"2024-09-12T03:17:10.920111+0000","last_fullsized":"2024-09-12T03:17:10.920111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.817101+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:17:19.817101+0000","last_peered":"2024-09-12T03:17:19.817101+0000","last_clean":"2024-09-12T03:17:19.817101+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:17:19.817101+0000","last_undegraded":"2024-09-12T03:17:19.817101+0000","last_fullsized":"2024-09-12T03:17:19.817101+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910273+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:17:10.910273+0000","last_peered":"2024-09-12T03:17:10.910273+0000","last_clean":"2024-09-12T03:17:10.910273+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:17:10.910273+0000","last_undegraded":"2024-09-12T03:17:10.910273+0000","last_fullsized":"2024-09-12T03:17:10.910273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'27","reported_seq":83,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:34.839650+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:17:34.839650+0000","last_peered":"2024-09-12T03:17:34.839650+0000","last_clean":"2024-09-12T03:17:34.839650+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:17:34.839650+0000","last_undegraded":"2024-09-12T03:17:34.839650+0000","last_fullsized":"2024-09-12T03:17:34.839650+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54369,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":38,"num_read_kb":71,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916262+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:17:10.916262+0000","last_peered":"2024-09-12T03:17:10.916262+0000","last_clean":"2024-09-12T03:17:10.916262+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:17:10.916262+0000","last_undegraded":"2024-09-12T03:17:10.916262+0000","last_fullsized":"2024-09-12T03:17:10.916262+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830417+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:17:20.830417+0000","last_peered":"2024-09-12T03:17:20.830417+0000","last_clean":"2024-09-12T03:17:20.830417+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:17:20.830417+0000","last_undegraded":"2024-09-12T03:17:20.830417+0000","last_fullsized":"2024-09-12T03:17:20.830417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915552+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:17:10.915552+0000","last_peered":"2024-09-12T03:17:10.915552+0000","last_clean":"2024-09-12T03:17:10.915552+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:17:10.915552+0000","last_undegraded":"2024-09-12T03:17:10.915552+0000","last_fullsized":"2024-09-12T03:17:10.915552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832531+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:17:20.832531+0000","last_peered":"2024-09-12T03:17:20.832531+0000","last_clean":"2024-09-12T03:17:20.832531+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:17:20.832531+0000","last_undegraded":"2024-09-12T03:17:20.832531+0000","last_fullsized":"2024-09-12T03:17:20.832531+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920453+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:17:10.920453+0000","last_peered":"2024-09-12T03:17:10.920453+0000","last_clean":"2024-09-12T03:17:10.920453+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:17:10.920453+0000","last_undegraded":"2024-09-12T03:17:10.920453+0000","last_fullsized":"2024-09-12T03:17:10.920453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":45,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:35.839646+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:17:35.839646+0000","last_peered":"2024-09-12T03:17:35.839646+0000","last_clean":"2024-09-12T03:17:35.839646+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:17:35.839646+0000","last_undegraded":"2024-09-12T03:17:35.839646+0000","last_fullsized":"2024-09-12T03:17:35.839646+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":22,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919772+0000","last_change":"2024-09-12T03:17:09.928028+0000","last_active":"2024-09-12T03:17:10.919772+0000","last_peered":"2024-09-12T03:17:10.919772+0000","last_clean":"2024-09-12T03:17:10.919772+0000","last_became_active":"2024-09-12T03:17:09.927706+0000","last_became_peered":"2024-09-12T03:17:09.927706+0000","last_unstale":"2024-09-12T03:17:10.919772+0000","last_undegraded":"2024-09-12T03:17:10.919772+0000","last_fullsized":"2024-09-12T03:17:10.919772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:39.496602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":36,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.824423+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:17:19.824423+0000","last_peered":"2024-09-12T03:17:19.824423+0000","last_clean":"2024-09-12T03:17:19.824423+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:17:19.824423+0000","last_undegraded":"2024-09-12T03:17:19.824423+0000","last_fullsized":"2024-09-12T03:17:19.824423+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916185+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:17:10.916185+0000","last_peered":"2024-09-12T03:17:10.916185+0000","last_clean":"2024-09-12T03:17:10.916185+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:17:10.916185+0000","last_undegraded":"2024-09-12T03:17:10.916185+0000","last_fullsized":"2024-09-12T03:17:10.916185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.814808+0000","last_change":"2024-09-12T03:17:07.831968+0000","last_active":"2024-09-12T03:17:19.814808+0000","last_peered":"2024-09-12T03:17:19.814808+0000","last_clean":"2024-09-12T03:17:19.814808+0000","last_became_active":"2024-09-12T03:17:07.828479+0000","last_became_peered":"2024-09-12T03:17:07.828479+0000","last_unstale":"2024-09-12T03:17:19.814808+0000","last_undegraded":"2024-09-12T03:17:19.814808+0000","last_fullsized":"2024-09-12T03:17:19.814808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:55.104152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910685+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:17:10.910685+0000","last_peered":"2024-09-12T03:17:10.910685+0000","last_clean":"2024-09-12T03:17:10.910685+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:17:10.910685+0000","last_undegraded":"2024-09-12T03:17:10.910685+0000","last_fullsized":"2024-09-12T03:17:10.910685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.829356+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:17:20.829356+0000","last_peered":"2024-09-12T03:17:20.829356+0000","last_clean":"2024-09-12T03:17:20.829356+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:17:20.829356+0000","last_undegraded":"2024-09-12T03:17:20.829356+0000","last_fullsized":"2024-09-12T03:17:20.829356+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920448+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:17:10.920448+0000","last_peered":"2024-09-12T03:17:10.920448+0000","last_clean":"2024-09-12T03:17:10.920448+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:17:10.920448+0000","last_undegraded":"2024-09-12T03:17:10.920448+0000","last_fullsized":"2024-09-12T03:17:10.920448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.825142+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:17:19.825142+0000","last_peered":"2024-09-12T03:17:19.825142+0000","last_clean":"2024-09-12T03:17:19.825142+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:17:19.825142+0000","last_undegraded":"2024-09-12T03:17:19.825142+0000","last_fullsized":"2024-09-12T03:17:19.825142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.130685+0000","last_change":"2024-09-12T03:17:09.925753+0000","last_active":"2024-09-12T03:17:12.130685+0000","last_peered":"2024-09-12T03:17:12.130685+0000","last_clean":"2024-09-12T03:17:12.130685+0000","last_became_active":"2024-09-12T03:17:09.925426+0000","last_became_peered":"2024-09-12T03:17:09.925426+0000","last_unstale":"2024-09-12T03:17:12.130685+0000","last_undegraded":"2024-09-12T03:17:12.130685+0000","last_fullsized":"2024-09-12T03:17:12.130685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:13.548318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":39,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:33.852953+0000","last_change":"2024-09-12T03:17:07.981943+0000","last_active":"2024-09-12T03:17:33.852953+0000","last_peered":"2024-09-12T03:17:33.852953+0000","last_clean":"2024-09-12T03:17:33.852953+0000","last_became_active":"2024-09-12T03:17:07.981581+0000","last_became_peered":"2024-09-12T03:17:07.981581+0000","last_unstale":"2024-09-12T03:17:33.852953+0000","last_undegraded":"2024-09-12T03:17:33.852953+0000","last_fullsized":"2024-09-12T03:17:33.852953+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:49.351106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920766+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:17:10.920766+0000","last_peered":"2024-09-12T03:17:10.920766+0000","last_clean":"2024-09-12T03:17:10.920766+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:17:10.920766+0000","last_undegraded":"2024-09-12T03:17:10.920766+0000","last_fullsized":"2024-09-12T03:17:10.920766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919845+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:17:10.919845+0000","last_peered":"2024-09-12T03:17:10.919845+0000","last_clean":"2024-09-12T03:17:10.919845+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:17:10.919845+0000","last_undegraded":"2024-09-12T03:17:10.919845+0000","last_fullsized":"2024-09-12T03:17:10.919845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832258+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:17:20.832258+0000","last_peered":"2024-09-12T03:17:20.832258+0000","last_clean":"2024-09-12T03:17:20.832258+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:17:20.832258+0000","last_undegraded":"2024-09-12T03:17:20.832258+0000","last_fullsized":"2024-09-12T03:17:20.832258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911998+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:17:10.911998+0000","last_peered":"2024-09-12T03:17:10.911998+0000","last_clean":"2024-09-12T03:17:10.911998+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:17:10.911998+0000","last_undegraded":"2024-09-12T03:17:10.911998+0000","last_fullsized":"2024-09-12T03:17:10.911998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:18.964618+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:17:18.964618+0000","last_peered":"2024-09-12T03:17:18.964618+0000","last_clean":"2024-09-12T03:17:18.964618+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:17:18.964618+0000","last_undegraded":"2024-09-12T03:17:18.964618+0000","last_fullsized":"2024-09-12T03:17:18.964618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915411+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:17:10.915411+0000","last_peered":"2024-09-12T03:17:10.915411+0000","last_clean":"2024-09-12T03:17:10.915411+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:17:10.915411+0000","last_undegraded":"2024-09-12T03:17:10.915411+0000","last_fullsized":"2024-09-12T03:17:10.915411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":38,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:31.833568+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:17:31.833568+0000","last_peered":"2024-09-12T03:17:31.833568+0000","last_clean":"2024-09-12T03:17:31.833568+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:17:31.833568+0000","last_undegraded":"2024-09-12T03:17:31.833568+0000","last_fullsized":"2024-09-12T03:17:31.833568+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917802+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:17:10.917802+0000","last_peered":"2024-09-12T03:17:10.917802+0000","last_clean":"2024-09-12T03:17:10.917802+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:17:10.917802+0000","last_undegraded":"2024-09-12T03:17:10.917802+0000","last_fullsized":"2024-09-12T03:17:10.917802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":40,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:35.854576+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:17:35.854576+0000","last_peered":"2024-09-12T03:17:35.854576+0000","last_clean":"2024-09-12T03:17:35.854576+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:17:35.854576+0000","last_undegraded":"2024-09-12T03:17:35.854576+0000","last_fullsized":"2024-09-12T03:17:35.854576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915132+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:17:10.915132+0000","last_peered":"2024-09-12T03:17:10.915132+0000","last_clean":"2024-09-12T03:17:10.915132+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:17:10.915132+0000","last_undegraded":"2024-09-12T03:17:10.915132+0000","last_fullsized":"2024-09-12T03:17:10.915132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":19,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917406+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:17:10.917406+0000","last_peered":"2024-09-12T03:17:10.917406+0000","last_clean":"2024-09-12T03:17:10.917406+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:17:10.917406+0000","last_undegraded":"2024-09-12T03:17:10.917406+0000","last_fullsized":"2024-09-12T03:17:10.917406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915742+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:17:10.915742+0000","last_peered":"2024-09-12T03:17:10.915742+0000","last_clean":"2024-09-12T03:17:10.915742+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:17:10.915742+0000","last_undegraded":"2024-09-12T03:17:10.915742+0000","last_fullsized":"2024-09-12T03:17:10.915742+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.826570+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:17:19.826570+0000","last_peered":"2024-09-12T03:17:19.826570+0000","last_clean":"2024-09-12T03:17:19.826570+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:17:19.826570+0000","last_undegraded":"2024-09-12T03:17:19.826570+0000","last_fullsized":"2024-09-12T03:17:19.826570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916112+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:17:10.916112+0000","last_peered":"2024-09-12T03:17:10.916112+0000","last_clean":"2024-09-12T03:17:10.916112+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:17:10.916112+0000","last_undegraded":"2024-09-12T03:17:10.916112+0000","last_fullsized":"2024-09-12T03:17:10.916112+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'14","reported_seq":32,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:33.917801+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:17:33.917801+0000","last_peered":"2024-09-12T03:17:33.917801+0000","last_clean":"2024-09-12T03:17:33.917801+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:17:33.917801+0000","last_undegraded":"2024-09-12T03:17:33.917801+0000","last_fullsized":"2024-09-12T03:17:33.917801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912504+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:17:10.912504+0000","last_peered":"2024-09-12T03:17:10.912504+0000","last_clean":"2024-09-12T03:17:10.912504+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:17:10.912504+0000","last_undegraded":"2024-09-12T03:17:10.912504+0000","last_fullsized":"2024-09-12T03:17:10.912504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832160+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:17:20.832160+0000","last_peered":"2024-09-12T03:17:20.832160+0000","last_clean":"2024-09-12T03:17:20.832160+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:17:20.832160+0000","last_undegraded":"2024-09-12T03:17:20.832160+0000","last_fullsized":"2024-09-12T03:17:20.832160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919501+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:17:10.919501+0000","last_peered":"2024-09-12T03:17:10.919501+0000","last_clean":"2024-09-12T03:17:10.919501+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:17:10.919501+0000","last_undegraded":"2024-09-12T03:17:10.919501+0000","last_fullsized":"2024-09-12T03:17:10.919501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:17.957416+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:17:17.957416+0000","last_peered":"2024-09-12T03:17:17.957416+0000","last_clean":"2024-09-12T03:17:17.957416+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:17:17.957416+0000","last_undegraded":"2024-09-12T03:17:17.957416+0000","last_fullsized":"2024-09-12T03:17:17.957416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915998+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:17:10.915998+0000","last_peered":"2024-09-12T03:17:10.915998+0000","last_clean":"2024-09-12T03:17:10.915998+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:17:10.915998+0000","last_undegraded":"2024-09-12T03:17:10.915998+0000","last_fullsized":"2024-09-12T03:17:10.915998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":25,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831894+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:17:20.831894+0000","last_peered":"2024-09-12T03:17:20.831894+0000","last_clean":"2024-09-12T03:17:20.831894+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:17:20.831894+0000","last_undegraded":"2024-09-12T03:17:20.831894+0000","last_fullsized":"2024-09-12T03:17:20.831894+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910510+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:17:10.910510+0000","last_peered":"2024-09-12T03:17:10.910510+0000","last_clean":"2024-09-12T03:17:10.910510+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:17:10.910510+0000","last_undegraded":"2024-09-12T03:17:10.910510+0000","last_fullsized":"2024-09-12T03:17:10.910510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.953207+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:17:19.953207+0000","last_peered":"2024-09-12T03:17:19.953207+0000","last_clean":"2024-09-12T03:17:19.953207+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:17:19.953207+0000","last_undegraded":"2024-09-12T03:17:19.953207+0000","last_fullsized":"2024-09-12T03:17:19.953207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912158+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:17:10.912158+0000","last_peered":"2024-09-12T03:17:10.912158+0000","last_clean":"2024-09-12T03:17:10.912158+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:17:10.912158+0000","last_undegraded":"2024-09-12T03:17:10.912158+0000","last_fullsized":"2024-09-12T03:17:10.912158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":23,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:17.956474+0000","last_change":"2024-09-12T03:17:07.833527+0000","last_active":"2024-09-12T03:17:17.956474+0000","last_peered":"2024-09-12T03:17:17.956474+0000","last_clean":"2024-09-12T03:17:17.956474+0000","last_became_active":"2024-09-12T03:17:07.833008+0000","last_became_peered":"2024-09-12T03:17:07.833008+0000","last_unstale":"2024-09-12T03:17:17.956474+0000","last_undegraded":"2024-09-12T03:17:17.956474+0000","last_fullsized":"2024-09-12T03:17:17.956474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:48.886554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920866+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:17:10.920866+0000","last_peered":"2024-09-12T03:17:10.920866+0000","last_clean":"2024-09-12T03:17:10.920866+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:17:10.920866+0000","last_undegraded":"2024-09-12T03:17:10.920866+0000","last_fullsized":"2024-09-12T03:17:10.920866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916823+0000","last_change":"2024-09-12T03:17:09.921800+0000","last_active":"2024-09-12T03:17:10.916823+0000","last_peered":"2024-09-12T03:17:10.916823+0000","last_clean":"2024-09-12T03:17:10.916823+0000","last_became_active":"2024-09-12T03:17:09.921491+0000","last_became_peered":"2024-09-12T03:17:09.921491+0000","last_unstale":"2024-09-12T03:17:10.916823+0000","last_undegraded":"2024-09-12T03:17:10.916823+0000","last_fullsized":"2024-09-12T03:17:10.916823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:51:52.389297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831969+0000","last_change":"2024-09-12T03:17:07.830656+0000","last_active":"2024-09-12T03:17:20.831969+0000","last_peered":"2024-09-12T03:17:20.831969+0000","last_clean":"2024-09-12T03:17:20.831969+0000","last_became_active":"2024-09-12T03:17:07.828631+0000","last_became_peered":"2024-09-12T03:17:07.828631+0000","last_unstale":"2024-09-12T03:17:20.831969+0000","last_undegraded":"2024-09-12T03:17:20.831969+0000","last_fullsized":"2024-09-12T03:17:20.831969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:44.617551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910111+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:17:10.910111+0000","last_peered":"2024-09-12T03:17:10.910111+0000","last_clean":"2024-09-12T03:17:10.910111+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:17:10.910111+0000","last_undegraded":"2024-09-12T03:17:10.910111+0000","last_fullsized":"2024-09-12T03:17:10.910111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":31,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:21.827032+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:17:21.827032+0000","last_peered":"2024-09-12T03:17:21.827032+0000","last_clean":"2024-09-12T03:17:21.827032+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:17:21.827032+0000","last_undegraded":"2024-09-12T03:17:21.827032+0000","last_fullsized":"2024-09-12T03:17:21.827032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919966+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:17:10.919966+0000","last_peered":"2024-09-12T03:17:10.919966+0000","last_clean":"2024-09-12T03:17:10.919966+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:17:10.919966+0000","last_undegraded":"2024-09-12T03:17:10.919966+0000","last_fullsized":"2024-09-12T03:17:10.919966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.921227+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:17:10.921227+0000","last_peered":"2024-09-12T03:17:10.921227+0000","last_clean":"2024-09-12T03:17:10.921227+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:17:10.921227+0000","last_undegraded":"2024-09-12T03:17:10.921227+0000","last_fullsized":"2024-09-12T03:17:10.921227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.912309+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:17:10.912309+0000","last_peered":"2024-09-12T03:17:10.912309+0000","last_clean":"2024-09-12T03:17:10.912309+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:17:10.912309+0000","last_undegraded":"2024-09-12T03:17:10.912309+0000","last_fullsized":"2024-09-12T03:17:10.912309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":46,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:33.851411+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:17:33.851411+0000","last_peered":"2024-09-12T03:17:33.851411+0000","last_clean":"2024-09-12T03:17:33.851411+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:17:33.851411+0000","last_undegraded":"2024-09-12T03:17:33.851411+0000","last_fullsized":"2024-09-12T03:17:33.851411+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61577,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":190,"num_read_kb":161,"num_write":238,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":184731,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579923,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1066618,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234018,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32060,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713092,"statfs":{"total":95995035648,"available":95962206208,"internally_reserved":0,"allocated":897024,"data_stored":421135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462951,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1014048,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626476,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":420474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855409,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1013333,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018935,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":420728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247866,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":420299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444097,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":476369,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":937984,"data_stored":475756,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738443,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":421813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":56121,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":56734,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":56498,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:17:38.148 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:17:38.149 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T03:17:38.150 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T03:17:38.411 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:38.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 -- 172.21.15.19:0/3532321007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa01026d0 msgr2=0x7f6fa0108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/3532321007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa01026d0 0x7f6fa0108f60 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009ca0 tx=0x7f6f9802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 -- 172.21.15.19:0/3532321007 shutdown_connections 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/3532321007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa01026d0 0x7f6fa0108f60 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/3532321007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa0101d10 0x7f6fa0102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/3532321007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0100b10 0x7f6fa0100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 -- 172.21.15.19:0/3532321007 >> 172.21.15.19:0/3532321007 conn(0x7f6fa00fc2a0 msgr2=0x7f6fa00fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 -- 172.21.15.19:0/3532321007 shutdown_connections 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.880+0000 7f6fa4d01640 1 -- 172.21.15.19:0/3532321007 wait complete. 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 Processor -- start 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 -- start start 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 0x7f6fa01972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 0x7f6fa019e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010c7f0 con 0x7f6fa0100b10 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010c670 con 0x7f6fa0101d10 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6fa4d01640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa010c970 con 0x7f6fa01026d0 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6f9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6f9e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54844/0 (socket says 172.21.15.19:54844) 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.881+0000 7f6f9e59c640 1 -- 172.21.15.19:0/2317797529 learned_addr learned my addr 172.21.15.19:0/2317797529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9dd9b640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 0x7f6fa01972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9ed9d640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 0x7f6fa019e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 -- 172.21.15.19:0/2317797529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 msgr2=0x7f6fa019e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 0x7f6fa019e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 -- 172.21.15.19:0/2317797529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 msgr2=0x7f6fa01972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 0x7f6fa01972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:38.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 -- 172.21.15.19:0/2317797529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa019e990 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9dd9b640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 0x7f6fa01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9ed9d640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 0x7f6fa019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f9e59c640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c970 tx=0x7f6f9000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f90007c10 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f90007db0 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa019ec80 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9000f940 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.882+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa019f1c0 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.884+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa0107ec0 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.884+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6f9000fae0 con 0x7f6fa0100b10 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.885+0000 7f6f8f7fe640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 0x7f6f880a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:38.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.886+0000 7f6f9dd9b640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 0x7f6f880a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:38.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.886+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6f900cf3d0 con 0x7f6fa0100b10 2024-09-12T03:17:38.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.886+0000 7f6f9dd9b640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 0x7f6f880a5180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0198280 tx=0x7f6f94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:38.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:38.890+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f9009c410 con 0x7f6fa0100b10 2024-09-12T03:17:39.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.137+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f6fa002d9b0 con 0x7f6fa0100b10 2024-09-12T03:17:39.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.141+0000 7f6f8f7fe640 1 -- 172.21.15.19:0/2317797529 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f6f9009bd10 con 0x7f6fa0100b10 2024-09-12T03:17:39.141 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-09-12T03:17:39.141 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQBTXeJmualECBAAJNTWEE5huguCOlreh5NyUQ== 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.144+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 msgr2=0x7f6f880a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.144+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 0x7f6f880a5180 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0198280 tx=0x7f6f94008040 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.144+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 msgr2=0x7f6fa0196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.144+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6f9000c970 tx=0x7f6f9000ce30 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 shutdown_connections 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6fa01026d0 0x7f6fa019e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6fa0101d10 0x7f6fa01972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6f880a2cc0 0x7f6f880a5180 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 --2- 172.21.15.19:0/2317797529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6fa0100b10 0x7f6fa0196d60 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 >> 172.21.15.19:0/2317797529 conn(0x7f6fa00fc2a0 msgr2=0x7f6fa0105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:39.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 shutdown_connections 2024-09-12T03:17:39.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:39.145+0000 7f6fa4d01640 1 -- 172.21.15.19:0/2317797529 wait complete. 2024-09-12T03:17:39.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:39 smithi019 ceph-mon[30953]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-12T03:17:39.332 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:39 smithi019 ceph-mon[30953]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:39 smithi121 ceph-mon[34963]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-12T03:17:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:39 smithi121 ceph-mon[34963]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:39 smithi135 ceph-mon[35474]: pgmap v293: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-09-12T03:17:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:39 smithi135 ceph-mon[35474]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:17:39.985 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:17:39.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T03:17:39.986 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T03:17:40.024 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T03:17:40.024 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T03:17:40.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T03:17:40.324 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:40.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:40 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:17:40.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:40 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:17:40.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:40 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:17:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:40 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:17:40.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:40 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T03:17:40.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:40 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2317797529' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T03:17:40.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 -- 172.21.15.19:0/4061209365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8106f10 msgr2=0x7fdbe810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 --2- 172.21.15.19:0/4061209365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8106f10 0x7fdbe810ea90 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009ca0 tx=0x7fdbe002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 -- 172.21.15.19:0/4061209365 shutdown_connections 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 --2- 172.21.15.19:0/4061209365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8106f10 0x7fdbe810ea90 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 --2- 172.21.15.19:0/4061209365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8102790 0x7fdbe81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 --2- 172.21.15.19:0/4061209365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 0x7fdbe81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 -- 172.21.15.19:0/4061209365 >> 172.21.15.19:0/4061209365 conn(0x7fdbe80fd8c0 msgr2=0x7fdbe80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 -- 172.21.15.19:0/4061209365 shutdown_connections 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 -- 172.21.15.19:0/4061209365 wait complete. 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.812+0000 7fdbed22c640 1 Processor -- start 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 -- start start 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 0x7fdbe81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8106f10 0x7fdbe819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8111f70 con 0x7fdbe8102790 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe8111df0 con 0x7fdbe8106f10 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbed22c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe81120f0 con 0x7fdbe8101de0 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53876/0 (socket says 172.21.15.19:53876) 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 -- 172.21.15.19:0/972969101 learned_addr learned my addr 172.21.15.19:0/972969101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe759e640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8106f10 0x7fdbe819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe6d9d640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 0x7fdbe81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 -- 172.21.15.19:0/972969101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 msgr2=0x7fdbe81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 0x7fdbe81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 -- 172.21.15.19:0/972969101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8106f10 msgr2=0x7fdbe819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8106f10 0x7fdbe819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.813+0000 7fdbe659c640 1 -- 172.21.15.19:0/972969101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe819d380 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbe659c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc017070 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe819d670 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc009c30 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00f6b0 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.814+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe81a4f50 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.815+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbe81021c0 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.816+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fdbdc0109d0 con 0x7fdbe8102790 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.817+0000 7fdbcffff640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 0x7fdbd00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.817+0000 7fdbe6d9d640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 0x7fdbd00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:40.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.817+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fdbdc0cebe0 con 0x7fdbe8102790 2024-09-12T03:17:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.817+0000 7fdbe6d9d640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 0x7fdbd00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005fd0 tx=0x7fdbd8005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:40.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:40.820+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdbdc009de0 con 0x7fdbe8102790 2024-09-12T03:17:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.043+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fdbe8199730 con 0x7fdbe8102790 2024-09-12T03:17:41.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.048+0000 7fdbcffff640 1 -- 172.21.15.19:0/972969101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263201 (secure 0 0 0) 0x7fdbe8199730 con 0x7fdbe8102790 2024-09-12T03:17:41.052 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:41.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 msgr2=0x7fdbd00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 0x7fdbd00a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8005fd0 tx=0x7fdbd8005e20 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 msgr2=0x7fdbe8198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00d950 tx=0x7fdbdc00de10 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 shutdown_connections 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe8106f10 0x7fdbe819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbd00a2c10 0x7fdbd00a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe8102790 0x7fdbe8198910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 --2- 172.21.15.19:0/972969101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe8101de0 0x7fdbe81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 >> 172.21.15.19:0/972969101 conn(0x7fdbe80fd8c0 msgr2=0x7fdbe80fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 shutdown_connections 2024-09-12T03:17:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:41.051+0000 7fdbed22c640 1 -- 172.21.15.19:0/972969101 wait complete. 2024-09-12T03:17:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:41 smithi121 ceph-mon[34963]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T03:17:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:41 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/972969101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:17:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:41 smithi019 ceph-mon[30953]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T03:17:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:41 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/972969101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:17:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:41 smithi135 ceph-mon[35474]: pgmap v294: 97 pgs: 97 active+clean; 637 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.6 KiB/s wr, 11 op/s 2024-09-12T03:17:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:41 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/972969101' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T03:17:41.830 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":2264764467},{"type":"v1","addr":"172.21.15.19:6801","nonce":2264764467}]},"active_addr":"172.21.15.19:6801/2264764467","active_change":"2024-09-12T03:10:11.180177+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.19:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1274098548}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":1299949405}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":4125437594}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":3318220758}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.19:0","nonce":2807082656}]}]} 2024-09-12T03:17:41.836 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T03:17:41.836 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T03:17:41.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:42.103 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 -- 172.21.15.19:0/1846869511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230100a80 msgr2=0x7f2230100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 --2- 172.21.15.19:0/1846869511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230100a80 0x7f2230100ee0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2220009930 tx=0x7f222002a420 comp rx=0 tx=0).stop 2024-09-12T03:17:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 -- 172.21.15.19:0/1846869511 shutdown_connections 2024-09-12T03:17:42.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 --2- 172.21.15.19:0/1846869511 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230101420 0x7f223010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 --2- 172.21.15.19:0/1846869511 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230100a80 0x7f2230100ee0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 --2- 172.21.15.19:0/1846869511 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230106a80 0x7f2230106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 -- 172.21.15.19:0/1846869511 >> 172.21.15.19:0/1846869511 conn(0x7f22300fc7a0 msgr2=0x7f22300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 -- 172.21.15.19:0/1846869511 shutdown_connections 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.600+0000 7f223854a640 1 -- 172.21.15.19:0/1846869511 wait complete. 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 Processor -- start 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 -- start start 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 0x7f2230102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 0x7f22301030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2230110e80 con 0x7f2230101420 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2230110d00 con 0x7f2230106a80 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f223854a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2230111000 con 0x7f2230100a80 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f2235ae5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f2235ae5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53888/0 (socket says 172.21.15.19:53888) 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f2235ae5640 1 -- 172.21.15.19:0/762969251 learned_addr learned my addr 172.21.15.19:0/762969251 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f22362e6640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 0x7f2230102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.601+0000 7f2236ae7640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 0x7f22301030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 -- 172.21.15.19:0/762969251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 msgr2=0x7f2230102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 0x7f2230102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 -- 172.21.15.19:0/762969251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 msgr2=0x7f22301030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 0x7f22301030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 -- 172.21.15.19:0/762969251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22301a6910 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2236ae7640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 0x7f22301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f22362e6640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 0x7f2230102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f2235ae5640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2220009230 tx=0x7f2220004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222003f030 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2220003ad0 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220035070 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22301a6ba0 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.602+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22301a7080 con 0x7f2230101420 2024-09-12T03:17:42.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.603+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2230100ee0 con 0x7f2230101420 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.604+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2220042030 con 0x7f2230101420 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.605+0000 7f22277fe640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 0x7f221c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.605+0000 7f22362e6640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 0x7f221c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.605+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f22200ef360 con 0x7f2230101420 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.606+0000 7f22362e6640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 0x7f221c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f222c005fd0 tx=0x7f222c00b5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:42.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.609+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f22200bc210 con 0x7f2230101420 2024-09-12T03:17:42.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.785+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22301a7250 con 0x7f2230101420 2024-09-12T03:17:42.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.787+0000 7f22277fe640 1 -- 172.21.15.19:0/762969251 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22303 (secure 0 0 0) 0x7f22200bbb10 con 0x7f2230101420 2024-09-12T03:17:42.787 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:42.787 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:17:10.527091+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 msgr2=0x7f221c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 0x7f221c0a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f222c005fd0 tx=0x7f222c00b5d0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 msgr2=0x7f2230102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2220009230 tx=0x7f2220004270 comp rx=0 tx=0).stop 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 shutdown_connections 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2230106a80 0x7f22301030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f221c0a2c10 0x7f221c0a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2230101420 0x7f2230102b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 --2- 172.21.15.19:0/762969251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2230100a80 0x7f2230102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 >> 172.21.15.19:0/762969251 conn(0x7f22300fc7a0 msgr2=0x7f2230104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 shutdown_connections 2024-09-12T03:17:42.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:42.790+0000 7f223854a640 1 -- 172.21.15.19:0/762969251 wait complete. 2024-09-12T03:17:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:43 smithi121 ceph-mon[34963]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-12T03:17:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:43 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/762969251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:43 smithi019 ceph-mon[30953]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-12T03:17:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:43 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/762969251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:43 smithi135 ceph-mon[35474]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.9 KiB/s wr, 13 op/s 2024-09-12T03:17:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:43 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/762969251' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:43.637 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T03:17:43.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:17:43.898 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:44.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 -- 172.21.15.19:0/2321267462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 msgr2=0x7ff86c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 --2- 172.21.15.19:0/2321267462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c1021c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff868009900 tx=0x7ff86802a410 comp rx=0 tx=0).stop 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 -- 172.21.15.19:0/2321267462 shutdown_connections 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 --2- 172.21.15.19:0/2321267462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 --2- 172.21.15.19:0/2321267462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 --2- 172.21.15.19:0/2321267462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c1021c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 -- 172.21.15.19:0/2321267462 >> 172.21.15.19:0/2321267462 conn(0x7ff86c0fd8c0 msgr2=0x7ff86c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 -- 172.21.15.19:0/2321267462 shutdown_connections 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.368+0000 7ff873f14640 1 -- 172.21.15.19:0/2321267462 wait complete. 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 Processor -- start 2024-09-12T03:17:44.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 -- start start 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c111f30 con 0x7ff86c101de0 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c111db0 con 0x7ff86c106f10 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff873f14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1120b0 con 0x7ff86c102790 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53908/0 (socket says 172.21.15.19:53908) 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 -- 172.21.15.19:0/458914057 learned_addr learned my addr 172.21.15.19:0/458914057 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff8724b1640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff8714af640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 -- 172.21.15.19:0/458914057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 msgr2=0x7ff86c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 -- 172.21.15.19:0/458914057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 msgr2=0x7ff86c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff871cb0640 1 -- 172.21.15.19:0/458914057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c19d3e0 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff8724b1640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.369+0000 7ff8714af640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff871cb0640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff868005ab0 tx=0x7ff8680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86803f030 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c19d670 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff868003ce0 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff868035070 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.370+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c1a4fb0 con 0x7ff86c101de0 2024-09-12T03:17:44.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.371+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff86c1032c0 con 0x7ff86c101de0 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.372+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff868042030 con 0x7ff86c101de0 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.373+0000 7ff862ffd640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 0x7ff8580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.373+0000 7ff8714af640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 0x7ff8580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.373+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff8680efd40 con 0x7ff86c101de0 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.373+0000 7ff8714af640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 0x7ff8580a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0fdd00 tx=0x7ff85c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:44.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.376+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8680bcbf0 con 0x7ff86c101de0 2024-09-12T03:17:44.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.553+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff86c1021c0 con 0x7ff86c101de0 2024-09-12T03:17:44.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.555+0000 7ff862ffd640 1 -- 172.21.15.19:0/458914057 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22303 (secure 0 0 0) 0x7ff8680bc4f0 con 0x7ff86c101de0 2024-09-12T03:17:44.555 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:17:44.556 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":80,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:17:10.527091+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 msgr2=0x7ff8580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 0x7ff8580a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff86c0fdd00 tx=0x7ff85c00a5c0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 msgr2=0x7ff86c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff868005ab0 tx=0x7ff8680042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 shutdown_connections 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff86c106f10 0x7ff86c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff86c102790 0x7ff86c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8580a2c10 0x7ff8580a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 --2- 172.21.15.19:0/458914057 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff86c101de0 0x7ff86c198430 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 >> 172.21.15.19:0/458914057 conn(0x7ff86c0fd8c0 msgr2=0x7ff86c0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 shutdown_connections 2024-09-12T03:17:44.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:44.558+0000 7ff873f14640 1 -- 172.21.15.19:0/458914057 wait complete. 2024-09-12T03:17:45.350 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:45 smithi019 ceph-mon[30953]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T03:17:45.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:45 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/458914057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:45.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T03:17:45.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T03:17:45.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T03:17:45.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T03:17:45.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T03:17:45.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T03:17:45.387 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T03:17:45.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T03:17:45.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T03:17:45.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T03:17:45.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T03:17:45.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T03:17:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:45 smithi135 ceph-mon[35474]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T03:17:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:45 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/458914057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:45 smithi121 ceph-mon[34963]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-09-12T03:17:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:45 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/458914057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:17:46.133 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.139 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.140 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.185 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.186 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.205 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.213 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:46.216 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:47 smithi019 ceph-mon[30953]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T03:17:47.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 -- 172.21.15.19:0/3802159283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 msgr2=0x7f16f41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/3802159283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f41029b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009900 tx=0x7f16e402a480 comp rx=0 tx=0).stop 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 -- 172.21.15.19:0/3802159283 shutdown_connections 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/3802159283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 0x7f16f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/3802159283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f41029b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/3802159283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f4100e20 0x7f16f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 -- 172.21.15.19:0/3802159283 >> 172.21.15.19:0/3802159283 conn(0x7f16f40fc7d0 msgr2=0x7f16f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 -- 172.21.15.19:0/3802159283 shutdown_connections 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.456+0000 7f16fb5e9640 1 -- 172.21.15.19:0/3802159283 wait complete. 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 Processor -- start 2024-09-12T03:17:47.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 -- start start 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f4100e20 0x7f16f40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 0x7f16f40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4110f80 con 0x7f16f4102570 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4110e00 con 0x7f16f4100e20 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16fb5e9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16f4111100 con 0x7f16f4102ef0 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f40fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53920/0 (socket says 172.21.15.19:53920) 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 -- 172.21.15.19:0/1691489 learned_addr learned my addr 172.21.15.19:0/1691489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f9385640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f4100e20 0x7f16f40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f9b86640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 0x7f16f40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 -- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 msgr2=0x7f16f40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 0x7f16f40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 -- 172.21.15.19:0/1691489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f4100e20 msgr2=0x7f16f40fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16f4100e20 0x7f16f40fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f8b84640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16f41a69a0 con 0x7f16f4102570 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.457+0000 7f16f9b86640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f16f4102ef0 0x7f16f40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16f8b84640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f40fd1c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e403f070 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16e4031c30 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16f41a6c30 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e4035030 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.458+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16f41a7160 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.459+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f16f40628c0 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.460+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f16e4042070 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.460+0000 7f16ea7fc640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16e00a2c90 0x7f16e00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16f9385640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16e00a2c90 0x7f16e00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f16e40f0230 con 0x7f16f4102570 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16ea7fc640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] conn(0x7f16e00ac4e0 0x7f16e00ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f16e4042e50 con 0x7f16e00ac4e0 2024-09-12T03:17:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f16e40f05e0 con 0x7f16f4102570 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16f9385640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16e00a2c90 0x7f16e00a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005fd0 tx=0x7f16f0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.461+0000 7f16f9b86640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] conn(0x7f16e00ac4e0 0x7f16e00ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.462+0000 7f16f9b86640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] conn(0x7f16e00ac4e0 0x7f16e00ae900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.462+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== osd.5 v2:172.21.15.135:6808/3468418134 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f16e4042e50 con 0x7f16e00ac4e0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.476+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 --> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f16f4062cc0 con 0x7f16e00ac4e0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16ea7fc640 1 -- 172.21.15.19:0/1691489 <== osd.5 v2:172.21.15.135:6808/3468418134 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f16f4062cc0 con 0x7f16e00ac4e0 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] conn(0x7f16e00ac4e0 msgr2=0x7f16e00ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] conn(0x7f16e00ac4e0 0x7f16e00ae900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16e00a2c90 msgr2=0x7f16e00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16e00a2c90 0x7f16e00a5150 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f16f0005fd0 tx=0x7f16f0005850 comp rx=0 tx=0).stop 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 msgr2=0x7f16f40fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 --2- 172.21.15.19:0/1691489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f16f4102570 0x7f16f40fd1c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16e4006010 tx=0x7f16e4003c10 comp rx=0 tx=0).stop 2024-09-12T03:17:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16f9b86640 1 -- 172.21.15.19:0/1691489 reap_dead start 2024-09-12T03:17:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 shutdown_connections 2024-09-12T03:17:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 >> 172.21.15.19:0/1691489 conn(0x7f16f40fc7d0 msgr2=0x7f16f406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 shutdown_connections 2024-09-12T03:17:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:47.477+0000 7f16fb5e9640 1 -- 172.21.15.19:0/1691489 wait complete. 2024-09-12T03:17:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:47 smithi135 ceph-mon[35474]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T03:17:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:47 smithi121 ceph-mon[34963]: pgmap v297: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 -- 172.21.15.19:0/2214909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae681002f0 msgr2=0x7fae681006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/2214909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae681002f0 0x7fae681006d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fae58009cd0 tx=0x7fae5802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 -- 172.21.15.19:0/2214909810 shutdown_connections 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/2214909810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 0x7fae6810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/2214909810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae68100ca0 0x7fae68106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/2214909810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae681002f0 0x7fae681006d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 -- 172.21.15.19:0/2214909810 >> 172.21.15.19:0/2214909810 conn(0x7fae68077210 msgr2=0x7fae680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 -- 172.21.15.19:0/2214909810 shutdown_connections 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 -- 172.21.15.19:0/2214909810 wait complete. 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.701+0000 7fae6d4a6640 1 Processor -- start 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 -- start start 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae681002f0 0x7fae681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 0x7fae68198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 0x7fae6819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fae68111fa0 con 0x7fae68100ca0 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fae68111e20 con 0x7fae68106f50 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae6d4a6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fae68112120 con 0x7fae681002f0 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 0x7fae68198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 0x7fae68198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53976/0 (socket says 172.21.15.19:53976) 2024-09-12T03:17:48.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 -- 172.21.15.19:0/133007859 learned_addr learned my addr 172.21.15.19:0/133007859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae66ffd640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae681002f0 0x7fae681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 -- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae681002f0 msgr2=0x7fae681984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fae681002f0 0x7fae681984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 -- 172.21.15.19:0/133007859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 msgr2=0x7fae6819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae677fe640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 0x7fae6819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 0x7fae6819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6819d480 con 0x7fae68100ca0 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae677fe640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fae68106f50 0x7fae6819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.702+0000 7fae667fc640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 0x7fae68198a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00d740 tx=0x7fae5c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.703+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c017070 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.703+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6819d770 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.703+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae5c00de50 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.703+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae5c004e00 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.703+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae681a5050 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.704+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fae68062980 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.705+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fae5c004fa0 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae4ffff640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fae500a2c10 0x7fae500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae66ffd640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fae500a2c10 0x7fae500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fae5c0cef90 con 0x7fae68100ca0 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae4ffff640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] conn(0x7fae500ac460 0x7fae500ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fae5c001d30 con 0x7fae500ac460 2024-09-12T03:17:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fae5c0cf340 con 0x7fae68100ca0 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae66ffd640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fae500a2c10 0x7fae500a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae58006010 tx=0x7fae580093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.706+0000 7fae677fe640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] conn(0x7fae500ac460 0x7fae500ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.707+0000 7fae677fe640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] conn(0x7fae500ac460 0x7fae500ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.708+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== osd.2 v2:172.21.15.135:6800/2430507446 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fae5c001d30 con 0x7fae500ac460 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.721+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 --> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fae68062e00 con 0x7fae500ac460 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae4ffff640 1 -- 172.21.15.19:0/133007859 <== osd.2 v2:172.21.15.135:6800/2430507446 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fae68062e00 con 0x7fae500ac460 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] conn(0x7fae500ac460 msgr2=0x7fae500ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] conn(0x7fae500ac460 0x7fae500ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fae500a2c10 msgr2=0x7fae500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fae500a2c10 0x7fae500a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae58006010 tx=0x7fae580093f0 comp rx=0 tx=0).stop 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 msgr2=0x7fae68198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 --2- 172.21.15.19:0/133007859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fae68100ca0 0x7fae68198a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fae5c00d740 tx=0x7fae5c00dc00 comp rx=0 tx=0).stop 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae677fe640 1 -- 172.21.15.19:0/133007859 reap_dead start 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 shutdown_connections 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 >> 172.21.15.19:0/133007859 conn(0x7fae68077210 msgr2=0x7fae681052e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 shutdown_connections 2024-09-12T03:17:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.722+0000 7fae6d4a6640 1 -- 172.21.15.19:0/133007859 wait complete. 2024-09-12T03:17:49.284 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:49 smithi019 ceph-mon[30953]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- 172.21.15.19:0/3763469138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101270 msgr2=0x7febe810d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 --2- 172.21.15.19:0/3763469138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101270 0x7febe810d7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febe0009cd0 tx=0x7febe002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- 172.21.15.19:0/3763469138 shutdown_connections 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 --2- 172.21.15.19:0/3763469138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe8101270 0x7febe810d7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 --2- 172.21.15.19:0/3763469138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe81008d0 0x7febe8100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 --2- 172.21.15.19:0/3763469138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe81068d0 0x7febe8106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- 172.21.15.19:0/3763469138 >> 172.21.15.19:0/3763469138 conn(0x7febe80fc610 msgr2=0x7febe80fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- 172.21.15.19:0/3763469138 shutdown_connections 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- 172.21.15.19:0/3763469138 wait complete. 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 Processor -- start 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.368+0000 7febeefa0640 1 -- start start 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 0x7febe80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8101270 0x7febe80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe81068d0 0x7febe80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7febe8110d70 con 0x7febe81008d0 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7febe8110bf0 con 0x7febe8101270 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febeefa0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7febe8110ef0 con 0x7febe81068d0 2024-09-12T03:17:49.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 0x7febe80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 0x7febe80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53996/0 (socket says 172.21.15.19:53996) 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 -- 172.21.15.19:0/3365391803 learned_addr learned my addr 172.21.15.19:0/3365391803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febed79d640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8101270 0x7febe80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 -- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe81068d0 msgr2=0x7febe80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7febe81068d0 0x7febe80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 -- 172.21.15.19:0/3365391803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8101270 msgr2=0x7febe80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7febe8101270 0x7febe80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe81a6860 con 0x7febe81008d0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febedf9e640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 0x7febe80fcc80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febe400b4a0 tx=0x7febe400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.369+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4014070 con 0x7febe81008d0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.370+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febe4010af0 con 0x7febe81008d0 2024-09-12T03:17:49.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.370+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe81a6b50 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.370+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febe4017d10 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.370+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe81a7090 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.370+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febe8062900 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.372+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7febe4017070 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.372+0000 7febdeffd640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7febd40a2c10 0x7febd40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.372+0000 7febed79d640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7febd40a2c10 0x7febd40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.372+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7febe40ce720 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febdeffd640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] conn(0x7febd40ac460 0x7febd40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febe400dec0 con 0x7febd40ac460 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7febe40cec00 con 0x7febe81008d0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febed79d640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7febd40a2c10 0x7febd40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febe80fe710 tx=0x7febd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febee79f640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] conn(0x7febd40ac460 0x7febd40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.373+0000 7febee79f640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] conn(0x7febd40ac460 0x7febd40ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.374+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== osd.8 v2:172.21.15.135:6816/693996615 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febe400dec0 con 0x7febd40ac460 2024-09-12T03:17:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.388+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 --> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febe8062d80 con 0x7febd40ac460 2024-09-12T03:17:49.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febdeffd640 1 -- 172.21.15.19:0/3365391803 <== osd.8 v2:172.21.15.135:6816/693996615 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7febe8062d80 con 0x7febd40ac460 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] conn(0x7febd40ac460 msgr2=0x7febd40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] conn(0x7febd40ac460 0x7febd40ae880 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7febd40a2c10 msgr2=0x7febd40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7febd40a2c10 0x7febd40a50d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7febe80fe710 tx=0x7febd8008040 comp rx=0 tx=0).stop 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 msgr2=0x7febe80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:49.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 --2- 172.21.15.19:0/3365391803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7febe81008d0 0x7febe80fcc80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7febe400b4a0 tx=0x7febe400b960 comp rx=0 tx=0).stop 2024-09-12T03:17:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febee79f640 1 -- 172.21.15.19:0/3365391803 reap_dead start 2024-09-12T03:17:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 shutdown_connections 2024-09-12T03:17:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 >> 172.21.15.19:0/3365391803 conn(0x7febe80fc610 msgr2=0x7febe806a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 shutdown_connections 2024-09-12T03:17:49.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:49.389+0000 7febeefa0640 1 -- 172.21.15.19:0/3365391803 wait complete. 2024-09-12T03:17:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:49 smithi135 ceph-mon[35474]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:17:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:49 smithi121 ceph-mon[34963]: pgmap v298: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:17:50.176 INFO:teuthology.orchestra.run.smithi019.stdout:231928234022 2024-09-12T03:17:50.177 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T03:17:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 -- 172.21.15.19:0/211595696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 msgr2=0x7f965410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 --2- 172.21.15.19:0/211595696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965410ea90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f964c009ca0 tx=0x7f964c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 -- 172.21.15.19:0/211595696 shutdown_connections 2024-09-12T03:17:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 --2- 172.21.15.19:0/211595696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965410ea90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 --2- 172.21.15.19:0/211595696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654102790 0x7f96541067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 --2- 172.21.15.19:0/211595696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9654101de0 0x7f96541021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 -- 172.21.15.19:0/211595696 >> 172.21.15.19:0/211595696 conn(0x7f96540fd880 msgr2=0x7f96540ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 -- 172.21.15.19:0/211595696 shutdown_connections 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.330+0000 7f9659747640 1 -- 172.21.15.19:0/211595696 wait complete. 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 Processor -- start 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 -- start start 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654101de0 0x7f9654198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9654102790 0x7f9654198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9654111f70 con 0x7f9654106f10 2024-09-12T03:17:50.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9654111df0 con 0x7f9654101de0 2024-09-12T03:17:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9659747640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96541120f0 con 0x7f9654102790 2024-09-12T03:17:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f96537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f96537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54016/0 (socket says 172.21.15.19:54016) 2024-09-12T03:17:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f96537fe640 1 -- 172.21.15.19:0/634703786 learned_addr learned my addr 172.21.15.19:0/634703786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:50.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f964bfff640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9654102790 0x7f9654198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.331+0000 7f9652ffd640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654101de0 0x7f9654198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 -- 172.21.15.19:0/634703786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9654102790 msgr2=0x7f9654198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9654102790 0x7f9654198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 -- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654101de0 msgr2=0x7f9654198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654101de0 0x7f9654198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965419d3c0 con 0x7f9654106f10 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9652ffd640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9654101de0 0x7f9654198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:50.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f96537fe640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965419cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f964c031a20 tx=0x7f964c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c003c30 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f965419d650 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f964c003dd0 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f964c02fcf0 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.332+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96541a4f90 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.333+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96540fee60 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.335+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f964c004050 con 0x7f9654106f10 2024-09-12T03:17:50.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.335+0000 7f9650ff9640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96400a2c10 0x7f96400a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.335+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f964c0f04f0 con 0x7f9654106f10 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.335+0000 7f9652ffd640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96400a2c10 0x7f96400a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f9650ff9640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] conn(0x7f96400ac460 0x7f96400ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96400aef30 con 0x7f96400ac460 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f964c0f09d0 con 0x7f9654106f10 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f9652ffd640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96400a2c10 0x7f96400a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96440058c0 tx=0x7f9644005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f964bfff640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] conn(0x7f96400ac460 0x7f96400ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.336+0000 7f964bfff640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] conn(0x7f96400ac460 0x7f96400ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.337+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== osd.4 v2:172.21.15.121:6808/1363833707 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96400aef30 con 0x7f96400ac460 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.351+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 --> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f96541032c0 con 0x7f96400ac460 2024-09-12T03:17:50.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.351+0000 7f9650ff9640 1 -- 172.21.15.19:0/634703786 <== osd.4 v2:172.21.15.121:6808/1363833707 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f96541032c0 con 0x7f96400ac460 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] conn(0x7f96400ac460 msgr2=0x7f96400ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] conn(0x7f96400ac460 0x7f96400ae880 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96400a2c10 msgr2=0x7f96400a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96400a2c10 0x7f96400a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f96440058c0 tx=0x7f9644005850 comp rx=0 tx=0).stop 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 msgr2=0x7f965419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 --2- 172.21.15.19:0/634703786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9654106f10 0x7f965419cce0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f964c031a20 tx=0x7f964c031a50 comp rx=0 tx=0).stop 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f96537fe640 1 -- 172.21.15.19:0/634703786 reap_dead start 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 shutdown_connections 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 >> 172.21.15.19:0/634703786 conn(0x7f96540fd880 msgr2=0x7f96540fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 shutdown_connections 2024-09-12T03:17:50.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.352+0000 7f9659747640 1 -- 172.21.15.19:0/634703786 wait complete. 2024-09-12T03:17:50.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.562+0000 7fe862b39640 1 -- 172.21.15.19:0/769735111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1082a0 msgr2=0x7fe85c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.562+0000 7fe862b39640 1 --2- 172.21.15.19:0/769735111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1082a0 0x7fe85c10ec40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe854009cd0 tx=0x7fe85402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 -- 172.21.15.19:0/769735111 shutdown_connections 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- 172.21.15.19:0/769735111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1082a0 0x7fe85c10ec40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- 172.21.15.19:0/769735111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c1036e0 0x7fe85c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- 172.21.15.19:0/769735111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c102d30 0x7fe85c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 -- 172.21.15.19:0/769735111 >> 172.21.15.19:0/769735111 conn(0x7fe85c0fc7d0 msgr2=0x7fe85c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 -- 172.21.15.19:0/769735111 shutdown_connections 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 -- 172.21.15.19:0/769735111 wait complete. 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 Processor -- start 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 -- start start 2024-09-12T03:17:50.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c102d30 0x7fe85c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 0x7fe85c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.563+0000 7fe862b39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c1082a0 0x7fe85c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe862b39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c112100 con 0x7fe85c1036e0 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe862b39640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c111f80 con 0x7fe85c102d30 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe862b39640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe85c112280 con 0x7fe85c1082a0 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 0x7fe85c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 0x7fe85c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50826/0 (socket says 172.21.15.19:50826) 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 -- 172.21.15.19:0/87561544 learned_addr learned my addr 172.21.15.19:0/87561544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe8610d6640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c1082a0 0x7fe85c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe8608d5640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c102d30 0x7fe85c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 -- 172.21.15.19:0/87561544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c1082a0 msgr2=0x7fe85c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe85c1082a0 0x7fe85c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 -- 172.21.15.19:0/87561544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c102d30 msgr2=0x7fe85c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c102d30 0x7fe85c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85c1a6a10 con 0x7fe85c1036e0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe8608d5640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe85c102d30 0x7fe85c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe85bfff640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 0x7fe85c100ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c8c0 tx=0x7fe85000cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850007bb0 con 0x7fe85c1036e0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85c1a6d00 con 0x7fe85c1036e0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe850007d50 con 0x7fe85c1036e0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.564+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850018830 con 0x7fe85c1036e0 2024-09-12T03:17:50.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.565+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85c1a71e0 con 0x7fe85c1036e0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.565+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe85c0628c0 con 0x7fe85c1036e0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.566+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe8500189d0 con 0x7fe85c1036e0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe859ffb640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8440a2c10 0x7fe8440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fe8500cf700 con 0x7fe85c1036e0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe8608d5640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8440a2c10 0x7fe8440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe859ffb640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] conn(0x7fe8440ac460 0x7fe8440ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe8440aef30 con 0x7fe8440ac460 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.567+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe8500cfab0 con 0x7fe85c1036e0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.568+0000 7fe8610d6640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] conn(0x7fe8440ac460 0x7fe8440ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:50.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.568+0000 7fe8608d5640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8440a2c10 0x7fe8440a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.568+0000 7fe8610d6640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] conn(0x7fe8440ac460 0x7fe8440ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.569+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== osd.9 v2:172.21.15.19:6826/892182666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe8440aef30 con 0x7fe8440ac460 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.582+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 --> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe85c062d40 con 0x7fe8440ac460 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe859ffb640 1 -- 172.21.15.19:0/87561544 <== osd.9 v2:172.21.15.19:6826/892182666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe85c062d40 con 0x7fe8440ac460 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] conn(0x7fe8440ac460 msgr2=0x7fe8440ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] conn(0x7fe8440ac460 0x7fe8440ae880 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8440a2c10 msgr2=0x7fe8440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8440a2c10 0x7fe8440a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe84c006fd0 tx=0x7fe84c008040 comp rx=0 tx=0).stop 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 msgr2=0x7fe85c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 --2- 172.21.15.19:0/87561544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe85c1036e0 0x7fe85c100ed0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe85000c8c0 tx=0x7fe85000cd80 comp rx=0 tx=0).stop 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe8610d6640 1 -- 172.21.15.19:0/87561544 reap_dead start 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 shutdown_connections 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 >> 172.21.15.19:0/87561544 conn(0x7fe85c0fc7d0 msgr2=0x7fe85c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 shutdown_connections 2024-09-12T03:17:50.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:50.583+0000 7fe862b39640 1 -- 172.21.15.19:0/87561544 wait complete. 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 -- 172.21.15.19:0/168261430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 msgr2=0x7f94a8105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 --2- 172.21.15.19:0/168261430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a8105750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f949c009900 tx=0x7f949c02a410 comp rx=0 tx=0).stop 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 -- 172.21.15.19:0/168261430 shutdown_connections 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 --2- 172.21.15.19:0/168261430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 0x7f94a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 --2- 172.21.15.19:0/168261430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a8105750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 --2- 172.21.15.19:0/168261430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a8101480 0x7f94a8101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 -- 172.21.15.19:0/168261430 >> 172.21.15.19:0/168261430 conn(0x7f94a8077210 msgr2=0x7f94a80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 -- 172.21.15.19:0/168261430 shutdown_connections 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.399+0000 7f94acc8f640 1 -- 172.21.15.19:0/168261430 wait complete. 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 Processor -- start 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 -- start start 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a8101480 0x7f94a80fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a80fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 0x7f94a80fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8110fc0 con 0x7f94a8101e30 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8110e40 con 0x7f94a8105e80 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.400+0000 7f94acc8f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f94a8111140 con 0x7f94a8101480 2024-09-12T03:17:51.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a80fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a80fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53940/0 (socket says 172.21.15.19:53940) 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 -- 172.21.15.19:0/965664383 learned_addr learned my addr 172.21.15.19:0/965664383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a659c640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a8101480 0x7f94a80fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a6d9d640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 0x7f94a80fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 -- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a8101480 msgr2=0x7f94a80fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f94a8101480 0x7f94a80fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 -- 172.21.15.19:0/965664383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 msgr2=0x7f94a80fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 0x7f94a80fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a81a6af0 con 0x7f94a8101e30 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a6d9d640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f94a8105e80 0x7f94a80fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94a5d9b640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a80fcbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f949c005ab0 tx=0x7f949c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c03f030 con 0x7f94a8101e30 2024-09-12T03:17:51.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81a6d80 con 0x7f94a8101e30 2024-09-12T03:17:51.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f949c003ce0 con 0x7f94a8101e30 2024-09-12T03:17:51.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c035070 con 0x7f94a8101e30 2024-09-12T03:17:51.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.401+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a81a7200 con 0x7f94a8101e30 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.402+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94a8104ff0 con 0x7f94a8101e30 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.403+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f949c042030 con 0x7f94a8101e30 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94977fe640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f94900a2c10 0x7f94900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94a659c640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f94900a2c10 0x7f94900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f949c0efd10 con 0x7f94a8101e30 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94977fe640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] conn(0x7f94900ac460 0x7f94900ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:51.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94900aef30 con 0x7f94900ac460 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.404+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f949c0f01f0 con 0x7f94a8101e30 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.405+0000 7f94a659c640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f94900a2c10 0x7f94900a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9498009990 tx=0x7f9498009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.405+0000 7f94a6d9d640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] conn(0x7f94900ac460 0x7f94900ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.405+0000 7f94a6d9d640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] conn(0x7f94900ac460 0x7f94900ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.406+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== osd.11 v2:172.21.15.135:6824/3648701186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94900aef30 con 0x7f94900ac460 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.420+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 --> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f94a8102230 con 0x7f94900ac460 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.420+0000 7f94977fe640 1 -- 172.21.15.19:0/965664383 <== osd.11 v2:172.21.15.135:6824/3648701186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f94a8102230 con 0x7f94900ac460 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] conn(0x7f94900ac460 msgr2=0x7f94900ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] conn(0x7f94900ac460 0x7f94900ae880 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f94900a2c10 msgr2=0x7f94900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f94900a2c10 0x7f94900a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9498009990 tx=0x7f9498009110 comp rx=0 tx=0).stop 2024-09-12T03:17:51.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 msgr2=0x7f94a80fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 --2- 172.21.15.19:0/965664383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f94a8101e30 0x7f94a80fcbd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f949c005ab0 tx=0x7f949c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94a6d9d640 1 -- 172.21.15.19:0/965664383 reap_dead start 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 shutdown_connections 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 >> 172.21.15.19:0/965664383 conn(0x7f94a8077210 msgr2=0x7f94a80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 shutdown_connections 2024-09-12T03:17:51.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.421+0000 7f94acc8f640 1 -- 172.21.15.19:0/965664383 wait complete. 2024-09-12T03:17:51.287 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:51 smithi019 ceph-mon[30953]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:17:51.452 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:51 smithi121 ceph-mon[34963]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:17:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:51 smithi135 ceph-mon[35474]: pgmap v299: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:17:51.716 INFO:teuthology.orchestra.run.smithi019.stdout:150323855412 2024-09-12T03:17:51.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- 172.21.15.19:0/3137478607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc1036e0 msgr2=0x7f35dc107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 --2- 172.21.15.19:0/3137478607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc1036e0 0x7f35dc107b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009930 tx=0x7f35d002a420 comp rx=0 tx=0).stop 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- 172.21.15.19:0/3137478607 shutdown_connections 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 --2- 172.21.15.19:0/3137478607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc1082a0 0x7f35dc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 --2- 172.21.15.19:0/3137478607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc1036e0 0x7f35dc107b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 --2- 172.21.15.19:0/3137478607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35dc102d30 0x7f35dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- 172.21.15.19:0/3137478607 >> 172.21.15.19:0/3137478607 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- 172.21.15.19:0/3137478607 shutdown_connections 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- 172.21.15.19:0/3137478607 wait complete. 2024-09-12T03:17:52.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 Processor -- start 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.511+0000 7f35e2535640 1 -- start start 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 0x7f35dc1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc1036e0 0x7f35dc198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35dc1082a0 0x7f35dc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc1120c0 con 0x7f35dc102d30 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc111f40 con 0x7f35dc1082a0 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e2535640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f35dc112240 con 0x7f35dc1036e0 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 0x7f35dc1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 0x7f35dc1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53960/0 (socket says 172.21.15.19:53960) 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 -- 172.21.15.19:0/881757844 learned_addr learned my addr 172.21.15.19:0/881757844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35e0ad2640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35dc1082a0 0x7f35dc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35db7fe640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc1036e0 0x7f35dc198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 -- 172.21.15.19:0/881757844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc1036e0 msgr2=0x7f35dc198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f35dc1036e0 0x7f35dc198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 -- 172.21.15.19:0/881757844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35dc1082a0 msgr2=0x7f35dc19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35dc1082a0 0x7f35dc19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35dc19d570 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.512+0000 7f35dbfff640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 0x7f35dc1985c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c920 tx=0x7f35cc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.513+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc007c70 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.513+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35dc19d860 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.513+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35cc007e10 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.513+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35cc018890 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.513+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35dc1a5140 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.514+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35dc062980 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.515+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f35cc018a30 con 0x7f35dc102d30 2024-09-12T03:17:52.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.515+0000 7f35d97fa640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f35c40a2c10 0x7f35c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35db7fe640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f35c40a2c10 0x7f35c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f35cc0cf660 con 0x7f35dc102d30 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35d97fa640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] conn(0x7f35c40ac460 0x7f35c40ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35c40aef30 con 0x7f35c40ac460 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f35cc0cfb40 con 0x7f35dc102d30 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35e0ad2640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] conn(0x7f35c40ac460 0x7f35c40ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:52.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35db7fe640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f35c40a2c10 0x7f35c40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009230 tx=0x7f35d0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.516+0000 7f35e0ad2640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] conn(0x7f35c40ac460 0x7f35c40ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.517+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== osd.6 v2:172.21.15.19:6818/2104639727 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35c40aef30 con 0x7f35c40ac460 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 --> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35dc062e00 con 0x7f35c40ac460 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35d97fa640 1 -- 172.21.15.19:0/881757844 <== osd.6 v2:172.21.15.19:6818/2104639727 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f35dc062e00 con 0x7f35c40ac460 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] conn(0x7f35c40ac460 msgr2=0x7f35c40ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] conn(0x7f35c40ac460 0x7f35c40ae880 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f35c40a2c10 msgr2=0x7f35c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f35c40a2c10 0x7f35c40a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009230 tx=0x7f35d0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 msgr2=0x7f35dc1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e2535640 1 --2- 172.21.15.19:0/881757844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f35dc102d30 0x7f35dc1985c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f35cc00c920 tx=0x7f35cc00cde0 comp rx=0 tx=0).stop 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.531+0000 7f35e0ad2640 1 -- 172.21.15.19:0/881757844 reap_dead start 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.532+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 shutdown_connections 2024-09-12T03:17:52.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.532+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 >> 172.21.15.19:0/881757844 conn(0x7f35dc0fc7d0 msgr2=0x7f35dc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.532+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 shutdown_connections 2024-09-12T03:17:52.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:48.532+0000 7f35e2535640 1 -- 172.21.15.19:0/881757844 wait complete. 2024-09-12T03:17:52.892 INFO:teuthology.orchestra.run.smithi019.stdout:180388626480 2024-09-12T03:17:52.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T03:17:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:53 smithi019 ceph-mon[30953]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 -- 172.21.15.19:0/1429645004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 msgr2=0x7fec4c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 --2- 172.21.15.19:0/1429645004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c10ea90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fec44009ca0 tx=0x7fec4402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 -- 172.21.15.19:0/1429645004 shutdown_connections 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 --2- 172.21.15.19:0/1429645004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c10ea90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 --2- 172.21.15.19:0/1429645004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec4c102790 0x7fec4c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 --2- 172.21.15.19:0/1429645004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec4c101de0 0x7fec4c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 -- 172.21.15.19:0/1429645004 >> 172.21.15.19:0/1429645004 conn(0x7fec4c0fd8c0 msgr2=0x7fec4c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 -- 172.21.15.19:0/1429645004 shutdown_connections 2024-09-12T03:17:53.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.531+0000 7fec5129b640 1 -- 172.21.15.19:0/1429645004 wait complete. 2024-09-12T03:17:53.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 Processor -- start 2024-09-12T03:17:53.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 -- start start 2024-09-12T03:17:53.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec4c101de0 0x7fec4c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:53.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec4c102790 0x7fec4c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c112010 con 0x7fec4c106f10 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c111e90 con 0x7fec4c102790 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.532+0000 7fec5129b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c112190 con 0x7fec4c101de0 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50840/0 (socket says 172.21.15.19:50840) 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 -- 172.21.15.19:0/2777695058 learned_addr learned my addr 172.21.15.19:0/2777695058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4affd640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec4c101de0 0x7fec4c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4a7fc640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec4c102790 0x7fec4c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:53.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 -- 172.21.15.19:0/2777695058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec4c101de0 msgr2=0x7fec4c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec4c101de0 0x7fec4c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 -- 172.21.15.19:0/2777695058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec4c102790 msgr2=0x7fec4c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec4c102790 0x7fec4c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c19d540 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec4b7fe640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c19ce60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec44031a60 tx=0x7fec44031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44003b70 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c19d7d0 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec44003d10 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44004770 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.533+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1a5110 con 0x7fec4c106f10 2024-09-12T03:17:53.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.534+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec4c062980 con 0x7fec4c106f10 2024-09-12T03:17:53.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.536+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fec4402f790 con 0x7fec4c106f10 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec33fff640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec340a2c10 0x7fec340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fec440f05b0 con 0x7fec4c106f10 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec4affd640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec340a2c10 0x7fec340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec33fff640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] conn(0x7fec340ac460 0x7fec340ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec340aef30 con 0x7fec340ac460 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fec440f0960 con 0x7fec4c106f10 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.537+0000 7fec4a7fc640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] conn(0x7fec340ac460 0x7fec340ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.538+0000 7fec4affd640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec340a2c10 0x7fec340a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009990 tx=0x7fec3c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:53.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.538+0000 7fec4a7fc640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] conn(0x7fec340ac460 0x7fec340ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:53.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.539+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== osd.0 v2:172.21.15.19:6802/3926650955 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec340aef30 con 0x7fec340ac460 2024-09-12T03:17:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:53 smithi135 ceph-mon[35474]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T03:17:53.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.552+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 --> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec4c062d80 con 0x7fec340ac460 2024-09-12T03:17:53.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.552+0000 7fec33fff640 1 -- 172.21.15.19:0/2777695058 <== osd.0 v2:172.21.15.19:6802/3926650955 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec4c062d80 con 0x7fec340ac460 2024-09-12T03:17:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] conn(0x7fec340ac460 msgr2=0x7fec340ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] conn(0x7fec340ac460 0x7fec340ae880 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec340a2c10 msgr2=0x7fec340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec340a2c10 0x7fec340a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009990 tx=0x7fec3c009110 comp rx=0 tx=0).stop 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 msgr2=0x7fec4c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 --2- 172.21.15.19:0/2777695058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec4c106f10 0x7fec4c19ce60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fec44031a60 tx=0x7fec44031a90 comp rx=0 tx=0).stop 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec4b7fe640 1 -- 172.21.15.19:0/2777695058 reap_dead start 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 shutdown_connections 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 >> 172.21.15.19:0/2777695058 conn(0x7fec4c0fd8c0 msgr2=0x7fec4c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 shutdown_connections 2024-09-12T03:17:53.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:53.553+0000 7fec5129b640 1 -- 172.21.15.19:0/2777695058 wait complete. 2024-09-12T03:17:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:53 smithi121 ceph-mon[34963]: pgmap v300: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T03:17:54.319 INFO:teuthology.orchestra.run.smithi019.stdout:34359738447 2024-09-12T03:17:54.320 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T03:17:54.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3168267329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 msgr2=0x7fb7d8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3168267329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8103120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009900 tx=0x7fb7c802a480 comp rx=0 tx=0).stop 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3168267329 shutdown_connections 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3168267329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d810c170 0x7fb7d810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3168267329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7d8103660 0x7fb7d810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3168267329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8103120 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.906+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3168267329 >> 172.21.15.19:0/3168267329 conn(0x7fb7d80fc7a0 msgr2=0x7fb7d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.907+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3168267329 shutdown_connections 2024-09-12T03:17:54.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.907+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3168267329 wait complete. 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.907+0000 7fb7ddf54640 1 Processor -- start 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.907+0000 7fb7ddf54640 1 -- start start 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7d8103660 0x7fb7d81989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d810c170 0x7fb7d819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8111f70 con 0x7fb7d8102d40 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d8111df0 con 0x7fb7d8103660 2024-09-12T03:17:54.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d81120f0 con 0x7fb7d810c170 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50884/0 (socket says 172.21.15.19:50884) 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 -- 172.21.15.19:0/3425155905 learned_addr learned my addr 172.21.15.19:0/3425155905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d6ffd640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7d8103660 0x7fb7d81989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d7fff640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d810c170 0x7fb7d819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 -- 172.21.15.19:0/3425155905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d810c170 msgr2=0x7fb7d819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb7d810c170 0x7fb7d819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 -- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7d8103660 msgr2=0x7fb7d81989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb7d8103660 0x7fb7d81989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d819d420 con 0x7fb7d8102d40 2024-09-12T03:17:54.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d77fe640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8198470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8005ab0 tx=0x7fb7c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c803f030 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c8003ce0 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d819d6b0 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.908+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c8035070 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.909+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d81a4ff0 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.909+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb7d8062980 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.911+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb7c8042030 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.911+0000 7fb7d4ff9640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7c00a2c90 0x7fb7c00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.911+0000 7fb7d6ffd640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7c00a2c90 0x7fb7c00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.911+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb7c80efc70 con 0x7fb7d8102d40 2024-09-12T03:17:54.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d4ff9640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] conn(0x7fb7c00ac4e0 0x7fb7c00ae900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb7c8003b30 con 0x7fb7c00ac4e0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb7c80f0150 con 0x7fb7d8102d40 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d6ffd640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7c00a2c90 0x7fb7c00a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb7d810d620 tx=0x7fb7cc00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d7fff640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] conn(0x7fb7c00ac4e0 0x7fb7c00ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.912+0000 7fb7d7fff640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] conn(0x7fb7c00ac4e0 0x7fb7c00ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.913+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== osd.7 v2:172.21.15.121:6816/1039653795 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb7c8003b30 con 0x7fb7c00ac4e0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.927+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 --> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7d8062d80 con 0x7fb7c00ac4e0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.927+0000 7fb7d4ff9640 1 -- 172.21.15.19:0/3425155905 <== osd.7 v2:172.21.15.121:6816/1039653795 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7d8062d80 con 0x7fb7c00ac4e0 2024-09-12T03:17:54.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] conn(0x7fb7c00ac4e0 msgr2=0x7fb7c00ae900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] conn(0x7fb7c00ac4e0 0x7fb7c00ae900 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7c00a2c90 msgr2=0x7fb7c00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7c00a2c90 0x7fb7c00a5150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb7d810d620 tx=0x7fb7cc00a560 comp rx=0 tx=0).stop 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 msgr2=0x7fb7d8198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 --2- 172.21.15.19:0/3425155905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb7d8102d40 0x7fb7d8198470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8005ab0 tx=0x7fb7c80042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7d7fff640 1 -- 172.21.15.19:0/3425155905 reap_dead start 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 shutdown_connections 2024-09-12T03:17:54.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 >> 172.21.15.19:0/3425155905 conn(0x7fb7d80fc7a0 msgr2=0x7fb7d80fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 shutdown_connections 2024-09-12T03:17:54.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.928+0000 7fb7ddf54640 1 -- 172.21.15.19:0/3425155905 wait complete. 2024-09-12T03:17:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:55 smithi135 ceph-mon[35474]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:55.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:55 smithi121 ceph-mon[34963]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:55.655 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:55 smithi019 ceph-mon[30953]: pgmap v301: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:55.694 INFO:teuthology.orchestra.run.smithi019.stdout:266287972384 2024-09-12T03:17:55.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T03:17:56.115 INFO:teuthology.orchestra.run.smithi019.stdout:201863462955 2024-09-12T03:17:56.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T03:17:56.247 INFO:teuthology.orchestra.run.smithi019.stdout:317827579926 2024-09-12T03:17:56.248 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T03:17:56.670 INFO:teuthology.orchestra.run.smithi019.stdout:128849018938 2024-09-12T03:17:56.670 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T03:17:56.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 -- 172.21.15.19:0/99641953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890102790 msgr2=0x7fc8901067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 --2- 172.21.15.19:0/99641953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890102790 0x7fc8901067e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc880009900 tx=0x7fc88002a410 comp rx=0 tx=0).stop 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 -- 172.21.15.19:0/99641953 shutdown_connections 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 --2- 172.21.15.19:0/99641953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890106f10 0x7fc89010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 --2- 172.21.15.19:0/99641953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890102790 0x7fc8901067e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 --2- 172.21.15.19:0/99641953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc890101de0 0x7fc8901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 -- 172.21.15.19:0/99641953 >> 172.21.15.19:0/99641953 conn(0x7fc8900fd8c0 msgr2=0x7fc8900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 -- 172.21.15.19:0/99641953 shutdown_connections 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.793+0000 7fc8977a8640 1 -- 172.21.15.19:0/99641953 wait complete. 2024-09-12T03:17:56.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 Processor -- start 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 -- start start 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 0x7fc890198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890102790 0x7fc890198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc890106f10 0x7fc89019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc890111fe0 con 0x7fc890101de0 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc890111e60 con 0x7fc890106f10 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc8977a8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc890112160 con 0x7fc890102790 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.794+0000 7fc895544640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 0x7fc890198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 0x7fc890198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50862/0 (socket says 172.21.15.19:50862) 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 -- 172.21.15.19:0/3054221784 learned_addr learned my addr 172.21.15.19:0/3054221784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:56.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc894d43640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890102790 0x7fc890198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 -- 172.21.15.19:0/3054221784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890102790 msgr2=0x7fc890198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc890102790 0x7fc890198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895d45640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc890106f10 0x7fc89019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 -- 172.21.15.19:0/3054221784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc890106f10 msgr2=0x7fc89019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc890106f10 0x7fc89019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89019d4d0 con 0x7fc890101de0 2024-09-12T03:17:56.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc895544640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 0x7fc890198520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b4a0 tx=0x7fc88c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c014070 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc88c010af0 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89019d7c0 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c017d10 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.795+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8901a50a0 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.796+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc8900628c0 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.797+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc88c017070 con 0x7fc890101de0 2024-09-12T03:17:56.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc8867fc640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc87c0a2c10 0x7fc87c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc894d43640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc87c0a2c10 0x7fc87c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc88c0ce680 con 0x7fc890101de0 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc8867fc640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] conn(0x7fc87c0ac460 0x7fc87c0ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc88c01ae50 con 0x7fc87c0ac460 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.798+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fc88c0ceb60 con 0x7fc890101de0 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.799+0000 7fc895d45640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] conn(0x7fc87c0ac460 0x7fc87c0ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.799+0000 7fc894d43640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc87c0a2c10 0x7fc87c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880005df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:56.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.799+0000 7fc895d45640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] conn(0x7fc87c0ac460 0x7fc87c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.800+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== osd.3 v2:172.21.15.19:6810/3191237589 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc88c01ae50 con 0x7fc87c0ac460 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.813+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 --> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc890062cc0 con 0x7fc87c0ac460 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.813+0000 7fc8867fc640 1 -- 172.21.15.19:0/3054221784 <== osd.3 v2:172.21.15.19:6810/3191237589 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc890062cc0 con 0x7fc87c0ac460 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] conn(0x7fc87c0ac460 msgr2=0x7fc87c0ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] conn(0x7fc87c0ac460 0x7fc87c0ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc87c0a2c10 msgr2=0x7fc87c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc87c0a2c10 0x7fc87c0a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc880000c00 tx=0x7fc880005df0 comp rx=0 tx=0).stop 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 msgr2=0x7fc890198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 --2- 172.21.15.19:0/3054221784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc890101de0 0x7fc890198520 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00b4a0 tx=0x7fc88c00b960 comp rx=0 tx=0).stop 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc895d45640 1 -- 172.21.15.19:0/3054221784 reap_dead start 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 shutdown_connections 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 >> 172.21.15.19:0/3054221784 conn(0x7fc8900fd8c0 msgr2=0x7fc8900fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 shutdown_connections 2024-09-12T03:17:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:54.814+0000 7fc8977a8640 1 -- 172.21.15.19:0/3054221784 wait complete. 2024-09-12T03:17:57.365 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:57 smithi019 ceph-mon[30953]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:57.400 INFO:teuthology.orchestra.run.smithi019.stdout:98784247870 2024-09-12T03:17:57.400 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T03:17:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:57 smithi135 ceph-mon[35474]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:57 smithi121 ceph-mon[34963]: pgmap v302: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:17:57.731 INFO:teuthology.orchestra.run.smithi019.stdout:73014444100 2024-09-12T03:17:57.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T03:17:57.793 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:57.794 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:58.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 -- 172.21.15.19:0/1735472077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 msgr2=0x7f0d8410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 --2- 172.21.15.19:0/1735472077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8410df40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d74009900 tx=0x7f0d7402a410 comp rx=0 tx=0).stop 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 -- 172.21.15.19:0/1735472077 shutdown_connections 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 --2- 172.21.15.19:0/1735472077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d8410e480 0x7f0d84110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 --2- 172.21.15.19:0/1735472077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8410df40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 --2- 172.21.15.19:0/1735472077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d841008d0 0x7f0d84100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.331+0000 7f0d8c103640 1 -- 172.21.15.19:0/1735472077 >> 172.21.15.19:0/1735472077 conn(0x7f0d840fc7a0 msgr2=0x7f0d840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- 172.21.15.19:0/1735472077 shutdown_connections 2024-09-12T03:17:58.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- 172.21.15.19:0/1735472077 wait complete. 2024-09-12T03:17:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 Processor -- start 2024-09-12T03:17:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- start start 2024-09-12T03:17:58.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d841008d0 0x7f0d8419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d8410e480 0x7f0d8419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d841141b0 con 0x7f0d84101280 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84114030 con 0x7f0d841008d0 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8c103640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d84114330 con 0x7f0d8410e480 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8969e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8969e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8419abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50900/0 (socket says 172.21.15.19:50900) 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.332+0000 7f0d8969e640 1 -- 172.21.15.19:0/694912833 learned_addr learned my addr 172.21.15.19:0/694912833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8a6a0640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d8410e480 0x7f0d8419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d89e9f640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d841008d0 0x7f0d8419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 -- 172.21.15.19:0/694912833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d8410e480 msgr2=0x7f0d8419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d8410e480 0x7f0d8419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 -- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d841008d0 msgr2=0x7f0d8419a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d841008d0 0x7f0d8419a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8419f660 con 0x7f0d84101280 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8a6a0640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d8410e480 0x7f0d8419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8969e640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8419abf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d74006010 tx=0x7f0d740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d7403f030 con 0x7f0d84101280 2024-09-12T03:17:58.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8419f8f0 con 0x7f0d84101280 2024-09-12T03:17:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d74003ce0 con 0x7f0d84101280 2024-09-12T03:17:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d74035070 con 0x7f0d84101280 2024-09-12T03:17:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.333+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d841a7230 con 0x7f0d84101280 2024-09-12T03:17:58.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.334+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d84062980 con 0x7f0d84101280 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.335+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0d74042030 con 0x7f0d84101280 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d7affd640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d700a2c10 0x7f0d700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d89e9f640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d700a2c10 0x7f0d700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0d740efd00 con 0x7f0d84101280 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d7affd640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] conn(0x7f0d700ac460 0x7f0d700ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d7403be50 con 0x7f0d700ac460 2024-09-12T03:17:58.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.336+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f0d740f01e0 con 0x7f0d84101280 2024-09-12T03:17:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.337+0000 7f0d89e9f640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d700a2c10 0x7f0d700a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d80005fd0 tx=0x7f0d80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.337+0000 7f0d8a6a0640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] conn(0x7f0d700ac460 0x7f0d700ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:17:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.337+0000 7f0d8a6a0640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] conn(0x7f0d700ac460 0x7f0d700ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:17:58.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.338+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== osd.10 v2:172.21.15.121:6824/1462491703 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d7403be50 con 0x7f0d700ac460 2024-09-12T03:17:58.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.352+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 --> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d84062d80 con 0x7f0d700ac460 2024-09-12T03:17:58.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.352+0000 7f0d7affd640 1 -- 172.21.15.19:0/694912833 <== osd.10 v2:172.21.15.121:6824/1462491703 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0d84062d80 con 0x7f0d700ac460 2024-09-12T03:17:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] conn(0x7f0d700ac460 msgr2=0x7f0d700ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] conn(0x7f0d700ac460 0x7f0d700ae880 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d700a2c10 msgr2=0x7f0d700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d700a2c10 0x7f0d700a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0d80005fd0 tx=0x7f0d80005850 comp rx=0 tx=0).stop 2024-09-12T03:17:58.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 msgr2=0x7f0d8419abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 --2- 172.21.15.19:0/694912833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d84101280 0x7f0d8419abf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d74006010 tx=0x7f0d740042f0 comp rx=0 tx=0).stop 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8a6a0640 1 -- 172.21.15.19:0/694912833 reap_dead start 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 shutdown_connections 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 >> 172.21.15.19:0/694912833 conn(0x7f0d840fc7a0 msgr2=0x7f0d840feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 shutdown_connections 2024-09-12T03:17:58.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:17:58.353+0000 7f0d8c103640 1 -- 172.21.15.19:0/694912833 wait complete. 2024-09-12T03:17:59.158 INFO:teuthology.orchestra.run.smithi019.stdout:287762808859 2024-09-12T03:17:59.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T03:17:59.225 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.233 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.237 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.238 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.245 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.317 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:17:59.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:17:59 smithi019 ceph-mon[30953]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:17:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:17:59 smithi135 ceph-mon[35474]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:17:59.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:17:59 smithi121 ceph-mon[34963]: pgmap v303: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:01.303 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:01 smithi019 ceph-mon[30953]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-12T03:18:01.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 -- 172.21.15.19:0/2638710051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8103660 msgr2=0x7fa1a810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/2638710051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8103660 0x7fa1a810bb50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa190009900 tx=0x7fa19002a480 comp rx=0 tx=0).stop 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 -- 172.21.15.19:0/2638710051 shutdown_connections 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/2638710051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a810c170 0x7fa1a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/2638710051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8103660 0x7fa1a810bb50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/2638710051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 -- 172.21.15.19:0/2638710051 >> 172.21.15.19:0/2638710051 conn(0x7fa1a80fc7a0 msgr2=0x7fa1a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 -- 172.21.15.19:0/2638710051 shutdown_connections 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 -- 172.21.15.19:0/2638710051 wait complete. 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.420+0000 7fa1ad92f640 1 Processor -- start 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 -- start start 2024-09-12T03:18:01.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 0x7fa1a819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8111f70 con 0x7fa1a810c170 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8111df0 con 0x7fa1a8102d40 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1ad92f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81120f0 con 0x7fa1a8103660 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60192/0 (socket says 172.21.15.19:60192) 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 -- 172.21.15.19:0/376759708 learned_addr learned my addr 172.21.15.19:0/376759708 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a6ffd640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa19ffff640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 0x7fa1a819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 -- 172.21.15.19:0/376759708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 msgr2=0x7fa1a819abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 0x7fa1a819abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 -- 172.21.15.19:0/376759708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 msgr2=0x7fa1a819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 -- 172.21.15.19:0/376759708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819f640 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa19ffff640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 0x7fa1a819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a6ffd640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.421+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000cad0 tx=0x7fa1a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.422+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000ae20 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.422+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.422+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819f930 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.422+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000f6f0 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.422+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a7210 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.423+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8103120 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.424+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa1a00051d0 con 0x7fa1a810c170 2024-09-12T03:18:01.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.425+0000 7fa1ac92d640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 0x7fa1940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.425+0000 7fa1a6ffd640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 0x7fa1940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.425+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fa1a00cf3e0 con 0x7fa1a810c170 2024-09-12T03:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.425+0000 7fa1a6ffd640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 0x7fa1940a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1980058c0 tx=0x7fa19800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.428+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1a0014030 con 0x7fa1a810c170 2024-09-12T03:18:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:01 smithi135 ceph-mon[35474]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-12T03:18:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.608+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa1a819b9f0 con 0x7fa1a810c170 2024-09-12T03:18:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.608+0000 7fa1ac92d640 1 -- 172.21.15.19:0/376759708 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1a819b9f0 con 0x7fa1a810c170 2024-09-12T03:18:01.608 INFO:teuthology.orchestra.run.smithi019.stdout:98784247871 2024-09-12T03:18:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 msgr2=0x7fa1940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 0x7fa1940a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa1980058c0 tx=0x7fa19800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 msgr2=0x7fa1a819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000cad0 tx=0x7fa1a000cf90 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 shutdown_connections 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1940a2c10 0x7fa1940a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a810c170 0x7fa1a819ef60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8103660 0x7fa1a819abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 --2- 172.21.15.19:0/376759708 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8102d40 0x7fa1a819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 >> 172.21.15.19:0/376759708 conn(0x7fa1a80fc7a0 msgr2=0x7fa1a8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 shutdown_connections 2024-09-12T03:18:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.610+0000 7fa1ad92f640 1 -- 172.21.15.19:0/376759708 wait complete. 2024-09-12T03:18:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 -- 172.21.15.19:0/3258218255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4102d50 msgr2=0x7f99a4103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 --2- 172.21.15.19:0/3258218255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4102d50 0x7f99a4103130 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9994009900 tx=0x7f999402a480 comp rx=0 tx=0).stop 2024-09-12T03:18:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 -- 172.21.15.19:0/3258218255 shutdown_connections 2024-09-12T03:18:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 --2- 172.21.15.19:0/3258218255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a410c180 0x7f99a410e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 --2- 172.21.15.19:0/3258218255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99a4103670 0x7f99a410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 --2- 172.21.15.19:0/3258218255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4102d50 0x7f99a4103130 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 -- 172.21.15.19:0/3258218255 >> 172.21.15.19:0/3258218255 conn(0x7f99a40fc7d0 msgr2=0x7f99a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 -- 172.21.15.19:0/3258218255 shutdown_connections 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.613+0000 7f99a9e99640 1 -- 172.21.15.19:0/3258218255 wait complete. 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 Processor -- start 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 -- start start 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a4102d50 0x7f99a4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 0x7f99a41988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99a410c180 0x7f99a419cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4111fd0 con 0x7f99a4103670 2024-09-12T03:18:01.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4111e50 con 0x7f99a410c180 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.614+0000 7f99a9e99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99a4112150 con 0x7f99a4102d50 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 0x7f99a41988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 0x7f99a41988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60212/0 (socket says 172.21.15.19:60212) 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 -- 172.21.15.19:0/132664415 learned_addr learned my addr 172.21.15.19:0/132664415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a37fe640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a4102d50 0x7f99a4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a3fff640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99a410c180 0x7f99a419cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 -- 172.21.15.19:0/132664415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a4102d50 msgr2=0x7f99a4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a4102d50 0x7f99a4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 -- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99a410c180 msgr2=0x7f99a419cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f99a410c180 0x7f99a419cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a419d320 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a37fe640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f99a4102d50 0x7f99a4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f999affd640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 0x7f99a41988b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f998c00c970 tx=0x7f998c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c007bb0 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a419d610 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f998c007d50 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.615+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998c018820 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.616+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a41a4ef0 con 0x7f99a4103670 2024-09-12T03:18:01.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.616+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99a40fedc0 con 0x7f99a4103670 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.619+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f998c00f450 con 0x7f99a4103670 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.619+0000 7f99a17fa640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a37fe640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99900a2c10 0x7f99900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f998c0cf710 con 0x7f99a4103670 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a17fa640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] conn(0x7f99900ac460 0x7f99900ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:01.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f998c00c120 con 0x7f99900ac460 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f998c0cfbf0 con 0x7f99a4103670 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a37fe640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.620+0000 7f99a3fff640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] conn(0x7f99900ac460 0x7f99900ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.621+0000 7f99a3fff640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] conn(0x7f99900ac460 0x7f99900ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.621+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== osd.1 v2:172.21.15.121:6800/4159912273 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f998c00c120 con 0x7f99900ac460 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.635+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 --> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99a406b070 con 0x7f99900ac460 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.636+0000 7f99a17fa640 1 -- 172.21.15.19:0/132664415 <== osd.1 v2:172.21.15.121:6800/4159912273 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f99a406b070 con 0x7f99900ac460 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] conn(0x7f99900ac460 msgr2=0x7f99900ae880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] conn(0x7f99900ac460 0x7f99900ae880 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:01.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99900a2c10 msgr2=0x7f99900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99900a2c10 0x7f99900a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9994000c00 tx=0x7f9994005a20 comp rx=0 tx=0).stop 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 msgr2=0x7f99a41988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 --2- 172.21.15.19:0/132664415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f99a4103670 0x7f99a41988b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f998c00c970 tx=0x7f998c00ce30 comp rx=0 tx=0).stop 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a3fff640 1 -- 172.21.15.19:0/132664415 reap_dead start 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 shutdown_connections 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 >> 172.21.15.19:0/132664415 conn(0x7f99a40fc7d0 msgr2=0x7f99a40fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 shutdown_connections 2024-09-12T03:18:01.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.637+0000 7f99a9e99640 1 -- 172.21.15.19:0/132664415 wait complete. 2024-09-12T03:18:01.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:01 smithi121 ceph-mon[34963]: pgmap v304: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 -- 172.21.15.19:0/1665637877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 msgr2=0x7f8754103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 --2- 172.21.15.19:0/1665637877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754103110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8744009930 tx=0x7f874402a420 comp rx=0 tx=0).stop 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 -- 172.21.15.19:0/1665637877 shutdown_connections 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 --2- 172.21.15.19:0/1665637877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 0x7f875410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 --2- 172.21.15.19:0/1665637877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 0x7f8754107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 --2- 172.21.15.19:0/1665637877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754103110 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 -- 172.21.15.19:0/1665637877 >> 172.21.15.19:0/1665637877 conn(0x7f87540fc7d0 msgr2=0x7f87540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 -- 172.21.15.19:0/1665637877 shutdown_connections 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.145+0000 7f87593cf640 1 -- 172.21.15.19:0/1665637877 wait complete. 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 Processor -- start 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 -- start start 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 0x7f8754100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 0x7f8754101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87541120a0 con 0x7f8754102d30 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8754111f20 con 0x7f87541082a0 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.146+0000 7f87593cf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8754112220 con 0x7f87541036e0 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60218/0 (socket says 172.21.15.19:60218) 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 -- 172.21.15.19:0/1955043836 learned_addr learned my addr 172.21.15.19:0/1955043836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f87527fc640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 0x7f8754100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f87537fe640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 0x7f8754101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 -- 172.21.15.19:0/1955043836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 msgr2=0x7f8754100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 0x7f8754100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 -- 172.21.15.19:0/1955043836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 msgr2=0x7f8754101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 0x7f8754101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 -- 172.21.15.19:0/1955043836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87541a6a80 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f8752ffd640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8744009230 tx=0x7f8744004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874403f030 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8744003ad0 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87541a6d10 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744035070 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.147+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87541a71a0 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.148+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754103110 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.150+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8744042030 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.151+0000 7f873bfff640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 0x7f873c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.151+0000 7f87527fc640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 0x7f873c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.151+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f87440ef820 con 0x7f8754102d30 2024-09-12T03:18:02.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.151+0000 7f87527fc640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 0x7f873c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8754102420 tx=0x7f874800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.154+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87440bc180 con 0x7f8754102d30 2024-09-12T03:18:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.333+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f875410a2b0 con 0x7f8754102d30 2024-09-12T03:18:02.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:02 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2282743838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:18:02.334 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:02 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/376759708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:18:02.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.334+0000 7f873bfff640 1 -- 172.21.15.19:0/1955043836 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f87440c0480 con 0x7f8754102d30 2024-09-12T03:18:02.335 INFO:teuthology.orchestra.run.smithi019.stdout:201863462956 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 msgr2=0x7f873c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 0x7f873c0a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8754102420 tx=0x7f874800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 msgr2=0x7f8754100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8744009230 tx=0x7f8744004270 comp rx=0 tx=0).stop 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 shutdown_connections 2024-09-12T03:18:02.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87541082a0 0x7f8754101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87541036e0 0x7f8754100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f873c0a2c10 0x7f873c0a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 --2- 172.21.15.19:0/1955043836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8754102d30 0x7f8754100990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 >> 172.21.15.19:0/1955043836 conn(0x7f87540fc7d0 msgr2=0x7f8754105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 shutdown_connections 2024-09-12T03:18:02.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.336+0000 7f87593cf640 1 -- 172.21.15.19:0/1955043836 wait complete. 2024-09-12T03:18:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/1682471446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc109110 msgr2=0x7fc3dc110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/1682471446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc109110 0x7fc3dc110c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009ca0 tx=0x7fc3d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/1682471446 shutdown_connections 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/1682471446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc109110 0x7fc3dc110c90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/1682471446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc104990 0x7fc3dc1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/1682471446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/1682471446 >> 172.21.15.19:0/1682471446 conn(0x7fc3dc0ffac0 msgr2=0x7fc3dc101ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/1682471446 shutdown_connections 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.344+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/1682471446 wait complete. 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 Processor -- start 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 -- start start 2024-09-12T03:18:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc109110 0x7fc3dc19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc114120 con 0x7fc3dc104990 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc113fa0 con 0x7fc3dc109110 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3e0d4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc3dc1142a0 con 0x7fc3dc103fe0 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60228/0 (socket says 172.21.15.19:60228) 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3d9d9b640 1 -- 172.21.15.19:0/3135968101 learned_addr learned my addr 172.21.15.19:0/3135968101 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3dad9d640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc109110 0x7fc3dc19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.345+0000 7fc3da59c640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 -- 172.21.15.19:0/3135968101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 msgr2=0x7fc3dc19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 -- 172.21.15.19:0/3135968101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc109110 msgr2=0x7fc3dc19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc109110 0x7fc3dc19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 -- 172.21.15.19:0/3135968101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3dc19f450 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3da59c640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3d9d9b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c920 tx=0x7fc3d000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0007c70 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3dc19f740 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3d0007e10 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0018890 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.346+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3dc1a7020 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.347+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc1043c0 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.348+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc3d0018a50 con 0x7fc3dc104990 2024-09-12T03:18:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.349+0000 7fc3cb7fe640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.349+0000 7fc3da59c640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.349+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc3d00cf0e0 con 0x7fc3dc104990 2024-09-12T03:18:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.350+0000 7fc3da59c640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00abe0 tx=0x7fc3cc00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.354+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3d0014030 con 0x7fc3dc104990 2024-09-12T03:18:02.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.531+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc3dc1a7310 con 0x7fc3dc104990 2024-09-12T03:18:02.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.532+0000 7fc3cb7fe640 1 -- 172.21.15.19:0/3135968101 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc3d009c090 con 0x7fc3dc104990 2024-09-12T03:18:02.532 INFO:teuthology.orchestra.run.smithi019.stdout:231928234024 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.533+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 msgr2=0x7fc3c40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.533+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00abe0 tx=0x7fc3cc00a5c0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.533+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 msgr2=0x7fc3dc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.533+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c920 tx=0x7fc3d000cde0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 shutdown_connections 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc3dc109110 0x7fc3dc19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3c40a2c10 0x7fc3c40a50d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc3dc104990 0x7fc3dc19a9e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 --2- 172.21.15.19:0/3135968101 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc3dc103fe0 0x7fc3dc19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 >> 172.21.15.19:0/3135968101 conn(0x7fc3dc0ffac0 msgr2=0x7fc3dc100500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 shutdown_connections 2024-09-12T03:18:02.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.534+0000 7fc3e0d4b640 1 -- 172.21.15.19:0/3135968101 wait complete. 2024-09-12T03:18:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:02 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2282743838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:18:02.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:02 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/376759708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:18:02.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:02 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2282743838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T03:18:02.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:02 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/376759708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T03:18:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 -- 172.21.15.19:0/3375469898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 msgr2=0x7fc63c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3375469898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c10da40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ca0 tx=0x7fc62c02a950 comp rx=0 tx=0).stop 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 -- 172.21.15.19:0/3375469898 shutdown_connections 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3375469898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c10da40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3375469898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c102570 0x7fc63c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3375469898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c100e20 0x7fc63c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 -- 172.21.15.19:0/3375469898 >> 172.21.15.19:0/3375469898 conn(0x7fc63c0fc7d0 msgr2=0x7fc63c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 -- 172.21.15.19:0/3375469898 shutdown_connections 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 -- 172.21.15.19:0/3375469898 wait complete. 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.772+0000 7fc6446cf640 1 Processor -- start 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 -- start start 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 0x7fc63c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c102570 0x7fc63c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c110f80 con 0x7fc63c102ef0 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c110e00 con 0x7fc63c100e20 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc6446cf640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc63c111100 con 0x7fc63c102570 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60242/0 (socket says 172.21.15.19:60242) 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 -- 172.21.15.19:0/3754318512 learned_addr learned my addr 172.21.15.19:0/3754318512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc64246b640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 0x7fc63c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc641c6a640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c102570 0x7fc63c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 -- 172.21.15.19:0/3754318512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c102570 msgr2=0x7fc63c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c102570 0x7fc63c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 -- 172.21.15.19:0/3754318512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 msgr2=0x7fc63c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 0x7fc63c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 -- 172.21.15.19:0/3754318512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63c19d490 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc64246b640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 0x7fc63c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.773+0000 7fc642c6c640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc62c004af0 tx=0x7fc62c02f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.774+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c034040 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.774+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c030920 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.774+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63c19d720 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.774+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c004820 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.774+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63c1a5060 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.775+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc63c1021b0 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.776+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc62c037020 con 0x7fc63c102ef0 2024-09-12T03:18:02.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.776+0000 7fc63b7fe640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 0x7fc6240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.777+0000 7fc64246b640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 0x7fc6240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.777+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fc62c0efe80 con 0x7fc63c102ef0 2024-09-12T03:18:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.777+0000 7fc64246b640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 0x7fc6240a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc634009c20 tx=0x7fc634005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.781+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc62c0bcf40 con 0x7fc63c102ef0 2024-09-12T03:18:02.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.959+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fc63c10a0a0 con 0x7fc63c102ef0 2024-09-12T03:18:02.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.959+0000 7fc63b7fe640 1 -- 172.21.15.19:0/3754318512 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc62c0bc840 con 0x7fc63c102ef0 2024-09-12T03:18:02.959 INFO:teuthology.orchestra.run.smithi019.stdout:317827579929 2024-09-12T03:18:02.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 msgr2=0x7fc6240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 0x7fc6240a50d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc634009c20 tx=0x7fc634005f90 comp rx=0 tx=0).stop 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 msgr2=0x7fc63c19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc62c004af0 tx=0x7fc62c02f5b0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 shutdown_connections 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc6240a2c10 0x7fc6240a50d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc63c102ef0 0x7fc63c19cdb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc63c102570 0x7fc63c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 --2- 172.21.15.19:0/3754318512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc63c100e20 0x7fc63c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 >> 172.21.15.19:0/3754318512 conn(0x7fc63c0fc7d0 msgr2=0x7fc63c06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 shutdown_connections 2024-09-12T03:18:02.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:02.961+0000 7fc6446cf640 1 -- 172.21.15.19:0/3754318512 wait complete. 2024-09-12T03:18:03.294 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:03 smithi019 ceph-mon[30953]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:03.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1955043836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:18:03.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3135968101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:18:03.295 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:03 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3754318512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:18:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:03 smithi135 ceph-mon[35474]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1955043836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:18:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3135968101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:18:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:03 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3754318512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 -- 172.21.15.19:0/3126301716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 msgr2=0x7fee58100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/3126301716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee58100f00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fee54009900 tx=0x7fee5402a410 comp rx=0 tx=0).stop 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 -- 172.21.15.19:0/3126301716 shutdown_connections 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/3126301716 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 0x7fee58110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/3126301716 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/3126301716 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee58100f00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 -- 172.21.15.19:0/3126301716 >> 172.21.15.19:0/3126301716 conn(0x7fee580fc7d0 msgr2=0x7fee580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 -- 172.21.15.19:0/3126301716 shutdown_connections 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.303+0000 7fee5f3a1640 1 -- 172.21.15.19:0/3126301716 wait complete. 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 Processor -- start 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 -- start start 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 0x7fee5819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fee581142d0 con 0x7fee58100b20 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee58114150 con 0x7fee58101440 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5f3a1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fee58114450 con 0x7fee5810e500 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5c93c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5d13d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5d93e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 0x7fee5819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5d13d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60180/0 (socket says 172.21.15.19:60180) 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5c93c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37160/0 (socket says 172.21.15.19:37160) 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.304+0000 7fee5d13d640 1 -- 172.21.15.19:0/2282743838 learned_addr learned my addr 172.21.15.19:0/2282743838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 -- 172.21.15.19:0/2282743838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 msgr2=0x7fee5819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 0x7fee5819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 -- 172.21.15.19:0/2282743838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 msgr2=0x7fee5819ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 -- 172.21.15.19:0/2282743838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5819f6a0 con 0x7fee58100b20 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5c93c640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:03.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5d13d640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee5403f030 con 0x7fee58100b20 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stdout:73014444102 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee54003ce0 con 0x7fee58100b20 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee5819f930 con 0x7fee58100b20 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee54035070 con 0x7fee58100b20 2024-09-12T03:18:03.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.305+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee581a7270 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.306+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee5806a850 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.307+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee54042030 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.308+0000 7fee4e7fc640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 0x7fee440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.308+0000 7fee5c93c640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 0x7fee440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.308+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fee540efad0 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.309+0000 7fee5c93c640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 0x7fee440a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fee581027e0 tx=0x7fee48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.312+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fee540bc430 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.500+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fee58100f00 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.501+0000 7fee4e7fc640 1 -- 172.21.15.19:0/2282743838 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee540c0730 con 0x7fee58100b20 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 msgr2=0x7fee440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 0x7fee440a50d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fee581027e0 tx=0x7fee48005f90 comp rx=0 tx=0).stop 2024-09-12T03:18:03.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 msgr2=0x7fee5819a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fee54005ab0 tx=0x7fee540042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 shutdown_connections 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fee5810e500 0x7fee5819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee58101440 0x7fee5819ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fee440a2c10 0x7fee440a50d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 --2- 172.21.15.19:0/2282743838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fee58100b20 0x7fee5819a6f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 >> 172.21.15.19:0/2282743838 conn(0x7fee580fc7d0 msgr2=0x7fee580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 shutdown_connections 2024-09-12T03:18:03.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:01.503+0000 7fee5f3a1640 1 -- 172.21.15.19:0/2282743838 wait complete. 2024-09-12T03:18:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:03 smithi121 ceph-mon[34963]: pgmap v305: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1955043836' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T03:18:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3135968101' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T03:18:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:03 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3754318512' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T03:18:04.172 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444100 got 73014444102 for osd.2 2024-09-12T03:18:04.172 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:04.262 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:04 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- 172.21.15.19:0/1553788343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc103660 msgr2=0x7fb5fc10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 --2- 172.21.15.19:0/1553788343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc103660 0x7fb5fc10bb50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009900 tx=0x7fb5f002a480 comp rx=0 tx=0).stop 2024-09-12T03:18:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- 172.21.15.19:0/1553788343 shutdown_connections 2024-09-12T03:18:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 --2- 172.21.15.19:0/1553788343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 0x7fb5fc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 --2- 172.21.15.19:0/1553788343 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc103660 0x7fb5fc10bb50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 --2- 172.21.15.19:0/1553788343 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc102d40 0x7fb5fc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- 172.21.15.19:0/1553788343 >> 172.21.15.19:0/1553788343 conn(0x7fb5fc0fc7a0 msgr2=0x7fb5fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- 172.21.15.19:0/1553788343 shutdown_connections 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- 172.21.15.19:0/1553788343 wait complete. 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 Processor -- start 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.345+0000 7fb6026f9640 1 -- start start 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc103660 0x7fb5fc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 0x7fb5fc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc111f70 con 0x7fb5fc102d40 2024-09-12T03:18:04.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc111df0 con 0x7fb5fc10c170 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb6026f9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc1120f0 con 0x7fb5fc103660 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60272/0 (socket says 172.21.15.19:60272) 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 -- 172.21.15.19:0/2103135064 learned_addr learned my addr 172.21.15.19:0/2103135064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb600c96640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 0x7fb5fc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fb7fe640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc103660 0x7fb5fc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 -- 172.21.15.19:0/2103135064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc103660 msgr2=0x7fb5fc100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc103660 0x7fb5fc100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 -- 172.21.15.19:0/2103135064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 msgr2=0x7fb5fc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 0x7fb5fc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.346+0000 7fb5fbfff640 1 -- 172.21.15.19:0/2103135064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc1a6910 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb5fbfff640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00d950 tx=0x7fb5ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec017070 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5ec009c30 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc1a6c00 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00f6b0 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.347+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc1a7140 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.348+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5fc103120 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.349+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb5ec0109d0 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.350+0000 7fb5f97fa640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 0x7fb5e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.350+0000 7fb5fb7fe640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 0x7fb5e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.350+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb5ec0ced70 con 0x7fb5fc102d40 2024-09-12T03:18:04.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.350+0000 7fb5fb7fe640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 0x7fb5e40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc102430 tx=0x7fb5f0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.354+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5ec009de0 con 0x7fb5fc102d40 2024-09-12T03:18:04.452 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:04 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:04.522 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:04 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:18:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.530+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb5fc106c50 con 0x7fb5fc102d40 2024-09-12T03:18:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.531+0000 7fb5f97fa640 1 -- 172.21.15.19:0/2103135064 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb5ec09bc20 con 0x7fb5fc102d40 2024-09-12T03:18:04.531 INFO:teuthology.orchestra.run.smithi019.stdout:128849018941 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 msgr2=0x7fb5e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 0x7fb5e40a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc102430 tx=0x7fb5f0005a20 comp rx=0 tx=0).stop 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 msgr2=0x7fb5fc1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00d950 tx=0x7fb5ec00de10 comp rx=0 tx=0).stop 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 shutdown_connections 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb5fc10c170 0x7fb5fc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb5fc103660 0x7fb5fc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5e40a2c10 0x7fb5e40a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 --2- 172.21.15.19:0/2103135064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb5fc102d40 0x7fb5fc1009a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 >> 172.21.15.19:0/2103135064 conn(0x7fb5fc0fc7a0 msgr2=0x7fb5fc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 shutdown_connections 2024-09-12T03:18:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.533+0000 7fb6026f9640 1 -- 172.21.15.19:0/2103135064 wait complete. 2024-09-12T03:18:04.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 -- 172.21.15.19:0/460562605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 msgr2=0x7f84081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 --2- 172.21.15.19:0/460562605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f84081029b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f83f8009900 tx=0x7f83f802a480 comp rx=0 tx=0).stop 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 -- 172.21.15.19:0/460562605 shutdown_connections 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 --2- 172.21.15.19:0/460562605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408102ef0 0x7f840810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 --2- 172.21.15.19:0/460562605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f84081029b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 --2- 172.21.15.19:0/460562605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408100e20 0x7f8408101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 -- 172.21.15.19:0/460562605 >> 172.21.15.19:0/460562605 conn(0x7f84080fc7d0 msgr2=0x7f84080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 -- 172.21.15.19:0/460562605 shutdown_connections 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.682+0000 7f840fb99640 1 -- 172.21.15.19:0/460562605 wait complete. 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 Processor -- start 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 -- start start 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408100e20 0x7f8408199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 0x7f84081a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8408110f50 con 0x7f8408102570 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8408110dd0 con 0x7f8408102ef0 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840fb99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f84081110d0 con 0x7f8408100e20 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60288/0 (socket says 172.21.15.19:60288) 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 -- 172.21.15.19:0/3713208883 learned_addr learned my addr 172.21.15.19:0/3713208883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d935640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408100e20 0x7f8408199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840e136640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 0x7f84081a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 -- 172.21.15.19:0/3713208883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408100e20 msgr2=0x7f8408199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408100e20 0x7f8408199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 -- 172.21.15.19:0/3713208883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 msgr2=0x7f84081a43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.683+0000 7f840d134640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 0x7f84081a43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f840d134640 1 -- 172.21.15.19:0/3713208883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a4900 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f840e136640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 0x7f84081a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f840d134640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83f8006010 tx=0x7f83f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f803f070 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83f8031c30 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a4ad0 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8035030 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.684+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a4f20 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.685+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8408101200 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.686+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f83f8042070 con 0x7f8408102570 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.687+0000 7f83feffd640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 0x7f83f40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.687+0000 7f840d935640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 0x7f83f40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.687+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f83f80efcf0 con 0x7f8408102570 2024-09-12T03:18:04.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.687+0000 7f840d935640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 0x7f83f40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f840400bbe0 tx=0x7f840400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.690+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f83f8031de0 con 0x7f8408102570 2024-09-12T03:18:04.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.867+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8408109090 con 0x7f8408102570 2024-09-12T03:18:04.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.868+0000 7f83feffd640 1 -- 172.21.15.19:0/3713208883 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f83f80bc550 con 0x7f8408102570 2024-09-12T03:18:04.868 INFO:teuthology.orchestra.run.smithi019.stdout:180388626483 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 msgr2=0x7f83f40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 0x7f83f40a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f840400bbe0 tx=0x7f840400b390 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 msgr2=0x7f8408199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f83f8006010 tx=0x7f83f8003c10 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 shutdown_connections 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8408102ef0 0x7f84081a43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c10 0x7f83f40a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8408102570 0x7f8408199880 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 --2- 172.21.15.19:0/3713208883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8408100e20 0x7f8408199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 >> 172.21.15.19:0/3713208883 conn(0x7f84080fc7d0 msgr2=0x7f84080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 shutdown_connections 2024-09-12T03:18:04.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.870+0000 7f840fb99640 1 -- 172.21.15.19:0/3713208883 wait complete. 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 -- 172.21.15.19:0/1430068332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634102db0 msgr2=0x7f7634103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 --2- 172.21.15.19:0/1430068332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634102db0 0x7f7634103190 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7630009cd0 tx=0x7f763002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 -- 172.21.15.19:0/1430068332 shutdown_connections 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 --2- 172.21.15.19:0/1430068332 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76341082e0 0x7f763410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 --2- 172.21.15.19:0/1430068332 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7634103760 0x7f7634107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 --2- 172.21.15.19:0/1430068332 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634102db0 0x7f7634103190 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 -- 172.21.15.19:0/1430068332 >> 172.21.15.19:0/1430068332 conn(0x7f76340fc7d0 msgr2=0x7f76340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 -- 172.21.15.19:0/1430068332 shutdown_connections 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.882+0000 7f763baa8640 1 -- 172.21.15.19:0/1430068332 wait complete. 2024-09-12T03:18:04.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.883+0000 7f763baa8640 1 Processor -- start 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.883+0000 7f763baa8640 1 -- start start 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 0x7f76341985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 0x7f7634198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 0x7f763419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76341120b0 con 0x7f7634103760 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7634111f30 con 0x7f7634102db0 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763baa8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7634112230 con 0x7f76341082e0 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639043640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 0x7f7634198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 0x7f76341985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639043640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 0x7f7634198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60318/0 (socket says 172.21.15.19:60318) 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639043640 1 -- 172.21.15.19:0/3065949339 learned_addr learned my addr 172.21.15.19:0/3065949339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:04.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763a045640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 0x7f763419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 -- 172.21.15.19:0/3065949339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 msgr2=0x7f763419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 0x7f763419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 -- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 msgr2=0x7f7634198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 0x7f7634198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 -- 172.21.15.19:0/3065949339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763419d580 con 0x7f7634102db0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f763a045640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 0x7f763419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.884+0000 7f7639844640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 0x7f76341985d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7630000f80 tx=0x7f7630031eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.885+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630003e40 con 0x7f7634102db0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.885+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7630004520 con 0x7f7634102db0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.885+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763419d810 con 0x7f7634102db0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.885+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630030dc0 con 0x7f7634102db0 2024-09-12T03:18:04.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.885+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76341a5150 con 0x7f7634102db0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.886+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7634103190 con 0x7f7634102db0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.887+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7630004040 con 0x7f7634102db0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.888+0000 7f762affd640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 0x7f76200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.888+0000 7f7639043640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 0x7f76200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.889+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f763002e030 con 0x7f7634102db0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.889+0000 7f7639043640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 0x7f76200a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76340fede0 tx=0x7f7624005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:04.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:04.892+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f76300bd1a0 con 0x7f7634102db0 2024-09-12T03:18:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.074+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f763410a2c0 con 0x7f7634102db0 2024-09-12T03:18:05.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.075+0000 7f762affd640 1 -- 172.21.15.19:0/3065949339 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f76300bcaa0 con 0x7f7634102db0 2024-09-12T03:18:05.075 INFO:teuthology.orchestra.run.smithi019.stdout:34359738449 2024-09-12T03:18:05.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 msgr2=0x7f76200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 0x7f76200a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f76340fede0 tx=0x7f7624005f90 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 msgr2=0x7f76341985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 0x7f76341985d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7630000f80 tx=0x7f7630031eb0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 shutdown_connections 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76341082e0 0x7f763419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76200a2c90 0x7f76200a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7634103760 0x7f7634198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 --2- 172.21.15.19:0/3065949339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7634102db0 0x7f76341985d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.077+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 >> 172.21.15.19:0/3065949339 conn(0x7f76340fc7d0 msgr2=0x7f7634106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.078+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 shutdown_connections 2024-09-12T03:18:05.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.078+0000 7f763baa8640 1 -- 172.21.15.19:0/3065949339 wait complete. 2024-09-12T03:18:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2103135064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:18:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3713208883' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:18:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3065949339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:18:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:05 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2103135064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3713208883' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3065949339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:05 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626480 got 180388626483 for osd.6 2024-09-12T03:18:05.566 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: pgmap v306: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2103135064' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3713208883' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3065949339' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:05 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 -- 172.21.15.19:0/2410679759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 msgr2=0x7f27d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 --2- 172.21.15.19:0/2410679759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d01021c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009900 tx=0x7f27cc02a410 comp rx=0 tx=0).stop 2024-09-12T03:18:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 -- 172.21.15.19:0/2410679759 shutdown_connections 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 --2- 172.21.15.19:0/2410679759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0106f10 0x7f27d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 --2- 172.21.15.19:0/2410679759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0102790 0x7f27d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 --2- 172.21.15.19:0/2410679759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d01021c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 -- 172.21.15.19:0/2410679759 >> 172.21.15.19:0/2410679759 conn(0x7f27d00fd8c0 msgr2=0x7f27d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 -- 172.21.15.19:0/2410679759 shutdown_connections 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.846+0000 7f27d74f0640 1 -- 172.21.15.19:0/2410679759 wait complete. 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 Processor -- start 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 -- start start 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0102790 0x7f27d01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0106f10 0x7f27d019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d528c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.847+0000 7f27d74f0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0111f70 con 0x7f27d0101de0 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d74f0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27d0111df0 con 0x7f27d0102790 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d74f0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27d01120f0 con 0x7f27d0106f10 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60326/0 (socket says 172.21.15.19:60326) 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 -- 172.21.15.19:0/484742866 learned_addr learned my addr 172.21.15.19:0/484742866 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d5a8d640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0106f10 0x7f27d019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d4a8b640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0102790 0x7f27d01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 -- 172.21.15.19:0/484742866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0106f10 msgr2=0x7f27d019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0106f10 0x7f27d019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 -- 172.21.15.19:0/484742866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0102790 msgr2=0x7f27d01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0102790 0x7f27d01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 -- 172.21.15.19:0/484742866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27d019d410 con 0x7f27d0101de0 2024-09-12T03:18:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d528c640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27cc006010 tx=0x7f27cc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc03f030 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27cc003ce0 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc035070 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27d019d6a0 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.848+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27d01a4fe0 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.849+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27d01032c0 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.851+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27cc042030 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.851+0000 7f27c67fc640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 0x7f27bc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.851+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f27cc0ef880 con 0x7f27d0101de0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.851+0000 7f27d4a8b640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 0x7f27bc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.852+0000 7f27d4a8b640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 0x7f27bc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005fd0 tx=0x7f27c0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:05.855+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27cc0bc8f0 con 0x7f27d0101de0 2024-09-12T03:18:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.036+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f27d010a0f0 con 0x7f27d0101de0 2024-09-12T03:18:06.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.036+0000 7f27c67fc640 1 -- 172.21.15.19:0/484742866 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f27cc0bc1f0 con 0x7f27d0101de0 2024-09-12T03:18:06.037 INFO:teuthology.orchestra.run.smithi019.stdout:150323855416 2024-09-12T03:18:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 msgr2=0x7f27bc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 0x7f27bc0a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27c0005fd0 tx=0x7f27c0005850 comp rx=0 tx=0).stop 2024-09-12T03:18:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 msgr2=0x7f27d0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:06.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f27cc006010 tx=0x7f27cc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 shutdown_connections 2024-09-12T03:18:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27d0106f10 0x7f27d019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27d0102790 0x7f27d01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27bc0a2c10 0x7f27bc0a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 --2- 172.21.15.19:0/484742866 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27d0101de0 0x7f27d0198460 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 >> 172.21.15.19:0/484742866 conn(0x7f27d00fd8c0 msgr2=0x7f27d00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 shutdown_connections 2024-09-12T03:18:06.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:06.038+0000 7f27d74f0640 1 -- 172.21.15.19:0/484742866 wait complete. 2024-09-12T03:18:06.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:06 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/484742866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:18:06.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:06 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/484742866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:18:06.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:06 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/484742866' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T03:18:07.036 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738449 for osd.0 2024-09-12T03:18:07.036 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:07 smithi019 ceph-mon[30953]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:07.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 317827579926 got 317827579929 for osd.11 2024-09-12T03:18:07.465 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:07 smithi135 ceph-mon[35474]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:07.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:07 smithi121 ceph-mon[34963]: pgmap v307: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:07.906 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462955 got 201863462956 for osd.7 2024-09-12T03:18:07.906 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:09.010 INFO:teuthology.orchestra.run.smithi019.stdout:55834574923 2024-09-12T03:18:09.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T03:18:09.230 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:09 smithi019 ceph-mon[30953]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:09.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:09 smithi135 ceph-mon[35474]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:09.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:09 smithi121 ceph-mon[34963]: pgmap v308: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:09.914 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234022 got 231928234024 for osd.8 2024-09-12T03:18:09.914 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:09.933 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855412 got 150323855416 for osd.5 2024-09-12T03:18:09.933 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.065+0000 7ff05e3a5640 1 -- 172.21.15.19:0/4197773238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102ef0 msgr2=0x7ff05810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.065+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/4197773238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102ef0 0x7ff05810da40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff050009ca0 tx=0x7ff05002a950 comp rx=0 tx=0).stop 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 -- 172.21.15.19:0/4197773238 shutdown_connections 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/4197773238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102ef0 0x7ff05810da40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/4197773238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058102570 0x7ff0581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/4197773238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058100e20 0x7ff058101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 -- 172.21.15.19:0/4197773238 >> 172.21.15.19:0/4197773238 conn(0x7ff0580fc7d0 msgr2=0x7ff0580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:10.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 -- 172.21.15.19:0/4197773238 shutdown_connections 2024-09-12T03:18:10.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 -- 172.21.15.19:0/4197773238 wait complete. 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 Processor -- start 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.066+0000 7ff05e3a5640 1 -- start start 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058100e20 0x7ff058197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058102ef0 0x7ff05819bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff058110fc0 con 0x7ff058102570 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff058110e40 con 0x7ff058100e20 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05e3a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff058111140 con 0x7ff058102ef0 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:10.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60342/0 (socket says 172.21.15.19:60342) 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 -- 172.21.15.19:0/1431508449 learned_addr learned my addr 172.21.15.19:0/1431508449 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff057fff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058100e20 0x7ff058197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff05c942640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058102ef0 0x7ff05819bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 -- 172.21.15.19:0/1431508449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058102ef0 msgr2=0x7ff05819bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058102ef0 0x7ff05819bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 -- 172.21.15.19:0/1431508449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058100e20 msgr2=0x7ff058197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058100e20 0x7ff058197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 -- 172.21.15.19:0/1431508449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05819c3b0 con 0x7ff058102570 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.067+0000 7ff04ffff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b4a0 tx=0x7ff04000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.068+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040014070 con 0x7ff058102570 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.068+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05819c6a0 con 0x7ff058102570 2024-09-12T03:18:10.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.068+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff040010af0 con 0x7ff058102570 2024-09-12T03:18:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.068+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040017d10 con 0x7ff058102570 2024-09-12T03:18:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.068+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff058069f70 con 0x7ff058102570 2024-09-12T03:18:10.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.069+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff058101200 con 0x7ff058102570 2024-09-12T03:18:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.071+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff040017070 con 0x7ff058102570 2024-09-12T03:18:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.071+0000 7ff055ffb640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.072+0000 7ff057fff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.072+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff0400cee40 con 0x7ff058102570 2024-09-12T03:18:10.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.072+0000 7ff057fff640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 0x7ff0440a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff04800d5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:10.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.078+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff04009bf00 con 0x7ff058102570 2024-09-12T03:18:10.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.259+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff058198760 con 0x7ff058102570 2024-09-12T03:18:10.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.259+0000 7ff055ffb640 1 -- 172.21.15.19:0/1431508449 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff04009b800 con 0x7ff058102570 2024-09-12T03:18:10.259 INFO:teuthology.orchestra.run.smithi019.stdout:266287972387 2024-09-12T03:18:10.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 msgr2=0x7ff0440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 0x7ff0440a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff04800d5a0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 msgr2=0x7ff058197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff04000b4a0 tx=0x7ff04000b960 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 shutdown_connections 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff058102ef0 0x7ff05819bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff0440a2c10 0x7ff0440a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff058102570 0x7ff058197940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 --2- 172.21.15.19:0/1431508449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff058100e20 0x7ff058197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 >> 172.21.15.19:0/1431508449 conn(0x7ff0580fc7d0 msgr2=0x7ff058106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 shutdown_connections 2024-09-12T03:18:10.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:10.261+0000 7ff05e3a5640 1 -- 172.21.15.19:0/1431508449 wait complete. 2024-09-12T03:18:10.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247870 got 98784247871 for osd.3 2024-09-12T03:18:10.505 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:11.157 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972384 got 266287972387 for osd.9 2024-09-12T03:18:11.157 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:11 smithi019 ceph-mon[30953]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:11 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1431508449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:11 smithi135 ceph-mon[35474]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:11 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1431508449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:18:11.600 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:11.630 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018938 got 128849018941 for osd.4 2024-09-12T03:18:11.630 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:11.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:11 smithi121 ceph-mon[34963]: pgmap v309: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:11.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:11 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1431508449' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 -- 172.21.15.19:0/2037175017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c102d40 msgr2=0x7f6b1c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 --2- 172.21.15.19:0/2037175017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c102d40 0x7f6b1c103120 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009900 tx=0x7f6b0c02a480 comp rx=0 tx=0).stop 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 -- 172.21.15.19:0/2037175017 shutdown_connections 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 --2- 172.21.15.19:0/2037175017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1c10c170 0x7f6b1c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 --2- 172.21.15.19:0/2037175017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c103660 0x7f6b1c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 --2- 172.21.15.19:0/2037175017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c102d40 0x7f6b1c103120 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 -- 172.21.15.19:0/2037175017 >> 172.21.15.19:0/2037175017 conn(0x7f6b1c0fc7a0 msgr2=0x7f6b1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:12.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 -- 172.21.15.19:0/2037175017 shutdown_connections 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.091+0000 7f6b21895640 1 -- 172.21.15.19:0/2037175017 wait complete. 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 Processor -- start 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 -- start start 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c102d40 0x7f6b1c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1c103660 0x7f6b1c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111fb0 con 0x7f6b1c10c170 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c111e30 con 0x7f6b1c102d40 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b21895640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c112130 con 0x7f6b1c103660 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36234/0 (socket says 172.21.15.19:36234) 2024-09-12T03:18:12.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 -- 172.21.15.19:0/1142255298 learned_addr learned my addr 172.21.15.19:0/1142255298 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1affd640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c102d40 0x7f6b1c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 -- 172.21.15.19:0/1142255298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1c103660 msgr2=0x7f6b1c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1c103660 0x7f6b1c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 -- 172.21.15.19:0/1142255298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c102d40 msgr2=0x7f6b1c1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c102d40 0x7f6b1c1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.092+0000 7f6b1b7fe640 1 -- 172.21.15.19:0/1142255298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c19d460 con 0x7f6b1c10c170 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b1b7fe640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cad0 tx=0x7f6b1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400ae20 con 0x7f6b1c10c170 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b14005020 con 0x7f6b1c10c170 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c19d750 con 0x7f6b1c10c170 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400f6f0 con 0x7f6b1c10c170 2024-09-12T03:18:12.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.093+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c1a5030 con 0x7f6b1c10c170 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.094+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1c103120 con 0x7f6b1c10c170 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.095+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6b140051d0 con 0x7f6b1c10c170 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.096+0000 7f6b20893640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.096+0000 7f6b1affd640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.096+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6b140cf270 con 0x7f6b1c10c170 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.096+0000 7f6b1affd640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 0x7f6b040a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0096c0 tx=0x7f6b0c031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:12.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.100+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b14014030 con 0x7f6b1c10c170 2024-09-12T03:18:12.185 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:12.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.290+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f6b1c02d7c0 con 0x7f6b1c10c170 2024-09-12T03:18:12.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.291+0000 7f6b20893640 1 -- 172.21.15.19:0/1142255298 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6b1409c120 con 0x7f6b1c10c170 2024-09-12T03:18:12.291 INFO:teuthology.orchestra.run.smithi019.stdout:287762808862 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.292+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 msgr2=0x7f6b040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.292+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 0x7f6b040a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c0096c0 tx=0x7f6b0c031490 comp rx=0 tx=0).stop 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 msgr2=0x7f6b1c19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cad0 tx=0x7f6b1400cf90 comp rx=0 tx=0).stop 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 shutdown_connections 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6b040a2c10 0x7f6b040a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b1c10c170 0x7f6b1c19cd80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b1c103660 0x7f6b1c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 --2- 172.21.15.19:0/1142255298 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b1c102d40 0x7f6b1c1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 >> 172.21.15.19:0/1142255298 conn(0x7f6b1c0fc7a0 msgr2=0x7f6b1c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 shutdown_connections 2024-09-12T03:18:12.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:12.293+0000 7f6b21895640 1 -- 172.21.15.19:0/1142255298 wait complete. 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T03:18:12.407 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:12 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T03:18:12.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T03:18:12.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:12 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-09-12T03:18:12.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:12 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-09-12T03:18:13.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- 172.21.15.19:0/3414723297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c101e30 msgr2=0x7f127c105750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 --2- 172.21.15.19:0/3414723297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c101e30 0x7f127c105750 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1270009900 tx=0x7f127002a410 comp rx=0 tx=0).stop 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- 172.21.15.19:0/3414723297 shutdown_connections 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 --2- 172.21.15.19:0/3414723297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c105e80 0x7f127c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 --2- 172.21.15.19:0/3414723297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c101e30 0x7f127c105750 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 --2- 172.21.15.19:0/3414723297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 0x7f127c101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- 172.21.15.19:0/3414723297 >> 172.21.15.19:0/3414723297 conn(0x7f127c077210 msgr2=0x7f127c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- 172.21.15.19:0/3414723297 shutdown_connections 2024-09-12T03:18:13.184 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- 172.21.15.19:0/3414723297 wait complete. 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 Processor -- start 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.166+0000 7f128201f640 1 -- start start 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 0x7f127c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 0x7f127c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f127c110fc0 con 0x7f127c105e80 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f127c110e40 con 0x7f127c101480 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f128201f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f127c111140 con 0x7f127c101e30 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36250/0 (socket says 172.21.15.19:36250) 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127b7fe640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 0x7f127c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 -- 172.21.15.19:0/4229055166 learned_addr learned my addr 172.21.15.19:0/4229055166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:13.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127affd640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 0x7f127c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 -- 172.21.15.19:0/4229055166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 msgr2=0x7f127c0fcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 0x7f127c0fcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 -- 172.21.15.19:0/4229055166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 msgr2=0x7f127c0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 0x7f127c0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 -- 172.21.15.19:0/4229055166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c1a6a70 con 0x7f127c105e80 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127affd640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 0x7f127c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f127bfff640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f127400b970 tx=0x7f127400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.167+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1274019070 con 0x7f127c105e80 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.168+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c1a6d60 con 0x7f127c105e80 2024-09-12T03:18:13.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.168+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1274014440 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.168+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740075d0 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.168+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a72c0 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.169+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c101860 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.170+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f127401c070 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.170+0000 7f1278ff9640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 0x7f12640a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.171+0000 7f127b7fe640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 0x7f12640a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.171+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8211+0+0 (secure 0 0 0) 0x7f12740ce5d0 con 0x7f127c105e80 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.171+0000 7f127b7fe640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 0x7f12640a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f126c009990 tx=0x7f126c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:13.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.174+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f127409b4f0 con 0x7f127c105e80 2024-09-12T03:18:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.356+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f127c1a7550 con 0x7f127c105e80 2024-09-12T03:18:13.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.356+0000 7f1278ff9640 1 -- 172.21.15.19:0/4229055166 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f127409adf0 con 0x7f127c105e80 2024-09-12T03:18:13.357 INFO:teuthology.orchestra.run.smithi019.stdout:55834574925 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:18:13.357 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:18:13.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T03:18:13.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T03:18:13.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-12T03:18:13.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:18:13.358 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:13 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1142255298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:18:13.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 msgr2=0x7f12640a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:13.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 0x7f12640a5150 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f126c009990 tx=0x7f126c009110 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 msgr2=0x7f127c0fd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f127400b970 tx=0x7f127400bd70 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 shutdown_connections 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12640a2c90 0x7f12640a5150 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f127c105e80 0x7f127c0fd110 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f127c101e30 0x7f127c0fcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 --2- 172.21.15.19:0/4229055166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f127c101480 0x7f127c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 >> 172.21.15.19:0/4229055166 conn(0x7f127c077210 msgr2=0x7f127c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 shutdown_connections 2024-09-12T03:18:13.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:13.359+0000 7f128201f640 1 -- 172.21.15.19:0/4229055166 wait complete. 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:18:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T03:18:13.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T03:18:13.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-12T03:18:13.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:18:13.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:13 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1142255298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: pgmap v310: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T03:18:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:13 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1142255298' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T03:18:13.700 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808859 got 287762808862 for osd.10 2024-09-12T03:18:13.700 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:14.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:14 smithi019 ceph-mon[30953]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:18:14.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4229055166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:18:14.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574923 got 55834574925 for osd.1 2024-09-12T03:18:14.391 DEBUG:teuthology.parallel:result is None 2024-09-12T03:18:14.391 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:18:14.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:18:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:14 smithi135 ceph-mon[35474]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:18:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4229055166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:18:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:14 smithi121 ceph-mon[34963]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T03:18:14.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4229055166' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T03:18:14.893 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:15.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:15 smithi019 ceph-mon[30953]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T03:18:15.308 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:15 smithi019 ceph-mon[30953]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:18:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 -- 172.21.15.19:0/1978268574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101de0 msgr2=0x7fbff01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:15.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 --2- 172.21.15.19:0/1978268574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101de0 0x7fbff01021c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02a3f0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 -- 172.21.15.19:0/1978268574 shutdown_connections 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 --2- 172.21.15.19:0/1978268574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0106f10 0x7fbff010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 --2- 172.21.15.19:0/1978268574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0102790 0x7fbff01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 --2- 172.21.15.19:0/1978268574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0101de0 0x7fbff01021c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 -- 172.21.15.19:0/1978268574 >> 172.21.15.19:0/1978268574 conn(0x7fbff00fd8c0 msgr2=0x7fbff00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 -- 172.21.15.19:0/1978268574 shutdown_connections 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.391+0000 7fbff7a99640 1 -- 172.21.15.19:0/1978268574 wait complete. 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 Processor -- start 2024-09-12T03:18:15.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 -- start start 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 0x7fbff01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 0x7fbff01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111f90 con 0x7fbff0106f10 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0111e10 con 0x7fbff0102790 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff7a99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbff0112110 con 0x7fbff0101de0 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36254/0 (socket says 172.21.15.19:36254) 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 -- 172.21.15.19:0/3628228837 learned_addr learned my addr 172.21.15.19:0/3628228837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff5835640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 0x7fbff01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff5034640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 0x7fbff01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:15.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 -- 172.21.15.19:0/3628228837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 msgr2=0x7fbff01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 0x7fbff01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 -- 172.21.15.19:0/3628228837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 msgr2=0x7fbff01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 0x7fbff01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.392+0000 7fbff6036640 1 -- 172.21.15.19:0/3628228837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff019d460 con 0x7fbff0106f10 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbff5835640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 0x7fbff01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbff5034640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 0x7fbff01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbff6036640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cce0 tx=0x7fbfe800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe800f450 con 0x7fbff0106f10 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfe800fad0 con 0x7fbff0106f10 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff019d750 con 0x7fbff0106f10 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfe8018c10 con 0x7fbff0106f10 2024-09-12T03:18:15.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.393+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff01a5030 con 0x7fbff0106f10 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.394+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff01021c0 con 0x7fbff0106f10 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.395+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbfe800f5f0 con 0x7fbff0106f10 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.396+0000 7fbfe6ffd640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 0x7fbfdc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.396+0000 7fbff5835640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 0x7fbfdc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.396+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbfe80cfa20 con 0x7fbff0106f10 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.396+0000 7fbff5835640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 0x7fbfdc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:15.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.399+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbfe8014030 con 0x7fbff0106f10 2024-09-12T03:18:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:15 smithi135 ceph-mon[35474]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T03:18:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:15 smithi135 ceph-mon[35474]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:18:15.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.594+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbff0062630 con 0x7fbfdc0a2c10 2024-09-12T03:18:15.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.600+0000 7fbfe6ffd640 1 -- 172.21.15.19:0/3628228837 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258930 (secure 0 0 0) 0x7fbff0062630 con 0x7fbfdc0a2c10 2024-09-12T03:18:15.603 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:15.604 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 msgr2=0x7fbfdc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 0x7fbfdc0a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbfec000c00 tx=0x7fbfec032d50 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 msgr2=0x7fbff019cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbfe800cce0 tx=0x7fbfe800a620 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 shutdown_connections 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbfdc0a2c10 0x7fbfdc0a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbff0106f10 0x7fbff019cd80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbff0102790 0x7fbff01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 --2- 172.21.15.19:0/3628228837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbff0101de0 0x7fbff01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 >> 172.21.15.19:0/3628228837 conn(0x7fbff00fd8c0 msgr2=0x7fbff00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:15.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 shutdown_connections 2024-09-12T03:18:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:15.604+0000 7fbff7a99640 1 -- 172.21.15.19:0/3628228837 wait complete. 2024-09-12T03:18:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:15 smithi121 ceph-mon[34963]: pgmap v313: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 9 op/s 2024-09-12T03:18:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:15 smithi121 ceph-mon[34963]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-12T03:18:16.344 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-12T03:18:15.293787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":702,"num_read_kb":476,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356064,"kb_used_data":10948,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585760,"statfs":{"total":1151940427776,"available":1151575818240,"internally_reserved":0,"allocated":11210752,"data_stored":7093432,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001711"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910936+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:17:10.910936+0000","last_peered":"2024-09-12T03:17:10.910936+0000","last_clean":"2024-09-12T03:17:10.910936+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:17:10.910936+0000","last_undegraded":"2024-09-12T03:17:10.910936+0000","last_fullsized":"2024-09-12T03:17:10.910936+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919352+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:17:10.919352+0000","last_peered":"2024-09-12T03:17:10.919352+0000","last_clean":"2024-09-12T03:17:10.919352+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:17:10.919352+0000","last_undegraded":"2024-09-12T03:17:10.919352+0000","last_fullsized":"2024-09-12T03:17:10.919352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917721+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:17:10.917721+0000","last_peered":"2024-09-12T03:17:10.917721+0000","last_clean":"2024-09-12T03:17:10.917721+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:17:10.917721+0000","last_undegraded":"2024-09-12T03:17:10.917721+0000","last_fullsized":"2024-09-12T03:17:10.917721+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227583+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:13.227583+0000","last_peered":"2024-09-12T03:18:13.227583+0000","last_clean":"2024-09-12T03:18:13.227583+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:13.227583+0000","last_undegraded":"2024-09-12T03:18:13.227583+0000","last_fullsized":"2024-09-12T03:18:13.227583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223544+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:13.223544+0000","last_peered":"2024-09-12T03:18:13.223544+0000","last_clean":"2024-09-12T03:18:13.223544+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:13.223544+0000","last_undegraded":"2024-09-12T03:18:13.223544+0000","last_fullsized":"2024-09-12T03:18:13.223544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221903+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:13.221903+0000","last_peered":"2024-09-12T03:18:13.221903+0000","last_clean":"2024-09-12T03:18:13.221903+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:13.221903+0000","last_undegraded":"2024-09-12T03:18:13.221903+0000","last_fullsized":"2024-09-12T03:18:13.221903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919834+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:17:10.919834+0000","last_peered":"2024-09-12T03:17:10.919834+0000","last_clean":"2024-09-12T03:17:10.919834+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:17:10.919834+0000","last_undegraded":"2024-09-12T03:17:10.919834+0000","last_fullsized":"2024-09-12T03:17:10.919834+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227222+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:13.227222+0000","last_peered":"2024-09-12T03:18:13.227222+0000","last_clean":"2024-09-12T03:18:13.227222+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:13.227222+0000","last_undegraded":"2024-09-12T03:18:13.227222+0000","last_fullsized":"2024-09-12T03:18:13.227222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910435+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:17:10.910435+0000","last_peered":"2024-09-12T03:17:10.910435+0000","last_clean":"2024-09-12T03:17:10.910435+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:17:10.910435+0000","last_undegraded":"2024-09-12T03:17:10.910435+0000","last_fullsized":"2024-09-12T03:17:10.910435+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222751+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:13.222751+0000","last_peered":"2024-09-12T03:18:13.222751+0000","last_clean":"2024-09-12T03:18:13.222751+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:13.222751+0000","last_undegraded":"2024-09-12T03:18:13.222751+0000","last_fullsized":"2024-09-12T03:18:13.222751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920686+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:17:10.920686+0000","last_peered":"2024-09-12T03:17:10.920686+0000","last_clean":"2024-09-12T03:17:10.920686+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:17:10.920686+0000","last_undegraded":"2024-09-12T03:17:10.920686+0000","last_fullsized":"2024-09-12T03:17:10.920686+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916574+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:17:10.916574+0000","last_peered":"2024-09-12T03:17:10.916574+0000","last_clean":"2024-09-12T03:17:10.916574+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:17:10.916574+0000","last_undegraded":"2024-09-12T03:17:10.916574+0000","last_fullsized":"2024-09-12T03:17:10.916574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221504+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:12.221504+0000","last_peered":"2024-09-12T03:18:12.221504+0000","last_clean":"2024-09-12T03:18:12.221504+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:12.221504+0000","last_undegraded":"2024-09-12T03:18:12.221504+0000","last_fullsized":"2024-09-12T03:18:12.221504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.918076+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:17:10.918076+0000","last_peered":"2024-09-12T03:17:10.918076+0000","last_clean":"2024-09-12T03:17:10.918076+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:17:10.918076+0000","last_undegraded":"2024-09-12T03:17:10.918076+0000","last_fullsized":"2024-09-12T03:17:10.918076+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920627+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:17:10.920627+0000","last_peered":"2024-09-12T03:17:10.920627+0000","last_clean":"2024-09-12T03:17:10.920627+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:17:10.920627+0000","last_undegraded":"2024-09-12T03:17:10.920627+0000","last_fullsized":"2024-09-12T03:17:10.920627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.909906+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:17:10.909906+0000","last_peered":"2024-09-12T03:17:10.909906+0000","last_clean":"2024-09-12T03:17:10.909906+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:17:10.909906+0000","last_undegraded":"2024-09-12T03:17:10.909906+0000","last_fullsized":"2024-09-12T03:17:10.909906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.911086+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:17:10.911086+0000","last_peered":"2024-09-12T03:17:10.911086+0000","last_clean":"2024-09-12T03:17:10.911086+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:17:10.911086+0000","last_undegraded":"2024-09-12T03:17:10.911086+0000","last_fullsized":"2024-09-12T03:17:10.911086+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222851+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:13.222851+0000","last_peered":"2024-09-12T03:18:13.222851+0000","last_clean":"2024-09-12T03:18:13.222851+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:13.222851+0000","last_undegraded":"2024-09-12T03:18:13.222851+0000","last_fullsized":"2024-09-12T03:18:13.222851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916130+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:17:10.916130+0000","last_peered":"2024-09-12T03:17:10.916130+0000","last_clean":"2024-09-12T03:17:10.916130+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:17:10.916130+0000","last_undegraded":"2024-09-12T03:17:10.916130+0000","last_fullsized":"2024-09-12T03:17:10.916130+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910819+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:17:10.910819+0000","last_peered":"2024-09-12T03:17:10.910819+0000","last_clean":"2024-09-12T03:17:10.910819+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:17:10.910819+0000","last_undegraded":"2024-09-12T03:17:10.910819+0000","last_fullsized":"2024-09-12T03:17:10.910819+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221559+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:12.221559+0000","last_peered":"2024-09-12T03:18:12.221559+0000","last_clean":"2024-09-12T03:18:12.221559+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:12.221559+0000","last_undegraded":"2024-09-12T03:18:12.221559+0000","last_fullsized":"2024-09-12T03:18:12.221559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910084+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:17:10.910084+0000","last_peered":"2024-09-12T03:17:10.910084+0000","last_clean":"2024-09-12T03:17:10.910084+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:17:10.910084+0000","last_undegraded":"2024-09-12T03:17:10.910084+0000","last_fullsized":"2024-09-12T03:17:10.910084+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227500+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:13.227500+0000","last_peered":"2024-09-12T03:18:13.227500+0000","last_clean":"2024-09-12T03:18:13.227500+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:13.227500+0000","last_undegraded":"2024-09-12T03:18:13.227500+0000","last_fullsized":"2024-09-12T03:18:13.227500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229154+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:12.229154+0000","last_peered":"2024-09-12T03:18:12.229154+0000","last_clean":"2024-09-12T03:18:12.229154+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:12.229154+0000","last_undegraded":"2024-09-12T03:18:12.229154+0000","last_fullsized":"2024-09-12T03:18:12.229154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228434+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:13.228434+0000","last_peered":"2024-09-12T03:18:13.228434+0000","last_clean":"2024-09-12T03:18:13.228434+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:13.228434+0000","last_undegraded":"2024-09-12T03:18:13.228434+0000","last_fullsized":"2024-09-12T03:18:13.228434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.916458+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:17:10.916458+0000","last_peered":"2024-09-12T03:17:10.916458+0000","last_clean":"2024-09-12T03:17:10.916458+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:17:10.916458+0000","last_undegraded":"2024-09-12T03:17:10.916458+0000","last_fullsized":"2024-09-12T03:17:10.916458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910708+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:17:10.910708+0000","last_peered":"2024-09-12T03:17:10.910708+0000","last_clean":"2024-09-12T03:17:10.910708+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:17:10.910708+0000","last_undegraded":"2024-09-12T03:17:10.910708+0000","last_fullsized":"2024-09-12T03:17:10.910708+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221524+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:12.221524+0000","last_peered":"2024-09-12T03:18:12.221524+0000","last_clean":"2024-09-12T03:18:12.221524+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:12.221524+0000","last_undegraded":"2024-09-12T03:18:12.221524+0000","last_fullsized":"2024-09-12T03:18:12.221524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.917962+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:17:10.917962+0000","last_peered":"2024-09-12T03:17:10.917962+0000","last_clean":"2024-09-12T03:17:10.917962+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:17:10.917962+0000","last_undegraded":"2024-09-12T03:17:10.917962+0000","last_fullsized":"2024-09-12T03:17:10.917962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915844+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:17:10.915844+0000","last_peered":"2024-09-12T03:17:10.915844+0000","last_clean":"2024-09-12T03:17:10.915844+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:17:10.915844+0000","last_undegraded":"2024-09-12T03:17:10.915844+0000","last_fullsized":"2024-09-12T03:17:10.915844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.915971+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:17:10.915971+0000","last_peered":"2024-09-12T03:17:10.915971+0000","last_clean":"2024-09-12T03:17:10.915971+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:17:10.915971+0000","last_undegraded":"2024-09-12T03:17:10.915971+0000","last_fullsized":"2024-09-12T03:17:10.915971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910616+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:17:10.910616+0000","last_peered":"2024-09-12T03:17:10.910616+0000","last_clean":"2024-09-12T03:17:10.910616+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:17:10.910616+0000","last_undegraded":"2024-09-12T03:17:10.910616+0000","last_fullsized":"2024-09-12T03:17:10.910616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223333+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:13.223333+0000","last_peered":"2024-09-12T03:18:13.223333+0000","last_clean":"2024-09-12T03:18:13.223333+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:13.223333+0000","last_undegraded":"2024-09-12T03:18:13.223333+0000","last_fullsized":"2024-09-12T03:18:13.223333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229744+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:18:12.229744+0000","last_peered":"2024-09-12T03:18:12.229744+0000","last_clean":"2024-09-12T03:18:12.229744+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:18:12.229744+0000","last_undegraded":"2024-09-12T03:18:12.229744+0000","last_fullsized":"2024-09-12T03:18:12.229744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229805+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:12.229805+0000","last_peered":"2024-09-12T03:18:12.229805+0000","last_clean":"2024-09-12T03:18:12.229805+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:12.229805+0000","last_undegraded":"2024-09-12T03:18:12.229805+0000","last_fullsized":"2024-09-12T03:18:12.229805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221284+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:12.221284+0000","last_peered":"2024-09-12T03:18:12.221284+0000","last_clean":"2024-09-12T03:18:12.221284+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:12.221284+0000","last_undegraded":"2024-09-12T03:18:12.221284+0000","last_fullsized":"2024-09-12T03:18:12.221284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227325+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:13.227325+0000","last_peered":"2024-09-12T03:18:13.227325+0000","last_clean":"2024-09-12T03:18:13.227325+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:13.227325+0000","last_undegraded":"2024-09-12T03:18:13.227325+0000","last_fullsized":"2024-09-12T03:18:13.227325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229332+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:12.229332+0000","last_peered":"2024-09-12T03:18:12.229332+0000","last_clean":"2024-09-12T03:18:12.229332+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:12.229332+0000","last_undegraded":"2024-09-12T03:18:12.229332+0000","last_fullsized":"2024-09-12T03:18:12.229332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222614+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:13.222614+0000","last_peered":"2024-09-12T03:18:13.222614+0000","last_clean":"2024-09-12T03:18:13.222614+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:13.222614+0000","last_undegraded":"2024-09-12T03:18:13.222614+0000","last_fullsized":"2024-09-12T03:18:13.222614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.233405+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:18:14.233405+0000","last_peered":"2024-09-12T03:18:14.233405+0000","last_clean":"2024-09-12T03:18:14.233405+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:18:14.233405+0000","last_undegraded":"2024-09-12T03:18:14.233405+0000","last_fullsized":"2024-09-12T03:18:14.233405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910386+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:17:10.910386+0000","last_peered":"2024-09-12T03:17:10.910386+0000","last_clean":"2024-09-12T03:17:10.910386+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:17:10.910386+0000","last_undegraded":"2024-09-12T03:17:10.910386+0000","last_fullsized":"2024-09-12T03:17:10.910386+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830705+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:17:20.830705+0000","last_peered":"2024-09-12T03:17:20.830705+0000","last_clean":"2024-09-12T03:17:20.830705+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:17:20.830705+0000","last_undegraded":"2024-09-12T03:17:20.830705+0000","last_fullsized":"2024-09-12T03:17:20.830705+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920111+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:17:10.920111+0000","last_peered":"2024-09-12T03:17:10.920111+0000","last_clean":"2024-09-12T03:17:10.920111+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:17:10.920111+0000","last_undegraded":"2024-09-12T03:17:10.920111+0000","last_fullsized":"2024-09-12T03:17:10.920111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229822+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:12.229822+0000","last_peered":"2024-09-12T03:18:12.229822+0000","last_clean":"2024-09-12T03:18:12.229822+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:12.229822+0000","last_undegraded":"2024-09-12T03:18:12.229822+0000","last_fullsized":"2024-09-12T03:18:12.229822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910273+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:17:10.910273+0000","last_peered":"2024-09-12T03:17:10.910273+0000","last_clean":"2024-09-12T03:17:10.910273+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:17:10.910273+0000","last_undegraded":"2024-09-12T03:17:10.910273+0000","last_fullsized":"2024-09-12T03:17:10.910273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.891326+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:18:14.891326+0000","last_peered":"2024-09-12T03:18:14.891326+0000","last_clean":"2024-09-12T03:18:14.891326+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:18:14.891326+0000","last_undegraded":"2024-09-12T03:18:14.891326+0000","last_fullsized":"2024-09-12T03:18:14.891326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":118,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223911+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:13.223911+0000","last_peered":"2024-09-12T03:18:13.223911+0000","last_clean":"2024-09-12T03:18:13.223911+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:13.223911+0000","last_undegraded":"2024-09-12T03:18:13.223911+0000","last_fullsized":"2024-09-12T03:18:13.223911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":29,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.830417+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:17:20.830417+0000","last_peered":"2024-09-12T03:17:20.830417+0000","last_clean":"2024-09-12T03:17:20.830417+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:17:20.830417+0000","last_undegraded":"2024-09-12T03:17:20.830417+0000","last_fullsized":"2024-09-12T03:17:20.830417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221379+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:12.221379+0000","last_peered":"2024-09-12T03:18:12.221379+0000","last_clean":"2024-09-12T03:18:12.221379+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:12.221379+0000","last_undegraded":"2024-09-12T03:18:12.221379+0000","last_fullsized":"2024-09-12T03:18:12.221379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229511+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:12.229511+0000","last_peered":"2024-09-12T03:18:12.229511+0000","last_clean":"2024-09-12T03:18:12.229511+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:12.229511+0000","last_undegraded":"2024-09-12T03:18:12.229511+0000","last_fullsized":"2024-09-12T03:18:12.229511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920453+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:17:10.920453+0000","last_peered":"2024-09-12T03:17:10.920453+0000","last_clean":"2024-09-12T03:17:10.920453+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:17:10.920453+0000","last_undegraded":"2024-09-12T03:17:10.920453+0000","last_fullsized":"2024-09-12T03:17:10.920453+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":80,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:18:10.886031+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:18:10.886031+0000","last_peered":"2024-09-12T03:18:10.886031+0000","last_clean":"2024-09-12T03:18:10.886031+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:18:10.886031+0000","last_undegraded":"2024-09-12T03:18:10.886031+0000","last_fullsized":"2024-09-12T03:18:10.886031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":57,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919772+0000","last_change":"2024-09-12T03:17:09.928028+0000","last_active":"2024-09-12T03:17:10.919772+0000","last_peered":"2024-09-12T03:17:10.919772+0000","last_clean":"2024-09-12T03:17:10.919772+0000","last_became_active":"2024-09-12T03:17:09.927706+0000","last_became_peered":"2024-09-12T03:17:09.927706+0000","last_unstale":"2024-09-12T03:17:10.919772+0000","last_undegraded":"2024-09-12T03:17:10.919772+0000","last_fullsized":"2024-09-12T03:17:10.919772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:09:39.496602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229248+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:12.229248+0000","last_peered":"2024-09-12T03:18:12.229248+0000","last_clean":"2024-09-12T03:18:12.229248+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:12.229248+0000","last_undegraded":"2024-09-12T03:18:12.229248+0000","last_fullsized":"2024-09-12T03:18:12.229248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.814808+0000","last_change":"2024-09-12T03:17:07.831968+0000","last_active":"2024-09-12T03:17:19.814808+0000","last_peered":"2024-09-12T03:17:19.814808+0000","last_clean":"2024-09-12T03:17:19.814808+0000","last_became_active":"2024-09-12T03:17:07.828479+0000","last_became_peered":"2024-09-12T03:17:07.828479+0000","last_unstale":"2024-09-12T03:17:19.814808+0000","last_undegraded":"2024-09-12T03:17:19.814808+0000","last_fullsized":"2024-09-12T03:17:19.814808+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:12:55.104152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910685+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:17:10.910685+0000","last_peered":"2024-09-12T03:17:10.910685+0000","last_clean":"2024-09-12T03:17:10.910685+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:17:10.910685+0000","last_undegraded":"2024-09-12T03:17:10.910685+0000","last_fullsized":"2024-09-12T03:17:10.910685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221726+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:12.221726+0000","last_peered":"2024-09-12T03:18:12.221726+0000","last_clean":"2024-09-12T03:18:12.221726+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:12.221726+0000","last_undegraded":"2024-09-12T03:18:12.221726+0000","last_fullsized":"2024-09-12T03:18:12.221726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920448+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:17:10.920448+0000","last_peered":"2024-09-12T03:17:10.920448+0000","last_clean":"2024-09-12T03:17:10.920448+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:17:10.920448+0000","last_undegraded":"2024-09-12T03:17:10.920448+0000","last_fullsized":"2024-09-12T03:17:10.920448+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229949+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:12.229949+0000","last_peered":"2024-09-12T03:18:12.229949+0000","last_clean":"2024-09-12T03:18:12.229949+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:12.229949+0000","last_undegraded":"2024-09-12T03:18:12.229949+0000","last_fullsized":"2024-09-12T03:18:12.229949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:12.130685+0000","last_change":"2024-09-12T03:17:09.925753+0000","last_active":"2024-09-12T03:17:12.130685+0000","last_peered":"2024-09-12T03:17:12.130685+0000","last_clean":"2024-09-12T03:17:12.130685+0000","last_became_active":"2024-09-12T03:17:09.925426+0000","last_became_peered":"2024-09-12T03:17:09.925426+0000","last_unstale":"2024-09-12T03:17:12.130685+0000","last_undegraded":"2024-09-12T03:17:12.130685+0000","last_fullsized":"2024-09-12T03:17:12.130685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:51:13.548318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":74,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:18:08.902500+0000","last_change":"2024-09-12T03:17:07.981943+0000","last_active":"2024-09-12T03:18:08.902500+0000","last_peered":"2024-09-12T03:18:08.902500+0000","last_clean":"2024-09-12T03:18:08.902500+0000","last_became_active":"2024-09-12T03:17:07.981581+0000","last_became_peered":"2024-09-12T03:17:07.981581+0000","last_unstale":"2024-09-12T03:18:08.902500+0000","last_undegraded":"2024-09-12T03:18:08.902500+0000","last_fullsized":"2024-09-12T03:18:08.902500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:07:49.351106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920766+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:17:10.920766+0000","last_peered":"2024-09-12T03:17:10.920766+0000","last_clean":"2024-09-12T03:17:10.920766+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:17:10.920766+0000","last_undegraded":"2024-09-12T03:17:10.920766+0000","last_fullsized":"2024-09-12T03:17:10.920766+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":334,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919845+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:17:10.919845+0000","last_peered":"2024-09-12T03:17:10.919845+0000","last_clean":"2024-09-12T03:17:10.919845+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:17:10.919845+0000","last_undegraded":"2024-09-12T03:17:10.919845+0000","last_fullsized":"2024-09-12T03:17:10.919845+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":27,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832258+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:17:20.832258+0000","last_peered":"2024-09-12T03:17:20.832258+0000","last_clean":"2024-09-12T03:17:20.832258+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:17:20.832258+0000","last_undegraded":"2024-09-12T03:17:20.832258+0000","last_fullsized":"2024-09-12T03:17:20.832258+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227978+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:13.227978+0000","last_peered":"2024-09-12T03:18:13.227978+0000","last_clean":"2024-09-12T03:18:13.227978+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:13.227978+0000","last_undegraded":"2024-09-12T03:18:13.227978+0000","last_fullsized":"2024-09-12T03:18:13.227978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":22,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:18.964618+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:17:18.964618+0000","last_peered":"2024-09-12T03:17:18.964618+0000","last_clean":"2024-09-12T03:17:18.964618+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:17:18.964618+0000","last_undegraded":"2024-09-12T03:17:18.964618+0000","last_fullsized":"2024-09-12T03:17:18.964618+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221272+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:12.221272+0000","last_peered":"2024-09-12T03:18:12.221272+0000","last_clean":"2024-09-12T03:18:12.221272+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:12.221272+0000","last_undegraded":"2024-09-12T03:18:12.221272+0000","last_fullsized":"2024-09-12T03:18:12.221272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221476+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:18:12.221476+0000","last_peered":"2024-09-12T03:18:12.221476+0000","last_clean":"2024-09-12T03:18:12.221476+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:18:12.221476+0000","last_undegraded":"2024-09-12T03:18:12.221476+0000","last_fullsized":"2024-09-12T03:18:12.221476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229219+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:12.229219+0000","last_peered":"2024-09-12T03:18:12.229219+0000","last_clean":"2024-09-12T03:18:12.229219+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:12.229219+0000","last_undegraded":"2024-09-12T03:18:12.229219+0000","last_fullsized":"2024-09-12T03:18:12.229219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":75,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:18:10.910546+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:18:10.910546+0000","last_peered":"2024-09-12T03:18:10.910546+0000","last_clean":"2024-09-12T03:18:10.910546+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:18:10.910546+0000","last_undegraded":"2024-09-12T03:18:10.910546+0000","last_fullsized":"2024-09-12T03:18:10.910546+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221810+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:13.221810+0000","last_peered":"2024-09-12T03:18:13.221810+0000","last_clean":"2024-09-12T03:18:13.221810+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:13.221810+0000","last_undegraded":"2024-09-12T03:18:13.221810+0000","last_fullsized":"2024-09-12T03:18:13.221810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223007+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223007+0000","last_peered":"2024-09-12T03:18:13.223007+0000","last_clean":"2024-09-12T03:18:13.223007+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223007+0000","last_undegraded":"2024-09-12T03:18:13.223007+0000","last_fullsized":"2024-09-12T03:18:13.223007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222321+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:13.222321+0000","last_peered":"2024-09-12T03:18:13.222321+0000","last_clean":"2024-09-12T03:18:13.222321+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:13.222321+0000","last_undegraded":"2024-09-12T03:18:13.222321+0000","last_fullsized":"2024-09-12T03:18:13.222321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224119+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224119+0000","last_peered":"2024-09-12T03:18:13.224119+0000","last_clean":"2024-09-12T03:18:13.224119+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224119+0000","last_undegraded":"2024-09-12T03:18:13.224119+0000","last_fullsized":"2024-09-12T03:18:13.224119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221604+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:12.221604+0000","last_peered":"2024-09-12T03:18:12.221604+0000","last_clean":"2024-09-12T03:18:12.221604+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:12.221604+0000","last_undegraded":"2024-09-12T03:18:12.221604+0000","last_fullsized":"2024-09-12T03:18:12.221604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222863+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:13.222863+0000","last_peered":"2024-09-12T03:18:13.222863+0000","last_clean":"2024-09-12T03:18:13.222863+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:13.222863+0000","last_undegraded":"2024-09-12T03:18:13.222863+0000","last_fullsized":"2024-09-12T03:18:13.222863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":26,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.832160+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:17:20.832160+0000","last_peered":"2024-09-12T03:17:20.832160+0000","last_clean":"2024-09-12T03:17:20.832160+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:17:20.832160+0000","last_undegraded":"2024-09-12T03:17:20.832160+0000","last_fullsized":"2024-09-12T03:17:20.832160+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919501+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:17:10.919501+0000","last_peered":"2024-09-12T03:17:10.919501+0000","last_clean":"2024-09-12T03:17:10.919501+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:17:10.919501+0000","last_undegraded":"2024-09-12T03:17:10.919501+0000","last_fullsized":"2024-09-12T03:17:10.919501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:17.957416+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:17:17.957416+0000","last_peered":"2024-09-12T03:17:17.957416+0000","last_clean":"2024-09-12T03:17:17.957416+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:17:17.957416+0000","last_undegraded":"2024-09-12T03:17:17.957416+0000","last_fullsized":"2024-09-12T03:17:17.957416+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223193+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:13.223193+0000","last_peered":"2024-09-12T03:18:13.223193+0000","last_clean":"2024-09-12T03:18:13.223193+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:13.223193+0000","last_undegraded":"2024-09-12T03:18:13.223193+0000","last_fullsized":"2024-09-12T03:18:13.223193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910510+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:17:10.910510+0000","last_peered":"2024-09-12T03:17:10.910510+0000","last_clean":"2024-09-12T03:17:10.910510+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:17:10.910510+0000","last_undegraded":"2024-09-12T03:17:10.910510+0000","last_fullsized":"2024-09-12T03:17:10.910510+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":21,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:19.953207+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:17:19.953207+0000","last_peered":"2024-09-12T03:17:19.953207+0000","last_clean":"2024-09-12T03:17:19.953207+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:17:19.953207+0000","last_undegraded":"2024-09-12T03:17:19.953207+0000","last_fullsized":"2024-09-12T03:17:19.953207+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228053+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:13.228053+0000","last_peered":"2024-09-12T03:18:13.228053+0000","last_clean":"2024-09-12T03:18:13.228053+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:13.228053+0000","last_undegraded":"2024-09-12T03:18:13.228053+0000","last_fullsized":"2024-09-12T03:18:13.228053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.920866+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:17:10.920866+0000","last_peered":"2024-09-12T03:17:10.920866+0000","last_clean":"2024-09-12T03:17:10.920866+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:17:10.920866+0000","last_undegraded":"2024-09-12T03:17:10.920866+0000","last_fullsized":"2024-09-12T03:17:10.920866+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"peering","last_fresh":"2024-09-12T03:17:13.121382+0000","last_change":"2024-09-12T03:18:12.228851+0000","last_active":"2024-09-12T03:18:12.228851+0000","last_peered":"2024-09-12T03:17:13.121382+0000","last_clean":"2024-09-12T03:17:13.121382+0000","last_became_active":"2024-09-12T03:17:09.921491+0000","last_became_peered":"2024-09-12T03:17:09.921491+0000","last_unstale":"2024-09-12T03:17:13.121382+0000","last_undegraded":"2024-09-12T03:17:13.121382+0000","last_fullsized":"2024-09-12T03:17:13.121382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":24,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:20.831969+0000","last_change":"2024-09-12T03:17:07.830656+0000","last_active":"2024-09-12T03:17:20.831969+0000","last_peered":"2024-09-12T03:17:20.831969+0000","last_clean":"2024-09-12T03:17:20.831969+0000","last_became_active":"2024-09-12T03:17:07.828631+0000","last_became_peered":"2024-09-12T03:17:07.828631+0000","last_unstale":"2024-09-12T03:17:20.831969+0000","last_undegraded":"2024-09-12T03:17:20.831969+0000","last_fullsized":"2024-09-12T03:17:20.831969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:37:44.617551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.910111+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:17:10.910111+0000","last_peered":"2024-09-12T03:17:10.910111+0000","last_clean":"2024-09-12T03:17:10.910111+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:17:10.910111+0000","last_undegraded":"2024-09-12T03:17:10.910111+0000","last_fullsized":"2024-09-12T03:17:10.910111+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-09-12T03:17:10.919966+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:17:10.919966+0000","last_peered":"2024-09-12T03:17:10.919966+0000","last_clean":"2024-09-12T03:17:10.919966+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:17:10.919966+0000","last_undegraded":"2024-09-12T03:17:10.919966+0000","last_fullsized":"2024-09-12T03:17:10.919966+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228427+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228427+0000","last_peered":"2024-09-12T03:18:13.228427+0000","last_clean":"2024-09-12T03:18:13.228427+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228427+0000","last_undegraded":"2024-09-12T03:18:13.228427+0000","last_fullsized":"2024-09-12T03:18:13.228427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228289+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:13.228289+0000","last_peered":"2024-09-12T03:18:13.228289+0000","last_clean":"2024-09-12T03:18:13.228289+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:13.228289+0000","last_undegraded":"2024-09-12T03:18:13.228289+0000","last_fullsized":"2024-09-12T03:18:13.228289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.913793+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:18:13.913793+0000","last_peered":"2024-09-12T03:18:13.913793+0000","last_clean":"2024-09-12T03:18:13.913793+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:18:13.913793+0000","last_undegraded":"2024-09-12T03:18:13.913793+0000","last_fullsized":"2024-09-12T03:18:13.913793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":576,"num_read_kb":367,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":200312,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579931,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":420827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":1089115,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":1014048,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":429117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855418,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1021976,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":420728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444105,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":481764,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32104,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713048,"statfs":{"total":95995035648,"available":95962161152,"internally_reserved":0,"allocated":942080,"data_stored":481151,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":905216,"data_stored":421813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61516,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":62129,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61893,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:18:16.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:18:16.650 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:18:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:16.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:16.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:16.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:16.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:17.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:18:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:18:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.142+0000 7f2747f14640 1 -- 172.21.15.19:0/2023701583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101e10 msgr2=0x7f27401021f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.142+0000 7f2747f14640 1 --2- 172.21.15.19:0/2023701583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101e10 0x7f27401021f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2738009900 tx=0x7f273802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 -- 172.21.15.19:0/2023701583 shutdown_connections 2024-09-12T03:18:17.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 --2- 172.21.15.19:0/2023701583 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740106f40 0x7f274010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 --2- 172.21.15.19:0/2023701583 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27401027c0 0x7f2740106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 --2- 172.21.15.19:0/2023701583 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2740101e10 0x7f27401021f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 -- 172.21.15.19:0/2023701583 >> 172.21.15.19:0/2023701583 conn(0x7f27400fd8d0 msgr2=0x7f27400ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 -- 172.21.15.19:0/2023701583 shutdown_connections 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 -- 172.21.15.19:0/2023701583 wait complete. 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 Processor -- start 2024-09-12T03:18:17.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 -- start start 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740101e10 0x7f2740198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.143+0000 7f2747f14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 0x7f274019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2740111fa0 con 0x7f27401027c0 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2740111e20 con 0x7f2740106f40 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2740112120 con 0x7f2740101e10 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36270/0 (socket says 172.21.15.19:36270) 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 -- 172.21.15.19:0/2824443388 learned_addr learned my addr 172.21.15.19:0/2824443388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2745cb0640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740101e10 0x7f2740198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27464b1640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 0x7f274019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 -- 172.21.15.19:0/2824443388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740101e10 msgr2=0x7f2740198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740101e10 0x7f2740198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 -- 172.21.15.19:0/2824443388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 msgr2=0x7f274019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 0x7f274019cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 -- 172.21.15.19:0/2824443388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274019d3e0 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27464b1640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 0x7f274019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f27454af640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f273400d950 tx=0x7f273400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2734017030 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2734004020 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274019d6d0 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27340047a0 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.144+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27401a4fb0 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.145+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27401021f0 con 0x7f27401027c0 2024-09-12T03:18:17.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.147+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f27340041d0 con 0x7f27401027c0 2024-09-12T03:18:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.147+0000 7f273effd640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.147+0000 7f2745cb0640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.147+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27340cf220 con 0x7f27401027c0 2024-09-12T03:18:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.148+0000 7f2745cb0640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 0x7f27280a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2738009230 tx=0x7f27380058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:17.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.151+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f273409c030 con 0x7f27401027c0 2024-09-12T03:18:17.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.324+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2740062630 con 0x7f27280a2c10 2024-09-12T03:18:17.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.330+0000 7f273effd640 1 -- 172.21.15.19:0/2824443388 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258956 (secure 0 0 0) 0x7f2740062630 con 0x7f27280a2c10 2024-09-12T03:18:17.332 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:17.333 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:18:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 msgr2=0x7f27280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 0x7f27280a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2738009230 tx=0x7f27380058d0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 msgr2=0x7f2740198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:17.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f273400d950 tx=0x7f273400de10 comp rx=0 tx=0).stop 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 shutdown_connections 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2740106f40 0x7f274019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27280a2c10 0x7f27280a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27401027c0 0x7f2740198970 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 --2- 172.21.15.19:0/2824443388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2740101e10 0x7f2740198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 >> 172.21.15.19:0/2824443388 conn(0x7f27400fd8d0 msgr2=0x7f27400fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 shutdown_connections 2024-09-12T03:18:17.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:17.334+0000 7f2747f14640 1 -- 172.21.15.19:0/2824443388 wait complete. 2024-09-12T03:18:18.004 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-12T03:18:17.294221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":664,"num_read_kb":473,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356256,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585568,"statfs":{"total":1151940427776,"available":1151575621632,"internally_reserved":0,"allocated":11366400,"data_stored":7196648,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":41,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225942+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:13.225942+0000","last_peered":"2024-09-12T03:18:13.225942+0000","last_clean":"2024-09-12T03:18:13.225942+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:13.225942+0000","last_undegraded":"2024-09-12T03:18:13.225942+0000","last_fullsized":"2024-09-12T03:18:13.225942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226078+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:13.226078+0000","last_peered":"2024-09-12T03:18:13.226078+0000","last_clean":"2024-09-12T03:18:13.226078+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:13.226078+0000","last_undegraded":"2024-09-12T03:18:13.226078+0000","last_fullsized":"2024-09-12T03:18:13.226078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225971+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:13.225971+0000","last_peered":"2024-09-12T03:18:13.225971+0000","last_clean":"2024-09-12T03:18:13.225971+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:13.225971+0000","last_undegraded":"2024-09-12T03:18:13.225971+0000","last_fullsized":"2024-09-12T03:18:13.225971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227583+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:13.227583+0000","last_peered":"2024-09-12T03:18:13.227583+0000","last_clean":"2024-09-12T03:18:13.227583+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:13.227583+0000","last_undegraded":"2024-09-12T03:18:13.227583+0000","last_fullsized":"2024-09-12T03:18:13.227583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223544+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:13.223544+0000","last_peered":"2024-09-12T03:18:13.223544+0000","last_clean":"2024-09-12T03:18:13.223544+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:13.223544+0000","last_undegraded":"2024-09-12T03:18:13.223544+0000","last_fullsized":"2024-09-12T03:18:13.223544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221903+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:13.221903+0000","last_peered":"2024-09-12T03:18:13.221903+0000","last_clean":"2024-09-12T03:18:13.221903+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:13.221903+0000","last_undegraded":"2024-09-12T03:18:13.221903+0000","last_fullsized":"2024-09-12T03:18:13.221903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223234+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:13.223234+0000","last_peered":"2024-09-12T03:18:13.223234+0000","last_clean":"2024-09-12T03:18:13.223234+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:13.223234+0000","last_undegraded":"2024-09-12T03:18:13.223234+0000","last_fullsized":"2024-09-12T03:18:13.223234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227222+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:13.227222+0000","last_peered":"2024-09-12T03:18:13.227222+0000","last_clean":"2024-09-12T03:18:13.227222+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:13.227222+0000","last_undegraded":"2024-09-12T03:18:13.227222+0000","last_fullsized":"2024-09-12T03:18:13.227222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230458+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:13.230458+0000","last_peered":"2024-09-12T03:18:13.230458+0000","last_clean":"2024-09-12T03:18:13.230458+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:13.230458+0000","last_undegraded":"2024-09-12T03:18:13.230458+0000","last_fullsized":"2024-09-12T03:18:13.230458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222751+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:13.222751+0000","last_peered":"2024-09-12T03:18:13.222751+0000","last_clean":"2024-09-12T03:18:13.222751+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:13.222751+0000","last_undegraded":"2024-09-12T03:18:13.222751+0000","last_fullsized":"2024-09-12T03:18:13.222751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224323+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:13.224323+0000","last_peered":"2024-09-12T03:18:13.224323+0000","last_clean":"2024-09-12T03:18:13.224323+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:13.224323+0000","last_undegraded":"2024-09-12T03:18:13.224323+0000","last_fullsized":"2024-09-12T03:18:13.224323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224385+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:13.224385+0000","last_peered":"2024-09-12T03:18:13.224385+0000","last_clean":"2024-09-12T03:18:13.224385+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:13.224385+0000","last_undegraded":"2024-09-12T03:18:13.224385+0000","last_fullsized":"2024-09-12T03:18:13.224385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221504+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:12.221504+0000","last_peered":"2024-09-12T03:18:12.221504+0000","last_clean":"2024-09-12T03:18:12.221504+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:12.221504+0000","last_undegraded":"2024-09-12T03:18:12.221504+0000","last_fullsized":"2024-09-12T03:18:12.221504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225501+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:13.225501+0000","last_peered":"2024-09-12T03:18:13.225501+0000","last_clean":"2024-09-12T03:18:13.225501+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:13.225501+0000","last_undegraded":"2024-09-12T03:18:13.225501+0000","last_fullsized":"2024-09-12T03:18:13.225501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225258+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:13.225258+0000","last_peered":"2024-09-12T03:18:13.225258+0000","last_clean":"2024-09-12T03:18:13.225258+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:13.225258+0000","last_undegraded":"2024-09-12T03:18:13.225258+0000","last_fullsized":"2024-09-12T03:18:13.225258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226050+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:13.226050+0000","last_peered":"2024-09-12T03:18:13.226050+0000","last_clean":"2024-09-12T03:18:13.226050+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:13.226050+0000","last_undegraded":"2024-09-12T03:18:13.226050+0000","last_fullsized":"2024-09-12T03:18:13.226050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222851+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:13.222851+0000","last_peered":"2024-09-12T03:18:13.222851+0000","last_clean":"2024-09-12T03:18:13.222851+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:13.222851+0000","last_undegraded":"2024-09-12T03:18:13.222851+0000","last_fullsized":"2024-09-12T03:18:13.222851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221546+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:13.221546+0000","last_peered":"2024-09-12T03:18:13.221546+0000","last_clean":"2024-09-12T03:18:13.221546+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:13.221546+0000","last_undegraded":"2024-09-12T03:18:13.221546+0000","last_fullsized":"2024-09-12T03:18:13.221546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227637+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:13.227637+0000","last_peered":"2024-09-12T03:18:13.227637+0000","last_clean":"2024-09-12T03:18:13.227637+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:13.227637+0000","last_undegraded":"2024-09-12T03:18:13.227637+0000","last_fullsized":"2024-09-12T03:18:13.227637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221559+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:12.221559+0000","last_peered":"2024-09-12T03:18:12.221559+0000","last_clean":"2024-09-12T03:18:12.221559+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:12.221559+0000","last_undegraded":"2024-09-12T03:18:12.221559+0000","last_fullsized":"2024-09-12T03:18:12.221559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230002+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:13.230002+0000","last_peered":"2024-09-12T03:18:13.230002+0000","last_clean":"2024-09-12T03:18:13.230002+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:13.230002+0000","last_undegraded":"2024-09-12T03:18:13.230002+0000","last_fullsized":"2024-09-12T03:18:13.230002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227500+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:13.227500+0000","last_peered":"2024-09-12T03:18:13.227500+0000","last_clean":"2024-09-12T03:18:13.227500+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:13.227500+0000","last_undegraded":"2024-09-12T03:18:13.227500+0000","last_fullsized":"2024-09-12T03:18:13.227500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229154+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:12.229154+0000","last_peered":"2024-09-12T03:18:12.229154+0000","last_clean":"2024-09-12T03:18:12.229154+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:12.229154+0000","last_undegraded":"2024-09-12T03:18:12.229154+0000","last_fullsized":"2024-09-12T03:18:12.229154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228434+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:13.228434+0000","last_peered":"2024-09-12T03:18:13.228434+0000","last_clean":"2024-09-12T03:18:13.228434+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:13.228434+0000","last_undegraded":"2024-09-12T03:18:13.228434+0000","last_fullsized":"2024-09-12T03:18:13.228434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221771+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:13.221771+0000","last_peered":"2024-09-12T03:18:13.221771+0000","last_clean":"2024-09-12T03:18:13.221771+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:13.221771+0000","last_undegraded":"2024-09-12T03:18:13.221771+0000","last_fullsized":"2024-09-12T03:18:13.221771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225026+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:13.225026+0000","last_peered":"2024-09-12T03:18:13.225026+0000","last_clean":"2024-09-12T03:18:13.225026+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:13.225026+0000","last_undegraded":"2024-09-12T03:18:13.225026+0000","last_fullsized":"2024-09-12T03:18:13.225026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221524+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:12.221524+0000","last_peered":"2024-09-12T03:18:12.221524+0000","last_clean":"2024-09-12T03:18:12.221524+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:12.221524+0000","last_undegraded":"2024-09-12T03:18:12.221524+0000","last_fullsized":"2024-09-12T03:18:12.221524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225810+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:13.225810+0000","last_peered":"2024-09-12T03:18:13.225810+0000","last_clean":"2024-09-12T03:18:13.225810+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:13.225810+0000","last_undegraded":"2024-09-12T03:18:13.225810+0000","last_fullsized":"2024-09-12T03:18:13.225810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221553+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:13.221553+0000","last_peered":"2024-09-12T03:18:13.221553+0000","last_clean":"2024-09-12T03:18:13.221553+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:13.221553+0000","last_undegraded":"2024-09-12T03:18:13.221553+0000","last_fullsized":"2024-09-12T03:18:13.221553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221469+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:13.221469+0000","last_peered":"2024-09-12T03:18:13.221469+0000","last_clean":"2024-09-12T03:18:13.221469+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:13.221469+0000","last_undegraded":"2024-09-12T03:18:13.221469+0000","last_fullsized":"2024-09-12T03:18:13.221469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225010+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:13.225010+0000","last_peered":"2024-09-12T03:18:13.225010+0000","last_clean":"2024-09-12T03:18:13.225010+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:13.225010+0000","last_undegraded":"2024-09-12T03:18:13.225010+0000","last_fullsized":"2024-09-12T03:18:13.225010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223333+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:13.223333+0000","last_peered":"2024-09-12T03:18:13.223333+0000","last_clean":"2024-09-12T03:18:13.223333+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:13.223333+0000","last_undegraded":"2024-09-12T03:18:13.223333+0000","last_fullsized":"2024-09-12T03:18:13.223333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229744+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:18:12.229744+0000","last_peered":"2024-09-12T03:18:12.229744+0000","last_clean":"2024-09-12T03:18:12.229744+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:18:12.229744+0000","last_undegraded":"2024-09-12T03:18:12.229744+0000","last_fullsized":"2024-09-12T03:18:12.229744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229805+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:12.229805+0000","last_peered":"2024-09-12T03:18:12.229805+0000","last_clean":"2024-09-12T03:18:12.229805+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:12.229805+0000","last_undegraded":"2024-09-12T03:18:12.229805+0000","last_fullsized":"2024-09-12T03:18:12.229805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221284+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:12.221284+0000","last_peered":"2024-09-12T03:18:12.221284+0000","last_clean":"2024-09-12T03:18:12.221284+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:12.221284+0000","last_undegraded":"2024-09-12T03:18:12.221284+0000","last_fullsized":"2024-09-12T03:18:12.221284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227325+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:13.227325+0000","last_peered":"2024-09-12T03:18:13.227325+0000","last_clean":"2024-09-12T03:18:13.227325+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:13.227325+0000","last_undegraded":"2024-09-12T03:18:13.227325+0000","last_fullsized":"2024-09-12T03:18:13.227325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229332+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:12.229332+0000","last_peered":"2024-09-12T03:18:12.229332+0000","last_clean":"2024-09-12T03:18:12.229332+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:12.229332+0000","last_undegraded":"2024-09-12T03:18:12.229332+0000","last_fullsized":"2024-09-12T03:18:12.229332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222614+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:13.222614+0000","last_peered":"2024-09-12T03:18:13.222614+0000","last_clean":"2024-09-12T03:18:13.222614+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:13.222614+0000","last_undegraded":"2024-09-12T03:18:13.222614+0000","last_fullsized":"2024-09-12T03:18:13.222614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.233405+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:18:14.233405+0000","last_peered":"2024-09-12T03:18:14.233405+0000","last_clean":"2024-09-12T03:18:14.233405+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:18:14.233405+0000","last_undegraded":"2024-09-12T03:18:14.233405+0000","last_fullsized":"2024-09-12T03:18:14.233405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230681+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:13.230681+0000","last_peered":"2024-09-12T03:18:13.230681+0000","last_clean":"2024-09-12T03:18:13.230681+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:13.230681+0000","last_undegraded":"2024-09-12T03:18:13.230681+0000","last_fullsized":"2024-09-12T03:18:13.230681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225820+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:13.225820+0000","last_peered":"2024-09-12T03:18:13.225820+0000","last_clean":"2024-09-12T03:18:13.225820+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:13.225820+0000","last_undegraded":"2024-09-12T03:18:13.225820+0000","last_fullsized":"2024-09-12T03:18:13.225820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229822+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:12.229822+0000","last_peered":"2024-09-12T03:18:12.229822+0000","last_clean":"2024-09-12T03:18:12.229822+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:12.229822+0000","last_undegraded":"2024-09-12T03:18:12.229822+0000","last_fullsized":"2024-09-12T03:18:12.229822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230416+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:13.230416+0000","last_peered":"2024-09-12T03:18:13.230416+0000","last_clean":"2024-09-12T03:18:13.230416+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:13.230416+0000","last_undegraded":"2024-09-12T03:18:13.230416+0000","last_fullsized":"2024-09-12T03:18:13.230416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.891326+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:18:14.891326+0000","last_peered":"2024-09-12T03:18:14.891326+0000","last_clean":"2024-09-12T03:18:14.891326+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:18:14.891326+0000","last_undegraded":"2024-09-12T03:18:14.891326+0000","last_fullsized":"2024-09-12T03:18:14.891326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":118,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223911+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:13.223911+0000","last_peered":"2024-09-12T03:18:13.223911+0000","last_clean":"2024-09-12T03:18:13.223911+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:13.223911+0000","last_undegraded":"2024-09-12T03:18:13.223911+0000","last_fullsized":"2024-09-12T03:18:13.223911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221379+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:12.221379+0000","last_peered":"2024-09-12T03:18:12.221379+0000","last_clean":"2024-09-12T03:18:12.221379+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:12.221379+0000","last_undegraded":"2024-09-12T03:18:12.221379+0000","last_fullsized":"2024-09-12T03:18:12.221379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229511+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:12.229511+0000","last_peered":"2024-09-12T03:18:12.229511+0000","last_clean":"2024-09-12T03:18:12.229511+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:12.229511+0000","last_undegraded":"2024-09-12T03:18:12.229511+0000","last_fullsized":"2024-09-12T03:18:12.229511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223375+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:13.223375+0000","last_peered":"2024-09-12T03:18:13.223375+0000","last_clean":"2024-09-12T03:18:13.223375+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:13.223375+0000","last_undegraded":"2024-09-12T03:18:13.223375+0000","last_fullsized":"2024-09-12T03:18:13.223375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.892617+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:18:15.892617+0000","last_peered":"2024-09-12T03:18:15.892617+0000","last_clean":"2024-09-12T03:18:15.892617+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:18:15.892617+0000","last_undegraded":"2024-09-12T03:18:15.892617+0000","last_fullsized":"2024-09-12T03:18:15.892617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225722+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:13.225722+0000","last_peered":"2024-09-12T03:18:13.225722+0000","last_clean":"2024-09-12T03:18:13.225722+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:13.225722+0000","last_undegraded":"2024-09-12T03:18:13.225722+0000","last_fullsized":"2024-09-12T03:18:13.225722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229248+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:12.229248+0000","last_peered":"2024-09-12T03:18:12.229248+0000","last_clean":"2024-09-12T03:18:12.229248+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:12.229248+0000","last_undegraded":"2024-09-12T03:18:12.229248+0000","last_fullsized":"2024-09-12T03:18:12.229248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225060+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:13.225060+0000","last_peered":"2024-09-12T03:18:13.225060+0000","last_clean":"2024-09-12T03:18:13.225060+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:13.225060+0000","last_undegraded":"2024-09-12T03:18:13.225060+0000","last_fullsized":"2024-09-12T03:18:13.225060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221726+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:12.221726+0000","last_peered":"2024-09-12T03:18:12.221726+0000","last_clean":"2024-09-12T03:18:12.221726+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:12.221726+0000","last_undegraded":"2024-09-12T03:18:12.221726+0000","last_fullsized":"2024-09-12T03:18:12.221726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224091+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:13.224091+0000","last_peered":"2024-09-12T03:18:13.224091+0000","last_clean":"2024-09-12T03:18:13.224091+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:13.224091+0000","last_undegraded":"2024-09-12T03:18:13.224091+0000","last_fullsized":"2024-09-12T03:18:13.224091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229949+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:12.229949+0000","last_peered":"2024-09-12T03:18:12.229949+0000","last_clean":"2024-09-12T03:18:12.229949+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:12.229949+0000","last_undegraded":"2024-09-12T03:18:12.229949+0000","last_fullsized":"2024-09-12T03:18:12.229949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235706+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:13.235706+0000","last_peered":"2024-09-12T03:18:13.235706+0000","last_clean":"2024-09-12T03:18:13.235706+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:13.235706+0000","last_undegraded":"2024-09-12T03:18:13.235706+0000","last_fullsized":"2024-09-12T03:18:13.235706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:16.236651+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:18:16.236651+0000","last_peered":"2024-09-12T03:18:16.236651+0000","last_clean":"2024-09-12T03:18:16.236651+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:18:16.236651+0000","last_undegraded":"2024-09-12T03:18:16.236651+0000","last_fullsized":"2024-09-12T03:18:16.236651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224420+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:13.224420+0000","last_peered":"2024-09-12T03:18:13.224420+0000","last_clean":"2024-09-12T03:18:13.224420+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:13.224420+0000","last_undegraded":"2024-09-12T03:18:13.224420+0000","last_fullsized":"2024-09-12T03:18:13.224420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223799+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223799+0000","last_peered":"2024-09-12T03:18:13.223799+0000","last_clean":"2024-09-12T03:18:13.223799+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223799+0000","last_undegraded":"2024-09-12T03:18:13.223799+0000","last_fullsized":"2024-09-12T03:18:13.223799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223939+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.223939+0000","last_peered":"2024-09-12T03:18:13.223939+0000","last_clean":"2024-09-12T03:18:13.223939+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.223939+0000","last_undegraded":"2024-09-12T03:18:13.223939+0000","last_fullsized":"2024-09-12T03:18:13.223939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227978+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:13.227978+0000","last_peered":"2024-09-12T03:18:13.227978+0000","last_clean":"2024-09-12T03:18:13.227978+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:13.227978+0000","last_undegraded":"2024-09-12T03:18:13.227978+0000","last_fullsized":"2024-09-12T03:18:13.227978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223720+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223720+0000","last_peered":"2024-09-12T03:18:13.223720+0000","last_clean":"2024-09-12T03:18:13.223720+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223720+0000","last_undegraded":"2024-09-12T03:18:13.223720+0000","last_fullsized":"2024-09-12T03:18:13.223720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221272+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:12.221272+0000","last_peered":"2024-09-12T03:18:12.221272+0000","last_clean":"2024-09-12T03:18:12.221272+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:12.221272+0000","last_undegraded":"2024-09-12T03:18:12.221272+0000","last_fullsized":"2024-09-12T03:18:12.221272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221476+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:18:12.221476+0000","last_peered":"2024-09-12T03:18:12.221476+0000","last_clean":"2024-09-12T03:18:12.221476+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:18:12.221476+0000","last_undegraded":"2024-09-12T03:18:12.221476+0000","last_fullsized":"2024-09-12T03:18:12.221476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229219+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:12.229219+0000","last_peered":"2024-09-12T03:18:12.229219+0000","last_clean":"2024-09-12T03:18:12.229219+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:12.229219+0000","last_undegraded":"2024-09-12T03:18:12.229219+0000","last_fullsized":"2024-09-12T03:18:12.229219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.916882+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:18:15.916882+0000","last_peered":"2024-09-12T03:18:15.916882+0000","last_clean":"2024-09-12T03:18:15.916882+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:18:15.916882+0000","last_undegraded":"2024-09-12T03:18:15.916882+0000","last_fullsized":"2024-09-12T03:18:15.916882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221810+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:13.221810+0000","last_peered":"2024-09-12T03:18:13.221810+0000","last_clean":"2024-09-12T03:18:13.221810+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:13.221810+0000","last_undegraded":"2024-09-12T03:18:13.221810+0000","last_fullsized":"2024-09-12T03:18:13.221810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223007+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223007+0000","last_peered":"2024-09-12T03:18:13.223007+0000","last_clean":"2024-09-12T03:18:13.223007+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223007+0000","last_undegraded":"2024-09-12T03:18:13.223007+0000","last_fullsized":"2024-09-12T03:18:13.223007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222321+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:13.222321+0000","last_peered":"2024-09-12T03:18:13.222321+0000","last_clean":"2024-09-12T03:18:13.222321+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:13.222321+0000","last_undegraded":"2024-09-12T03:18:13.222321+0000","last_fullsized":"2024-09-12T03:18:13.222321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224119+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224119+0000","last_peered":"2024-09-12T03:18:13.224119+0000","last_clean":"2024-09-12T03:18:13.224119+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224119+0000","last_undegraded":"2024-09-12T03:18:13.224119+0000","last_fullsized":"2024-09-12T03:18:13.224119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221604+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:12.221604+0000","last_peered":"2024-09-12T03:18:12.221604+0000","last_clean":"2024-09-12T03:18:12.221604+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:12.221604+0000","last_undegraded":"2024-09-12T03:18:12.221604+0000","last_fullsized":"2024-09-12T03:18:12.221604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222863+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:13.222863+0000","last_peered":"2024-09-12T03:18:13.222863+0000","last_clean":"2024-09-12T03:18:13.222863+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:13.222863+0000","last_undegraded":"2024-09-12T03:18:13.222863+0000","last_fullsized":"2024-09-12T03:18:13.222863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224342+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224342+0000","last_peered":"2024-09-12T03:18:13.224342+0000","last_clean":"2024-09-12T03:18:13.224342+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224342+0000","last_undegraded":"2024-09-12T03:18:13.224342+0000","last_fullsized":"2024-09-12T03:18:13.224342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226157+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:13.226157+0000","last_peered":"2024-09-12T03:18:13.226157+0000","last_clean":"2024-09-12T03:18:13.226157+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:13.226157+0000","last_undegraded":"2024-09-12T03:18:13.226157+0000","last_fullsized":"2024-09-12T03:18:13.226157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230600+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230600+0000","last_peered":"2024-09-12T03:18:13.230600+0000","last_clean":"2024-09-12T03:18:13.230600+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230600+0000","last_undegraded":"2024-09-12T03:18:13.230600+0000","last_fullsized":"2024-09-12T03:18:13.230600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223193+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:13.223193+0000","last_peered":"2024-09-12T03:18:13.223193+0000","last_clean":"2024-09-12T03:18:13.223193+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:13.223193+0000","last_undegraded":"2024-09-12T03:18:13.223193+0000","last_fullsized":"2024-09-12T03:18:13.223193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225267+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:13.225267+0000","last_peered":"2024-09-12T03:18:13.225267+0000","last_clean":"2024-09-12T03:18:13.225267+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:13.225267+0000","last_undegraded":"2024-09-12T03:18:13.225267+0000","last_fullsized":"2024-09-12T03:18:13.225267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228053+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:13.228053+0000","last_peered":"2024-09-12T03:18:13.228053+0000","last_clean":"2024-09-12T03:18:13.228053+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:13.228053+0000","last_undegraded":"2024-09-12T03:18:13.228053+0000","last_fullsized":"2024-09-12T03:18:13.228053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223554+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:13.223554+0000","last_peered":"2024-09-12T03:18:13.223554+0000","last_clean":"2024-09-12T03:18:13.223554+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:13.223554+0000","last_undegraded":"2024-09-12T03:18:13.223554+0000","last_fullsized":"2024-09-12T03:18:13.223554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"peering","last_fresh":"2024-09-12T03:17:13.121382+0000","last_change":"2024-09-12T03:18:12.228851+0000","last_active":"2024-09-12T03:18:12.228851+0000","last_peered":"2024-09-12T03:17:13.121382+0000","last_clean":"2024-09-12T03:17:13.121382+0000","last_became_active":"2024-09-12T03:17:09.921491+0000","last_became_peered":"2024-09-12T03:17:09.921491+0000","last_unstale":"2024-09-12T03:17:13.121382+0000","last_undegraded":"2024-09-12T03:17:13.121382+0000","last_fullsized":"2024-09-12T03:17:13.121382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225732+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:13.225732+0000","last_peered":"2024-09-12T03:18:13.225732+0000","last_clean":"2024-09-12T03:18:13.225732+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:13.225732+0000","last_undegraded":"2024-09-12T03:18:13.225732+0000","last_fullsized":"2024-09-12T03:18:13.225732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228427+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228427+0000","last_peered":"2024-09-12T03:18:13.228427+0000","last_clean":"2024-09-12T03:18:13.228427+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228427+0000","last_undegraded":"2024-09-12T03:18:13.228427+0000","last_fullsized":"2024-09-12T03:18:13.228427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228289+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:13.228289+0000","last_peered":"2024-09-12T03:18:13.228289+0000","last_clean":"2024-09-12T03:18:13.228289+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:13.228289+0000","last_undegraded":"2024-09-12T03:18:13.228289+0000","last_fullsized":"2024-09-12T03:18:13.228289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.913793+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:18:13.913793+0000","last_peered":"2024-09-12T03:18:13.913793+0000","last_clean":"2024-09-12T03:18:13.913793+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:18:13.913793+0000","last_undegraded":"2024-09-12T03:18:13.913793+0000","last_fullsized":"2024-09-12T03:18:13.913793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":538,"num_read_kb":364,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200916,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":1089115,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1031150,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":429117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855418,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1021976,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444106,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497696,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":966656,"data_stored":498253,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61516,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60959,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61893,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:18:18.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:18:18.271 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- 172.21.15.19:0/468689479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 msgr2=0x7fbe0810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/468689479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0810ec70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009930 tx=0x7fbe0002a420 comp rx=0 tx=0).stop 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- 172.21.15.19:0/468689479 shutdown_connections 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/468689479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0810ec70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/468689479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 0x7fbe08107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/468689479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- 172.21.15.19:0/468689479 >> 172.21.15.19:0/468689479 conn(0x7fbe080fc7d0 msgr2=0x7fbe080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- 172.21.15.19:0/468689479 shutdown_connections 2024-09-12T03:18:18.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- 172.21.15.19:0/468689479 wait complete. 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 Processor -- start 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.811+0000 7fbe0d48b640 1 -- start start 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 0x7fbe08198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112110 con 0x7fbe08108300 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08111f90 con 0x7fbe08103780 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe0d48b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112290 con 0x7fbe08102dd0 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:18.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36300/0 (socket says 172.21.15.19:36300) 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 -- 172.21.15.19:0/977400188 learned_addr learned my addr 172.21.15.19:0/977400188 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe067fc640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 0x7fbe08198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe06ffd640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 -- 172.21.15.19:0/977400188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 msgr2=0x7fbe081985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 -- 172.21.15.19:0/977400188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 msgr2=0x7fbe08198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 0x7fbe08198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 -- 172.21.15.19:0/977400188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0819d570 con 0x7fbe08108300 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe06ffd640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbe077fe640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe00005ab0 tx=0x7fbe000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:18.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0003f030 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.812+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe00003ce0 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.813+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0819d800 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.813+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00035070 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.813+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a5140 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.814+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe081031b0 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.815+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbe00042030 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.815+0000 7fbdeffff640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 0x7fbdf00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.815+0000 7fbe06ffd640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 0x7fbdf00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.816+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbe000ef5e0 con 0x7fbe08108300 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.816+0000 7fbe06ffd640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 0x7fbdf00a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009fa0 tx=0x7fbdf8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:18.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.819+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe000bc3f0 con 0x7fbe08108300 2024-09-12T03:18:18.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:18 smithi019 ceph-mon[30953]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:18.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:18 smithi019 ceph-mon[30953]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:18.996 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:18.996+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe08062630 con 0x7fbdf00a2c90 2024-09-12T03:18:19.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.002+0000 7fbdeffff640 1 -- 172.21.15.19:0/977400188 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258956 (secure 0 0 0) 0x7fbe08062630 con 0x7fbdf00a2c90 2024-09-12T03:18:19.004 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:19.005 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:18:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 msgr2=0x7fbdf00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:19.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 0x7fbdf00a5150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8009fa0 tx=0x7fbdf8009570 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 msgr2=0x7fbe0819ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbe00005ab0 tx=0x7fbe000042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 shutdown_connections 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbdf00a2c90 0x7fbdf00a5150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbe08108300 0x7fbe0819ce90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe08103780 0x7fbe08198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 --2- 172.21.15.19:0/977400188 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbe08102dd0 0x7fbe081985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 >> 172.21.15.19:0/977400188 conn(0x7fbe080fc7d0 msgr2=0x7fbe08105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 shutdown_connections 2024-09-12T03:18:19.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:19.005+0000 7fbe0d48b640 1 -- 172.21.15.19:0/977400188 wait complete. 2024-09-12T03:18:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:18 smithi135 ceph-mon[35474]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:18 smithi135 ceph-mon[35474]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:19.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:18 smithi121 ceph-mon[34963]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 4 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:19.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:18 smithi121 ceph-mon[34963]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:19.726 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:19 smithi019 ceph-mon[30953]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:19.760 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-12T03:18:17.294221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":664,"num_read_kb":473,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":295,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356256,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585568,"statfs":{"total":1151940427776,"available":1151575621632,"internally_reserved":0,"allocated":11366400,"data_stored":7196648,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":41,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001725"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225942+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:13.225942+0000","last_peered":"2024-09-12T03:18:13.225942+0000","last_clean":"2024-09-12T03:18:13.225942+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:13.225942+0000","last_undegraded":"2024-09-12T03:18:13.225942+0000","last_fullsized":"2024-09-12T03:18:13.225942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226078+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:13.226078+0000","last_peered":"2024-09-12T03:18:13.226078+0000","last_clean":"2024-09-12T03:18:13.226078+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:13.226078+0000","last_undegraded":"2024-09-12T03:18:13.226078+0000","last_fullsized":"2024-09-12T03:18:13.226078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225971+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:13.225971+0000","last_peered":"2024-09-12T03:18:13.225971+0000","last_clean":"2024-09-12T03:18:13.225971+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:13.225971+0000","last_undegraded":"2024-09-12T03:18:13.225971+0000","last_fullsized":"2024-09-12T03:18:13.225971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227583+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:13.227583+0000","last_peered":"2024-09-12T03:18:13.227583+0000","last_clean":"2024-09-12T03:18:13.227583+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:13.227583+0000","last_undegraded":"2024-09-12T03:18:13.227583+0000","last_fullsized":"2024-09-12T03:18:13.227583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223544+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:13.223544+0000","last_peered":"2024-09-12T03:18:13.223544+0000","last_clean":"2024-09-12T03:18:13.223544+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:13.223544+0000","last_undegraded":"2024-09-12T03:18:13.223544+0000","last_fullsized":"2024-09-12T03:18:13.223544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221903+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:13.221903+0000","last_peered":"2024-09-12T03:18:13.221903+0000","last_clean":"2024-09-12T03:18:13.221903+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:13.221903+0000","last_undegraded":"2024-09-12T03:18:13.221903+0000","last_fullsized":"2024-09-12T03:18:13.221903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223234+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:13.223234+0000","last_peered":"2024-09-12T03:18:13.223234+0000","last_clean":"2024-09-12T03:18:13.223234+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:13.223234+0000","last_undegraded":"2024-09-12T03:18:13.223234+0000","last_fullsized":"2024-09-12T03:18:13.223234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227222+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:13.227222+0000","last_peered":"2024-09-12T03:18:13.227222+0000","last_clean":"2024-09-12T03:18:13.227222+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:13.227222+0000","last_undegraded":"2024-09-12T03:18:13.227222+0000","last_fullsized":"2024-09-12T03:18:13.227222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230458+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:13.230458+0000","last_peered":"2024-09-12T03:18:13.230458+0000","last_clean":"2024-09-12T03:18:13.230458+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:13.230458+0000","last_undegraded":"2024-09-12T03:18:13.230458+0000","last_fullsized":"2024-09-12T03:18:13.230458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222751+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:13.222751+0000","last_peered":"2024-09-12T03:18:13.222751+0000","last_clean":"2024-09-12T03:18:13.222751+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:13.222751+0000","last_undegraded":"2024-09-12T03:18:13.222751+0000","last_fullsized":"2024-09-12T03:18:13.222751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224323+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:13.224323+0000","last_peered":"2024-09-12T03:18:13.224323+0000","last_clean":"2024-09-12T03:18:13.224323+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:13.224323+0000","last_undegraded":"2024-09-12T03:18:13.224323+0000","last_fullsized":"2024-09-12T03:18:13.224323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224385+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:13.224385+0000","last_peered":"2024-09-12T03:18:13.224385+0000","last_clean":"2024-09-12T03:18:13.224385+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:13.224385+0000","last_undegraded":"2024-09-12T03:18:13.224385+0000","last_fullsized":"2024-09-12T03:18:13.224385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221504+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:12.221504+0000","last_peered":"2024-09-12T03:18:12.221504+0000","last_clean":"2024-09-12T03:18:12.221504+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:12.221504+0000","last_undegraded":"2024-09-12T03:18:12.221504+0000","last_fullsized":"2024-09-12T03:18:12.221504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225501+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:13.225501+0000","last_peered":"2024-09-12T03:18:13.225501+0000","last_clean":"2024-09-12T03:18:13.225501+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:13.225501+0000","last_undegraded":"2024-09-12T03:18:13.225501+0000","last_fullsized":"2024-09-12T03:18:13.225501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225258+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:13.225258+0000","last_peered":"2024-09-12T03:18:13.225258+0000","last_clean":"2024-09-12T03:18:13.225258+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:13.225258+0000","last_undegraded":"2024-09-12T03:18:13.225258+0000","last_fullsized":"2024-09-12T03:18:13.225258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226050+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:13.226050+0000","last_peered":"2024-09-12T03:18:13.226050+0000","last_clean":"2024-09-12T03:18:13.226050+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:13.226050+0000","last_undegraded":"2024-09-12T03:18:13.226050+0000","last_fullsized":"2024-09-12T03:18:13.226050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222851+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:13.222851+0000","last_peered":"2024-09-12T03:18:13.222851+0000","last_clean":"2024-09-12T03:18:13.222851+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:13.222851+0000","last_undegraded":"2024-09-12T03:18:13.222851+0000","last_fullsized":"2024-09-12T03:18:13.222851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221546+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:13.221546+0000","last_peered":"2024-09-12T03:18:13.221546+0000","last_clean":"2024-09-12T03:18:13.221546+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:13.221546+0000","last_undegraded":"2024-09-12T03:18:13.221546+0000","last_fullsized":"2024-09-12T03:18:13.221546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227637+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:13.227637+0000","last_peered":"2024-09-12T03:18:13.227637+0000","last_clean":"2024-09-12T03:18:13.227637+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:13.227637+0000","last_undegraded":"2024-09-12T03:18:13.227637+0000","last_fullsized":"2024-09-12T03:18:13.227637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221559+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:12.221559+0000","last_peered":"2024-09-12T03:18:12.221559+0000","last_clean":"2024-09-12T03:18:12.221559+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:12.221559+0000","last_undegraded":"2024-09-12T03:18:12.221559+0000","last_fullsized":"2024-09-12T03:18:12.221559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230002+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:13.230002+0000","last_peered":"2024-09-12T03:18:13.230002+0000","last_clean":"2024-09-12T03:18:13.230002+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:13.230002+0000","last_undegraded":"2024-09-12T03:18:13.230002+0000","last_fullsized":"2024-09-12T03:18:13.230002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227500+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:13.227500+0000","last_peered":"2024-09-12T03:18:13.227500+0000","last_clean":"2024-09-12T03:18:13.227500+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:13.227500+0000","last_undegraded":"2024-09-12T03:18:13.227500+0000","last_fullsized":"2024-09-12T03:18:13.227500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229154+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:12.229154+0000","last_peered":"2024-09-12T03:18:12.229154+0000","last_clean":"2024-09-12T03:18:12.229154+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:12.229154+0000","last_undegraded":"2024-09-12T03:18:12.229154+0000","last_fullsized":"2024-09-12T03:18:12.229154+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228434+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:13.228434+0000","last_peered":"2024-09-12T03:18:13.228434+0000","last_clean":"2024-09-12T03:18:13.228434+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:13.228434+0000","last_undegraded":"2024-09-12T03:18:13.228434+0000","last_fullsized":"2024-09-12T03:18:13.228434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221771+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:13.221771+0000","last_peered":"2024-09-12T03:18:13.221771+0000","last_clean":"2024-09-12T03:18:13.221771+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:13.221771+0000","last_undegraded":"2024-09-12T03:18:13.221771+0000","last_fullsized":"2024-09-12T03:18:13.221771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225026+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:13.225026+0000","last_peered":"2024-09-12T03:18:13.225026+0000","last_clean":"2024-09-12T03:18:13.225026+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:13.225026+0000","last_undegraded":"2024-09-12T03:18:13.225026+0000","last_fullsized":"2024-09-12T03:18:13.225026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221524+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:12.221524+0000","last_peered":"2024-09-12T03:18:12.221524+0000","last_clean":"2024-09-12T03:18:12.221524+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:12.221524+0000","last_undegraded":"2024-09-12T03:18:12.221524+0000","last_fullsized":"2024-09-12T03:18:12.221524+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225810+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:13.225810+0000","last_peered":"2024-09-12T03:18:13.225810+0000","last_clean":"2024-09-12T03:18:13.225810+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:13.225810+0000","last_undegraded":"2024-09-12T03:18:13.225810+0000","last_fullsized":"2024-09-12T03:18:13.225810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221553+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:13.221553+0000","last_peered":"2024-09-12T03:18:13.221553+0000","last_clean":"2024-09-12T03:18:13.221553+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:13.221553+0000","last_undegraded":"2024-09-12T03:18:13.221553+0000","last_fullsized":"2024-09-12T03:18:13.221553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221469+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:13.221469+0000","last_peered":"2024-09-12T03:18:13.221469+0000","last_clean":"2024-09-12T03:18:13.221469+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:13.221469+0000","last_undegraded":"2024-09-12T03:18:13.221469+0000","last_fullsized":"2024-09-12T03:18:13.221469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225010+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:13.225010+0000","last_peered":"2024-09-12T03:18:13.225010+0000","last_clean":"2024-09-12T03:18:13.225010+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:13.225010+0000","last_undegraded":"2024-09-12T03:18:13.225010+0000","last_fullsized":"2024-09-12T03:18:13.225010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223333+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:13.223333+0000","last_peered":"2024-09-12T03:18:13.223333+0000","last_clean":"2024-09-12T03:18:13.223333+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:13.223333+0000","last_undegraded":"2024-09-12T03:18:13.223333+0000","last_fullsized":"2024-09-12T03:18:13.223333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":83,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229744+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:18:12.229744+0000","last_peered":"2024-09-12T03:18:12.229744+0000","last_clean":"2024-09-12T03:18:12.229744+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:18:12.229744+0000","last_undegraded":"2024-09-12T03:18:12.229744+0000","last_fullsized":"2024-09-12T03:18:12.229744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229805+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:12.229805+0000","last_peered":"2024-09-12T03:18:12.229805+0000","last_clean":"2024-09-12T03:18:12.229805+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:12.229805+0000","last_undegraded":"2024-09-12T03:18:12.229805+0000","last_fullsized":"2024-09-12T03:18:12.229805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221284+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:12.221284+0000","last_peered":"2024-09-12T03:18:12.221284+0000","last_clean":"2024-09-12T03:18:12.221284+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:12.221284+0000","last_undegraded":"2024-09-12T03:18:12.221284+0000","last_fullsized":"2024-09-12T03:18:12.221284+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227325+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:13.227325+0000","last_peered":"2024-09-12T03:18:13.227325+0000","last_clean":"2024-09-12T03:18:13.227325+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:13.227325+0000","last_undegraded":"2024-09-12T03:18:13.227325+0000","last_fullsized":"2024-09-12T03:18:13.227325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229332+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:12.229332+0000","last_peered":"2024-09-12T03:18:12.229332+0000","last_clean":"2024-09-12T03:18:12.229332+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:12.229332+0000","last_undegraded":"2024-09-12T03:18:12.229332+0000","last_fullsized":"2024-09-12T03:18:12.229332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222614+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:13.222614+0000","last_peered":"2024-09-12T03:18:13.222614+0000","last_clean":"2024-09-12T03:18:13.222614+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:13.222614+0000","last_undegraded":"2024-09-12T03:18:13.222614+0000","last_fullsized":"2024-09-12T03:18:13.222614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":135,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.233405+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:18:14.233405+0000","last_peered":"2024-09-12T03:18:14.233405+0000","last_clean":"2024-09-12T03:18:14.233405+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:18:14.233405+0000","last_undegraded":"2024-09-12T03:18:14.233405+0000","last_fullsized":"2024-09-12T03:18:14.233405+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230681+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:13.230681+0000","last_peered":"2024-09-12T03:18:13.230681+0000","last_clean":"2024-09-12T03:18:13.230681+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:13.230681+0000","last_undegraded":"2024-09-12T03:18:13.230681+0000","last_fullsized":"2024-09-12T03:18:13.230681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225820+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:13.225820+0000","last_peered":"2024-09-12T03:18:13.225820+0000","last_clean":"2024-09-12T03:18:13.225820+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:13.225820+0000","last_undegraded":"2024-09-12T03:18:13.225820+0000","last_fullsized":"2024-09-12T03:18:13.225820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229822+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:12.229822+0000","last_peered":"2024-09-12T03:18:12.229822+0000","last_clean":"2024-09-12T03:18:12.229822+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:12.229822+0000","last_undegraded":"2024-09-12T03:18:12.229822+0000","last_fullsized":"2024-09-12T03:18:12.229822+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230416+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:13.230416+0000","last_peered":"2024-09-12T03:18:13.230416+0000","last_clean":"2024-09-12T03:18:13.230416+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:13.230416+0000","last_undegraded":"2024-09-12T03:18:13.230416+0000","last_fullsized":"2024-09-12T03:18:13.230416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":172,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:14.891326+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:18:14.891326+0000","last_peered":"2024-09-12T03:18:14.891326+0000","last_clean":"2024-09-12T03:18:14.891326+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:18:14.891326+0000","last_undegraded":"2024-09-12T03:18:14.891326+0000","last_fullsized":"2024-09-12T03:18:14.891326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":118,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223911+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:13.223911+0000","last_peered":"2024-09-12T03:18:13.223911+0000","last_clean":"2024-09-12T03:18:13.223911+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:13.223911+0000","last_undegraded":"2024-09-12T03:18:13.223911+0000","last_fullsized":"2024-09-12T03:18:13.223911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221379+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:12.221379+0000","last_peered":"2024-09-12T03:18:12.221379+0000","last_clean":"2024-09-12T03:18:12.221379+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:12.221379+0000","last_undegraded":"2024-09-12T03:18:12.221379+0000","last_fullsized":"2024-09-12T03:18:12.221379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229511+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:12.229511+0000","last_peered":"2024-09-12T03:18:12.229511+0000","last_clean":"2024-09-12T03:18:12.229511+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:12.229511+0000","last_undegraded":"2024-09-12T03:18:12.229511+0000","last_fullsized":"2024-09-12T03:18:12.229511+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223375+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:13.223375+0000","last_peered":"2024-09-12T03:18:13.223375+0000","last_clean":"2024-09-12T03:18:13.223375+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:13.223375+0000","last_undegraded":"2024-09-12T03:18:13.223375+0000","last_fullsized":"2024-09-12T03:18:13.223375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.892617+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:18:15.892617+0000","last_peered":"2024-09-12T03:18:15.892617+0000","last_clean":"2024-09-12T03:18:15.892617+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:18:15.892617+0000","last_undegraded":"2024-09-12T03:18:15.892617+0000","last_fullsized":"2024-09-12T03:18:15.892617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225722+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:13.225722+0000","last_peered":"2024-09-12T03:18:13.225722+0000","last_clean":"2024-09-12T03:18:13.225722+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:13.225722+0000","last_undegraded":"2024-09-12T03:18:13.225722+0000","last_fullsized":"2024-09-12T03:18:13.225722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229248+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:12.229248+0000","last_peered":"2024-09-12T03:18:12.229248+0000","last_clean":"2024-09-12T03:18:12.229248+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:12.229248+0000","last_undegraded":"2024-09-12T03:18:12.229248+0000","last_fullsized":"2024-09-12T03:18:12.229248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225060+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:13.225060+0000","last_peered":"2024-09-12T03:18:13.225060+0000","last_clean":"2024-09-12T03:18:13.225060+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:13.225060+0000","last_undegraded":"2024-09-12T03:18:13.225060+0000","last_fullsized":"2024-09-12T03:18:13.225060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221726+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:12.221726+0000","last_peered":"2024-09-12T03:18:12.221726+0000","last_clean":"2024-09-12T03:18:12.221726+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:12.221726+0000","last_undegraded":"2024-09-12T03:18:12.221726+0000","last_fullsized":"2024-09-12T03:18:12.221726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224091+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:13.224091+0000","last_peered":"2024-09-12T03:18:13.224091+0000","last_clean":"2024-09-12T03:18:13.224091+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:13.224091+0000","last_undegraded":"2024-09-12T03:18:13.224091+0000","last_fullsized":"2024-09-12T03:18:13.224091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":24,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229949+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:12.229949+0000","last_peered":"2024-09-12T03:18:12.229949+0000","last_clean":"2024-09-12T03:18:12.229949+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:12.229949+0000","last_undegraded":"2024-09-12T03:18:12.229949+0000","last_fullsized":"2024-09-12T03:18:12.229949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235706+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:13.235706+0000","last_peered":"2024-09-12T03:18:13.235706+0000","last_clean":"2024-09-12T03:18:13.235706+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:13.235706+0000","last_undegraded":"2024-09-12T03:18:13.235706+0000","last_fullsized":"2024-09-12T03:18:13.235706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:16.236651+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:18:16.236651+0000","last_peered":"2024-09-12T03:18:16.236651+0000","last_clean":"2024-09-12T03:18:16.236651+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:18:16.236651+0000","last_undegraded":"2024-09-12T03:18:16.236651+0000","last_fullsized":"2024-09-12T03:18:16.236651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224420+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:13.224420+0000","last_peered":"2024-09-12T03:18:13.224420+0000","last_clean":"2024-09-12T03:18:13.224420+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:13.224420+0000","last_undegraded":"2024-09-12T03:18:13.224420+0000","last_fullsized":"2024-09-12T03:18:13.224420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223799+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223799+0000","last_peered":"2024-09-12T03:18:13.223799+0000","last_clean":"2024-09-12T03:18:13.223799+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223799+0000","last_undegraded":"2024-09-12T03:18:13.223799+0000","last_fullsized":"2024-09-12T03:18:13.223799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223939+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.223939+0000","last_peered":"2024-09-12T03:18:13.223939+0000","last_clean":"2024-09-12T03:18:13.223939+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.223939+0000","last_undegraded":"2024-09-12T03:18:13.223939+0000","last_fullsized":"2024-09-12T03:18:13.223939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227978+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:13.227978+0000","last_peered":"2024-09-12T03:18:13.227978+0000","last_clean":"2024-09-12T03:18:13.227978+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:13.227978+0000","last_undegraded":"2024-09-12T03:18:13.227978+0000","last_fullsized":"2024-09-12T03:18:13.227978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223720+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223720+0000","last_peered":"2024-09-12T03:18:13.223720+0000","last_clean":"2024-09-12T03:18:13.223720+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223720+0000","last_undegraded":"2024-09-12T03:18:13.223720+0000","last_fullsized":"2024-09-12T03:18:13.223720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221272+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:12.221272+0000","last_peered":"2024-09-12T03:18:12.221272+0000","last_clean":"2024-09-12T03:18:12.221272+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:12.221272+0000","last_undegraded":"2024-09-12T03:18:12.221272+0000","last_fullsized":"2024-09-12T03:18:12.221272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":80,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221476+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:18:12.221476+0000","last_peered":"2024-09-12T03:18:12.221476+0000","last_clean":"2024-09-12T03:18:12.221476+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:18:12.221476+0000","last_undegraded":"2024-09-12T03:18:12.221476+0000","last_fullsized":"2024-09-12T03:18:12.221476+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.229219+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:12.229219+0000","last_peered":"2024-09-12T03:18:12.229219+0000","last_clean":"2024-09-12T03:18:12.229219+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:12.229219+0000","last_undegraded":"2024-09-12T03:18:12.229219+0000","last_fullsized":"2024-09-12T03:18:12.229219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.916882+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:18:15.916882+0000","last_peered":"2024-09-12T03:18:15.916882+0000","last_clean":"2024-09-12T03:18:15.916882+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:18:15.916882+0000","last_undegraded":"2024-09-12T03:18:15.916882+0000","last_fullsized":"2024-09-12T03:18:15.916882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221810+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:13.221810+0000","last_peered":"2024-09-12T03:18:13.221810+0000","last_clean":"2024-09-12T03:18:13.221810+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:13.221810+0000","last_undegraded":"2024-09-12T03:18:13.221810+0000","last_fullsized":"2024-09-12T03:18:13.221810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223007+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223007+0000","last_peered":"2024-09-12T03:18:13.223007+0000","last_clean":"2024-09-12T03:18:13.223007+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223007+0000","last_undegraded":"2024-09-12T03:18:13.223007+0000","last_fullsized":"2024-09-12T03:18:13.223007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222321+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:13.222321+0000","last_peered":"2024-09-12T03:18:13.222321+0000","last_clean":"2024-09-12T03:18:13.222321+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:13.222321+0000","last_undegraded":"2024-09-12T03:18:13.222321+0000","last_fullsized":"2024-09-12T03:18:13.222321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224119+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224119+0000","last_peered":"2024-09-12T03:18:13.224119+0000","last_clean":"2024-09-12T03:18:13.224119+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224119+0000","last_undegraded":"2024-09-12T03:18:13.224119+0000","last_fullsized":"2024-09-12T03:18:13.224119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-12T03:18:12.221604+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:12.221604+0000","last_peered":"2024-09-12T03:18:12.221604+0000","last_clean":"2024-09-12T03:18:12.221604+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:12.221604+0000","last_undegraded":"2024-09-12T03:18:12.221604+0000","last_fullsized":"2024-09-12T03:18:12.221604+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222863+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:13.222863+0000","last_peered":"2024-09-12T03:18:13.222863+0000","last_clean":"2024-09-12T03:18:13.222863+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:13.222863+0000","last_undegraded":"2024-09-12T03:18:13.222863+0000","last_fullsized":"2024-09-12T03:18:13.222863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224342+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224342+0000","last_peered":"2024-09-12T03:18:13.224342+0000","last_clean":"2024-09-12T03:18:13.224342+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224342+0000","last_undegraded":"2024-09-12T03:18:13.224342+0000","last_fullsized":"2024-09-12T03:18:13.224342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226157+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:13.226157+0000","last_peered":"2024-09-12T03:18:13.226157+0000","last_clean":"2024-09-12T03:18:13.226157+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:13.226157+0000","last_undegraded":"2024-09-12T03:18:13.226157+0000","last_fullsized":"2024-09-12T03:18:13.226157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230600+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230600+0000","last_peered":"2024-09-12T03:18:13.230600+0000","last_clean":"2024-09-12T03:18:13.230600+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230600+0000","last_undegraded":"2024-09-12T03:18:13.230600+0000","last_fullsized":"2024-09-12T03:18:13.230600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223193+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:13.223193+0000","last_peered":"2024-09-12T03:18:13.223193+0000","last_clean":"2024-09-12T03:18:13.223193+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:13.223193+0000","last_undegraded":"2024-09-12T03:18:13.223193+0000","last_fullsized":"2024-09-12T03:18:13.223193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225267+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:13.225267+0000","last_peered":"2024-09-12T03:18:13.225267+0000","last_clean":"2024-09-12T03:18:13.225267+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:13.225267+0000","last_undegraded":"2024-09-12T03:18:13.225267+0000","last_fullsized":"2024-09-12T03:18:13.225267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228053+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:13.228053+0000","last_peered":"2024-09-12T03:18:13.228053+0000","last_clean":"2024-09-12T03:18:13.228053+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:13.228053+0000","last_undegraded":"2024-09-12T03:18:13.228053+0000","last_fullsized":"2024-09-12T03:18:13.228053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223554+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:13.223554+0000","last_peered":"2024-09-12T03:18:13.223554+0000","last_clean":"2024-09-12T03:18:13.223554+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:13.223554+0000","last_undegraded":"2024-09-12T03:18:13.223554+0000","last_fullsized":"2024-09-12T03:18:13.223554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":80,"state":"peering","last_fresh":"2024-09-12T03:17:13.121382+0000","last_change":"2024-09-12T03:18:12.228851+0000","last_active":"2024-09-12T03:18:12.228851+0000","last_peered":"2024-09-12T03:17:13.121382+0000","last_clean":"2024-09-12T03:17:13.121382+0000","last_became_active":"2024-09-12T03:17:09.921491+0000","last_became_peered":"2024-09-12T03:17:09.921491+0000","last_unstale":"2024-09-12T03:17:13.121382+0000","last_undegraded":"2024-09-12T03:17:13.121382+0000","last_fullsized":"2024-09-12T03:17:13.121382+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,4,8],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225732+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:13.225732+0000","last_peered":"2024-09-12T03:18:13.225732+0000","last_clean":"2024-09-12T03:18:13.225732+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:13.225732+0000","last_undegraded":"2024-09-12T03:18:13.225732+0000","last_fullsized":"2024-09-12T03:18:13.225732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228427+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228427+0000","last_peered":"2024-09-12T03:18:13.228427+0000","last_clean":"2024-09-12T03:18:13.228427+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228427+0000","last_undegraded":"2024-09-12T03:18:13.228427+0000","last_fullsized":"2024-09-12T03:18:13.228427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228289+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:13.228289+0000","last_peered":"2024-09-12T03:18:13.228289+0000","last_clean":"2024-09-12T03:18:13.228289+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:13.228289+0000","last_undegraded":"2024-09-12T03:18:13.228289+0000","last_fullsized":"2024-09-12T03:18:13.228289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.913793+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:18:13.913793+0000","last_peered":"2024-09-12T03:18:13.913793+0000","last_clean":"2024-09-12T03:18:13.913793+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:18:13.913793+0000","last_undegraded":"2024-09-12T03:18:13.913793+0000","last_fullsized":"2024-09-12T03:18:13.913793+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":55,"num_read_kb":55,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":538,"num_read_kb":364,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200916,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808863,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":1089115,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972389,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1031150,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626485,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":897024,"data_stored":429117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855418,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":974848,"data_stored":1021976,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444106,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497696,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":966656,"data_stored":498253,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61516,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60959,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61893,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:18:19.764 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-12T03:18:19.764 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 14, 'reported_epoch': 80, 'state': 'peering', 'last_fresh': '2024-09-12T03:17:13.121382+0000', 'last_change': '2024-09-12T03:18:12.228851+0000', 'last_active': '2024-09-12T03:18:12.228851+0000', 'last_peered': '2024-09-12T03:17:13.121382+0000', 'last_clean': '2024-09-12T03:17:13.121382+0000', 'last_became_active': '2024-09-12T03:17:09.921491+0000', 'last_became_peered': '2024-09-12T03:17:09.921491+0000', 'last_unstale': '2024-09-12T03:17:13.121382+0000', 'last_undegraded': '2024-09-12T03:17:13.121382+0000', 'last_fullsized': '2024-09-12T03:17:13.121382+0000', 'mapping_epoch': 81, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 79, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-12T03:17:08.894426+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-12T03:17:08.894426+0000', 'last_clean_scrub_stamp': '2024-09-12T03:17:08.894426+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 8, 1], 'acting': [5, 8, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 4, 8], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-12T03:18:19.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph status --format=json 2024-09-12T03:18:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:20.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:19 smithi135 ceph-mon[35474]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:20.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:19 smithi121 ceph-mon[34963]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 -- 172.21.15.19:0/3809805393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c101420 msgr2=0x7f7b7c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 --2- 172.21.15.19:0/3809805393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c101420 0x7f7b7c10d950 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009cd0 tx=0x7f7b7402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 -- 172.21.15.19:0/3809805393 shutdown_connections 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 --2- 172.21.15.19:0/3809805393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c101420 0x7f7b7c10d950 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 --2- 172.21.15.19:0/3809805393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c100a80 0x7f7b7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 --2- 172.21.15.19:0/3809805393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c106a80 0x7f7b7c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 -- 172.21.15.19:0/3809805393 >> 172.21.15.19:0/3809805393 conn(0x7f7b7c0fc7a0 msgr2=0x7f7b7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 -- 172.21.15.19:0/3809805393 shutdown_connections 2024-09-12T03:18:20.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 -- 172.21.15.19:0/3809805393 wait complete. 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.521+0000 7f7b81b11640 1 Processor -- start 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 -- start start 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c100a80 0x7f7b7c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 0x7f7b7c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c110ef0 con 0x7f7b7c106a80 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c110d70 con 0x7f7b7c100a80 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b81b11640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c111070 con 0x7f7b7c101420 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59448/0 (socket says 172.21.15.19:59448) 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 -- 172.21.15.19:0/2456850916 learned_addr learned my addr 172.21.15.19:0/2456850916 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7b7fe640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c100a80 0x7f7b7c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7affd640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 0x7f7b7c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 -- 172.21.15.19:0/2456850916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 msgr2=0x7f7b7c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 0x7f7b7c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 -- 172.21.15.19:0/2456850916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c100a80 msgr2=0x7f7b7c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c100a80 0x7f7b7c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7bfff640 1 -- 172.21.15.19:0/2456850916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c1a46e0 con 0x7f7b7c106a80 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.522+0000 7f7b7affd640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 0x7f7b7c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b7bfff640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009ca0 tx=0x7f7b74003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74004240 con 0x7f7b7c106a80 2024-09-12T03:18:20.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b740305a0 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c1a4970 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7403a660 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.523+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c1a4eb0 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.524+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b7c100ee0 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.525+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7b74030750 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.526+0000 7f7b78ff9640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 0x7f7b680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.526+0000 7f7b7b7fe640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 0x7f7b680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.526+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7b7402e030 con 0x7f7b7c106a80 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.526+0000 7f7b7b7fe640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 0x7f7b680a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c005e10 tx=0x7f7b6c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.529+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b740bd4d0 con 0x7f7b7c106a80 2024-09-12T03:18:20.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.775+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7b7c1a5140 con 0x7f7b7c106a80 2024-09-12T03:18:20.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:20 smithi019 ceph-mon[30953]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:20.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:20 smithi019 ceph-mon[30953]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:18:20.775 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:20 smithi019 ceph-mon[30953]: Cluster is now healthy 2024-09-12T03:18:20.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.775+0000 7f7b78ff9640 1 -- 172.21.15.19:0/2456850916 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3182 (secure 0 0 0) 0x7f7b740bcdd0 con 0x7f7b7c106a80 2024-09-12T03:18:20.776 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:20.776 INFO:teuthology.orchestra.run.smithi019.stdout:{"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":442,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1726110975,"num_in_osds":12,"osd_in_since":1726110952,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":657574,"bytes_used":364834816,"bytes_avail":1151575592960,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":236,"recovering_keys_per_sec":0,"num_objects_recovered":11,"num_bytes_recovered":1891,"num_keys_recovered":0,"read_bytes_sec":3327,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14577},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14589},{"filesystem_id":1,"rank":4,"name":"k","status":"up:active","gid":24407},{"filesystem_id":1,"rank":3,"name":"l","status":"up:active","gid":24421},{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24422},{"filesystem_id":1,"rank":1,"name":"h","status":"up:standby-replay","gid":24443},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24457},{"filesystem_id":1,"rank":2,"name":"c","status":"up:active","gid":24463},{"filesystem_id":1,"rank":1,"name":"b","status":"up:active","gid":24467},{"filesystem_id":1,"rank":0,"name":"i","status":"up:active","gid":24487}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.19:8443/"}},"servicemap":{"epoch":21,"modified":"2024-09-12T03:18:15.293921+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 msgr2=0x7f7b680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 0x7f7b680a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c005e10 tx=0x7f7b6c005da0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 msgr2=0x7f7b7c199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009ca0 tx=0x7f7b74003ba0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 shutdown_connections 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b680a2c10 0x7f7b680a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7b7c106a80 0x7f7b7c199da0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7b7c101420 0x7f7b7c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 --2- 172.21.15.19:0/2456850916 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b7c100a80 0x7f7b7c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 >> 172.21.15.19:0/2456850916 conn(0x7f7b7c0fc7a0 msgr2=0x7f7b7c0fcc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 shutdown_connections 2024-09-12T03:18:20.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:20.778+0000 7f7b81b11640 1 -- 172.21.15.19:0/2456850916 wait complete. 2024-09-12T03:18:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:20 smithi135 ceph-mon[35474]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:20 smithi135 ceph-mon[35474]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:18:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:20 smithi135 ceph-mon[35474]: Cluster is now healthy 2024-09-12T03:18:21.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:20 smithi121 ceph-mon[34963]: pgmap v316: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1 op/s; 236 B/s, 1 objects/s recovering 2024-09-12T03:18:21.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:20 smithi121 ceph-mon[34963]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-12T03:18:21.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:20 smithi121 ceph-mon[34963]: Cluster is now healthy 2024-09-12T03:18:21.488 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-12T03:18:21.488 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:18:21.751 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:21.872 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2456850916' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:18:22.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2456850916' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:18:22.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2456850916' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-12T03:18:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1132562723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 msgr2=0x7fd7d01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1132562723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d01067f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009900 tx=0x7fd7c002a410 comp rx=0 tx=0).stop 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1132562723 shutdown_connections 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1132562723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0106f20 0x7fd7d010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1132562723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d01067f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1132562723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0101df0 0x7fd7d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1132562723 >> 172.21.15.19:0/1132562723 conn(0x7fd7d00fd8f0 msgr2=0x7fd7d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1132562723 shutdown_connections 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1132562723 wait complete. 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 Processor -- start 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.267+0000 7fd7d6cd5640 1 -- start start 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 0x7fd7d0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0106f20 0x7fd7d019ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0111fa0 con 0x7fd7d01027a0 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0111e20 con 0x7fd7d0106f20 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d6cd5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0112120 con 0x7fd7d0101df0 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59464/0 (socket says 172.21.15.19:59464) 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 -- 172.21.15.19:0/1947425418 learned_addr learned my addr 172.21.15.19:0/1947425418 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d5272640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0106f20 0x7fd7d019ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d4a71640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 0x7fd7d0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 -- 172.21.15.19:0/1947425418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 msgr2=0x7fd7d0198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 0x7fd7d0198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 -- 172.21.15.19:0/1947425418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0106f20 msgr2=0x7fd7d019ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0106f20 0x7fd7d019ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 -- 172.21.15.19:0/1947425418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d019d530 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7d4a71640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 0x7fd7d0198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.268+0000 7fd7c7fff640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006010 tx=0x7fd7c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.269+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c003f030 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.269+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d019d7c0 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.269+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7c0003ce0 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.269+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0035070 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.269+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d01a5100 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.270+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d01021d0 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.271+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd7c0042030 con 0x7fd7d01027a0 2024-09-12T03:18:22.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.271+0000 7fd7c5ffb640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 0x7fd7b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.272+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd7c00efb40 con 0x7fd7d01027a0 2024-09-12T03:18:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.272+0000 7fd7d4a71640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 0x7fd7b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.272+0000 7fd7d4a71640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 0x7fd7b80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc005fd0 tx=0x7fd7cc00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:22.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.275+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7c00bc9b0 con 0x7fd7d01027a0 2024-09-12T03:18:22.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.446+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7d0062630 con 0x7fd7b80a2c10 2024-09-12T03:18:22.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.452+0000 7fd7c5ffb640 1 -- 172.21.15.19:0/1947425418 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258991 (secure 0 0 0) 0x7fd7d0062630 con 0x7fd7b80a2c10 2024-09-12T03:18:22.455 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:22.456 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:18:22.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.455+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 msgr2=0x7fd7b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.455+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 0x7fd7b80a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc005fd0 tx=0x7fd7cc00b5f0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.455+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 msgr2=0x7fd7d0198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.455+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006010 tx=0x7fd7c00042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 shutdown_connections 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd7d0106f20 0x7fd7d019ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd7b80a2c10 0x7fd7b80a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd7d01027a0 0x7fd7d0198ac0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 --2- 172.21.15.19:0/1947425418 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd7d0101df0 0x7fd7d0198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 >> 172.21.15.19:0/1947425418 conn(0x7fd7d00fd8f0 msgr2=0x7fd7d00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:22.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 shutdown_connections 2024-09-12T03:18:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:22.456+0000 7fd7d6cd5640 1 -- 172.21.15.19:0/1947425418 wait complete. 2024-09-12T03:18:22.900 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:22 smithi019 ceph-mon[30953]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3 op/s; 208 B/s, 1 objects/s recovering 2024-09-12T03:18:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:22 smithi135 ceph-mon[35474]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3 op/s; 208 B/s, 1 objects/s recovering 2024-09-12T03:18:23.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:22 smithi121 ceph-mon[34963]: pgmap v317: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 3 op/s; 208 B/s, 1 objects/s recovering 2024-09-12T03:18:23.151 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-12T03:18:21.295362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":699,"num_read_kb":498,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356284,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585540,"statfs":{"total":1151940427776,"available":1151575592960,"internally_reserved":0,"allocated":11390976,"data_stored":7213566,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":46,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.075274"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225942+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:13.225942+0000","last_peered":"2024-09-12T03:18:13.225942+0000","last_clean":"2024-09-12T03:18:13.225942+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:13.225942+0000","last_undegraded":"2024-09-12T03:18:13.225942+0000","last_fullsized":"2024-09-12T03:18:13.225942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226078+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:13.226078+0000","last_peered":"2024-09-12T03:18:13.226078+0000","last_clean":"2024-09-12T03:18:13.226078+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:13.226078+0000","last_undegraded":"2024-09-12T03:18:13.226078+0000","last_fullsized":"2024-09-12T03:18:13.226078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225971+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:13.225971+0000","last_peered":"2024-09-12T03:18:13.225971+0000","last_clean":"2024-09-12T03:18:13.225971+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:13.225971+0000","last_undegraded":"2024-09-12T03:18:13.225971+0000","last_fullsized":"2024-09-12T03:18:13.225971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227583+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:13.227583+0000","last_peered":"2024-09-12T03:18:13.227583+0000","last_clean":"2024-09-12T03:18:13.227583+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:13.227583+0000","last_undegraded":"2024-09-12T03:18:13.227583+0000","last_fullsized":"2024-09-12T03:18:13.227583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223544+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:13.223544+0000","last_peered":"2024-09-12T03:18:13.223544+0000","last_clean":"2024-09-12T03:18:13.223544+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:13.223544+0000","last_undegraded":"2024-09-12T03:18:13.223544+0000","last_fullsized":"2024-09-12T03:18:13.223544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221903+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:13.221903+0000","last_peered":"2024-09-12T03:18:13.221903+0000","last_clean":"2024-09-12T03:18:13.221903+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:13.221903+0000","last_undegraded":"2024-09-12T03:18:13.221903+0000","last_fullsized":"2024-09-12T03:18:13.221903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223234+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:13.223234+0000","last_peered":"2024-09-12T03:18:13.223234+0000","last_clean":"2024-09-12T03:18:13.223234+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:13.223234+0000","last_undegraded":"2024-09-12T03:18:13.223234+0000","last_fullsized":"2024-09-12T03:18:13.223234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227222+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:13.227222+0000","last_peered":"2024-09-12T03:18:13.227222+0000","last_clean":"2024-09-12T03:18:13.227222+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:13.227222+0000","last_undegraded":"2024-09-12T03:18:13.227222+0000","last_fullsized":"2024-09-12T03:18:13.227222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230458+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:13.230458+0000","last_peered":"2024-09-12T03:18:13.230458+0000","last_clean":"2024-09-12T03:18:13.230458+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:13.230458+0000","last_undegraded":"2024-09-12T03:18:13.230458+0000","last_fullsized":"2024-09-12T03:18:13.230458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222751+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:13.222751+0000","last_peered":"2024-09-12T03:18:13.222751+0000","last_clean":"2024-09-12T03:18:13.222751+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:13.222751+0000","last_undegraded":"2024-09-12T03:18:13.222751+0000","last_fullsized":"2024-09-12T03:18:13.222751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224323+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:13.224323+0000","last_peered":"2024-09-12T03:18:13.224323+0000","last_clean":"2024-09-12T03:18:13.224323+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:13.224323+0000","last_undegraded":"2024-09-12T03:18:13.224323+0000","last_fullsized":"2024-09-12T03:18:13.224323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224385+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:13.224385+0000","last_peered":"2024-09-12T03:18:13.224385+0000","last_clean":"2024-09-12T03:18:13.224385+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:13.224385+0000","last_undegraded":"2024-09-12T03:18:13.224385+0000","last_fullsized":"2024-09-12T03:18:13.224385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220415+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:13.220415+0000","last_peered":"2024-09-12T03:18:13.220415+0000","last_clean":"2024-09-12T03:18:13.220415+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:13.220415+0000","last_undegraded":"2024-09-12T03:18:13.220415+0000","last_fullsized":"2024-09-12T03:18:13.220415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225501+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:13.225501+0000","last_peered":"2024-09-12T03:18:13.225501+0000","last_clean":"2024-09-12T03:18:13.225501+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:13.225501+0000","last_undegraded":"2024-09-12T03:18:13.225501+0000","last_fullsized":"2024-09-12T03:18:13.225501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225258+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:13.225258+0000","last_peered":"2024-09-12T03:18:13.225258+0000","last_clean":"2024-09-12T03:18:13.225258+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:13.225258+0000","last_undegraded":"2024-09-12T03:18:13.225258+0000","last_fullsized":"2024-09-12T03:18:13.225258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226050+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:13.226050+0000","last_peered":"2024-09-12T03:18:13.226050+0000","last_clean":"2024-09-12T03:18:13.226050+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:13.226050+0000","last_undegraded":"2024-09-12T03:18:13.226050+0000","last_fullsized":"2024-09-12T03:18:13.226050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222851+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:13.222851+0000","last_peered":"2024-09-12T03:18:13.222851+0000","last_clean":"2024-09-12T03:18:13.222851+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:13.222851+0000","last_undegraded":"2024-09-12T03:18:13.222851+0000","last_fullsized":"2024-09-12T03:18:13.222851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221546+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:13.221546+0000","last_peered":"2024-09-12T03:18:13.221546+0000","last_clean":"2024-09-12T03:18:13.221546+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:13.221546+0000","last_undegraded":"2024-09-12T03:18:13.221546+0000","last_fullsized":"2024-09-12T03:18:13.221546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227637+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:13.227637+0000","last_peered":"2024-09-12T03:18:13.227637+0000","last_clean":"2024-09-12T03:18:13.227637+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:13.227637+0000","last_undegraded":"2024-09-12T03:18:13.227637+0000","last_fullsized":"2024-09-12T03:18:13.227637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220468+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:13.220468+0000","last_peered":"2024-09-12T03:18:13.220468+0000","last_clean":"2024-09-12T03:18:13.220468+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:13.220468+0000","last_undegraded":"2024-09-12T03:18:13.220468+0000","last_fullsized":"2024-09-12T03:18:13.220468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230002+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:13.230002+0000","last_peered":"2024-09-12T03:18:13.230002+0000","last_clean":"2024-09-12T03:18:13.230002+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:13.230002+0000","last_undegraded":"2024-09-12T03:18:13.230002+0000","last_fullsized":"2024-09-12T03:18:13.230002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227500+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:13.227500+0000","last_peered":"2024-09-12T03:18:13.227500+0000","last_clean":"2024-09-12T03:18:13.227500+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:13.227500+0000","last_undegraded":"2024-09-12T03:18:13.227500+0000","last_fullsized":"2024-09-12T03:18:13.227500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225592+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:13.225592+0000","last_peered":"2024-09-12T03:18:13.225592+0000","last_clean":"2024-09-12T03:18:13.225592+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:13.225592+0000","last_undegraded":"2024-09-12T03:18:13.225592+0000","last_fullsized":"2024-09-12T03:18:13.225592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228434+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:13.228434+0000","last_peered":"2024-09-12T03:18:13.228434+0000","last_clean":"2024-09-12T03:18:13.228434+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:13.228434+0000","last_undegraded":"2024-09-12T03:18:13.228434+0000","last_fullsized":"2024-09-12T03:18:13.228434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221771+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:13.221771+0000","last_peered":"2024-09-12T03:18:13.221771+0000","last_clean":"2024-09-12T03:18:13.221771+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:13.221771+0000","last_undegraded":"2024-09-12T03:18:13.221771+0000","last_fullsized":"2024-09-12T03:18:13.221771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225026+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:13.225026+0000","last_peered":"2024-09-12T03:18:13.225026+0000","last_clean":"2024-09-12T03:18:13.225026+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:13.225026+0000","last_undegraded":"2024-09-12T03:18:13.225026+0000","last_fullsized":"2024-09-12T03:18:13.225026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220571+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:13.220571+0000","last_peered":"2024-09-12T03:18:13.220571+0000","last_clean":"2024-09-12T03:18:13.220571+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:13.220571+0000","last_undegraded":"2024-09-12T03:18:13.220571+0000","last_fullsized":"2024-09-12T03:18:13.220571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225810+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:13.225810+0000","last_peered":"2024-09-12T03:18:13.225810+0000","last_clean":"2024-09-12T03:18:13.225810+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:13.225810+0000","last_undegraded":"2024-09-12T03:18:13.225810+0000","last_fullsized":"2024-09-12T03:18:13.225810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221553+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:13.221553+0000","last_peered":"2024-09-12T03:18:13.221553+0000","last_clean":"2024-09-12T03:18:13.221553+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:13.221553+0000","last_undegraded":"2024-09-12T03:18:13.221553+0000","last_fullsized":"2024-09-12T03:18:13.221553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221469+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:13.221469+0000","last_peered":"2024-09-12T03:18:13.221469+0000","last_clean":"2024-09-12T03:18:13.221469+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:13.221469+0000","last_undegraded":"2024-09-12T03:18:13.221469+0000","last_fullsized":"2024-09-12T03:18:13.221469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225010+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:13.225010+0000","last_peered":"2024-09-12T03:18:13.225010+0000","last_clean":"2024-09-12T03:18:13.225010+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:13.225010+0000","last_undegraded":"2024-09-12T03:18:13.225010+0000","last_fullsized":"2024-09-12T03:18:13.225010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223333+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:13.223333+0000","last_peered":"2024-09-12T03:18:13.223333+0000","last_clean":"2024-09-12T03:18:13.223333+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:13.223333+0000","last_undegraded":"2024-09-12T03:18:13.223333+0000","last_fullsized":"2024-09-12T03:18:13.223333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":90,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:16.894343+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:18:16.894343+0000","last_peered":"2024-09-12T03:18:16.894343+0000","last_clean":"2024-09-12T03:18:16.894343+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:18:16.894343+0000","last_undegraded":"2024-09-12T03:18:16.894343+0000","last_fullsized":"2024-09-12T03:18:16.894343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226600+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:13.226600+0000","last_peered":"2024-09-12T03:18:13.226600+0000","last_clean":"2024-09-12T03:18:13.226600+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:13.226600+0000","last_undegraded":"2024-09-12T03:18:13.226600+0000","last_fullsized":"2024-09-12T03:18:13.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227325+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:13.227325+0000","last_peered":"2024-09-12T03:18:13.227325+0000","last_clean":"2024-09-12T03:18:13.227325+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:13.227325+0000","last_undegraded":"2024-09-12T03:18:13.227325+0000","last_fullsized":"2024-09-12T03:18:13.227325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222614+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:13.222614+0000","last_peered":"2024-09-12T03:18:13.222614+0000","last_clean":"2024-09-12T03:18:13.222614+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:13.222614+0000","last_undegraded":"2024-09-12T03:18:13.222614+0000","last_fullsized":"2024-09-12T03:18:13.222614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":145,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:19.240495+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:18:19.240495+0000","last_peered":"2024-09-12T03:18:19.240495+0000","last_clean":"2024-09-12T03:18:19.240495+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:18:19.240495+0000","last_undegraded":"2024-09-12T03:18:19.240495+0000","last_fullsized":"2024-09-12T03:18:19.240495+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230681+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:13.230681+0000","last_peered":"2024-09-12T03:18:13.230681+0000","last_clean":"2024-09-12T03:18:13.230681+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:13.230681+0000","last_undegraded":"2024-09-12T03:18:13.230681+0000","last_fullsized":"2024-09-12T03:18:13.230681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225820+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:13.225820+0000","last_peered":"2024-09-12T03:18:13.225820+0000","last_clean":"2024-09-12T03:18:13.225820+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:13.225820+0000","last_undegraded":"2024-09-12T03:18:13.225820+0000","last_fullsized":"2024-09-12T03:18:13.225820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230416+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:13.230416+0000","last_peered":"2024-09-12T03:18:13.230416+0000","last_clean":"2024-09-12T03:18:13.230416+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:13.230416+0000","last_undegraded":"2024-09-12T03:18:13.230416+0000","last_fullsized":"2024-09-12T03:18:13.230416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":182,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:19.898059+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:18:19.898059+0000","last_peered":"2024-09-12T03:18:19.898059+0000","last_clean":"2024-09-12T03:18:19.898059+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:18:19.898059+0000","last_undegraded":"2024-09-12T03:18:19.898059+0000","last_fullsized":"2024-09-12T03:18:19.898059+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":130,"num_read_kb":123,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223911+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:13.223911+0000","last_peered":"2024-09-12T03:18:13.223911+0000","last_clean":"2024-09-12T03:18:13.223911+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:13.223911+0000","last_undegraded":"2024-09-12T03:18:13.223911+0000","last_fullsized":"2024-09-12T03:18:13.223911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220250+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:13.220250+0000","last_peered":"2024-09-12T03:18:13.220250+0000","last_clean":"2024-09-12T03:18:13.220250+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:13.220250+0000","last_undegraded":"2024-09-12T03:18:13.220250+0000","last_fullsized":"2024-09-12T03:18:13.220250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223375+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:13.223375+0000","last_peered":"2024-09-12T03:18:13.223375+0000","last_clean":"2024-09-12T03:18:13.223375+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:13.223375+0000","last_undegraded":"2024-09-12T03:18:13.223375+0000","last_fullsized":"2024-09-12T03:18:13.223375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.892617+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:18:15.892617+0000","last_peered":"2024-09-12T03:18:15.892617+0000","last_clean":"2024-09-12T03:18:15.892617+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:18:15.892617+0000","last_undegraded":"2024-09-12T03:18:15.892617+0000","last_fullsized":"2024-09-12T03:18:15.892617+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":62,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225722+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:13.225722+0000","last_peered":"2024-09-12T03:18:13.225722+0000","last_clean":"2024-09-12T03:18:13.225722+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:13.225722+0000","last_undegraded":"2024-09-12T03:18:13.225722+0000","last_fullsized":"2024-09-12T03:18:13.225722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225695+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:13.225695+0000","last_peered":"2024-09-12T03:18:13.225695+0000","last_clean":"2024-09-12T03:18:13.225695+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:13.225695+0000","last_undegraded":"2024-09-12T03:18:13.225695+0000","last_fullsized":"2024-09-12T03:18:13.225695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225060+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:13.225060+0000","last_peered":"2024-09-12T03:18:13.225060+0000","last_clean":"2024-09-12T03:18:13.225060+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:13.225060+0000","last_undegraded":"2024-09-12T03:18:13.225060+0000","last_fullsized":"2024-09-12T03:18:13.225060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224091+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:13.224091+0000","last_peered":"2024-09-12T03:18:13.224091+0000","last_clean":"2024-09-12T03:18:13.224091+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:13.224091+0000","last_undegraded":"2024-09-12T03:18:13.224091+0000","last_fullsized":"2024-09-12T03:18:13.224091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235706+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:13.235706+0000","last_peered":"2024-09-12T03:18:13.235706+0000","last_clean":"2024-09-12T03:18:13.235706+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:13.235706+0000","last_undegraded":"2024-09-12T03:18:13.235706+0000","last_fullsized":"2024-09-12T03:18:13.235706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:16.236651+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:18:16.236651+0000","last_peered":"2024-09-12T03:18:16.236651+0000","last_clean":"2024-09-12T03:18:16.236651+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:18:16.236651+0000","last_undegraded":"2024-09-12T03:18:16.236651+0000","last_fullsized":"2024-09-12T03:18:16.236651+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":5,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224420+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:13.224420+0000","last_peered":"2024-09-12T03:18:13.224420+0000","last_clean":"2024-09-12T03:18:13.224420+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:13.224420+0000","last_undegraded":"2024-09-12T03:18:13.224420+0000","last_fullsized":"2024-09-12T03:18:13.224420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223799+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223799+0000","last_peered":"2024-09-12T03:18:13.223799+0000","last_clean":"2024-09-12T03:18:13.223799+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223799+0000","last_undegraded":"2024-09-12T03:18:13.223799+0000","last_fullsized":"2024-09-12T03:18:13.223799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223939+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.223939+0000","last_peered":"2024-09-12T03:18:13.223939+0000","last_clean":"2024-09-12T03:18:13.223939+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.223939+0000","last_undegraded":"2024-09-12T03:18:13.223939+0000","last_fullsized":"2024-09-12T03:18:13.223939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227978+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:13.227978+0000","last_peered":"2024-09-12T03:18:13.227978+0000","last_clean":"2024-09-12T03:18:13.227978+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:13.227978+0000","last_undegraded":"2024-09-12T03:18:13.227978+0000","last_fullsized":"2024-09-12T03:18:13.227978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223720+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223720+0000","last_peered":"2024-09-12T03:18:13.223720+0000","last_clean":"2024-09-12T03:18:13.223720+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223720+0000","last_undegraded":"2024-09-12T03:18:13.223720+0000","last_fullsized":"2024-09-12T03:18:13.223720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220163+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:13.220163+0000","last_peered":"2024-09-12T03:18:13.220163+0000","last_clean":"2024-09-12T03:18:13.220163+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:13.220163+0000","last_undegraded":"2024-09-12T03:18:13.220163+0000","last_fullsized":"2024-09-12T03:18:13.220163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":87,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:16.893475+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:18:16.893475+0000","last_peered":"2024-09-12T03:18:16.893475+0000","last_clean":"2024-09-12T03:18:16.893475+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:18:16.893475+0000","last_undegraded":"2024-09-12T03:18:16.893475+0000","last_fullsized":"2024-09-12T03:18:16.893475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":60,"num_read_kb":60,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225647+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:13.225647+0000","last_peered":"2024-09-12T03:18:13.225647+0000","last_clean":"2024-09-12T03:18:13.225647+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:13.225647+0000","last_undegraded":"2024-09-12T03:18:13.225647+0000","last_fullsized":"2024-09-12T03:18:13.225647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":84,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:15.916882+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:18:15.916882+0000","last_peered":"2024-09-12T03:18:15.916882+0000","last_clean":"2024-09-12T03:18:15.916882+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:18:15.916882+0000","last_undegraded":"2024-09-12T03:18:15.916882+0000","last_fullsized":"2024-09-12T03:18:15.916882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221810+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:13.221810+0000","last_peered":"2024-09-12T03:18:13.221810+0000","last_clean":"2024-09-12T03:18:13.221810+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:13.221810+0000","last_undegraded":"2024-09-12T03:18:13.221810+0000","last_fullsized":"2024-09-12T03:18:13.221810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223007+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223007+0000","last_peered":"2024-09-12T03:18:13.223007+0000","last_clean":"2024-09-12T03:18:13.223007+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223007+0000","last_undegraded":"2024-09-12T03:18:13.223007+0000","last_fullsized":"2024-09-12T03:18:13.223007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222321+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:13.222321+0000","last_peered":"2024-09-12T03:18:13.222321+0000","last_clean":"2024-09-12T03:18:13.222321+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:13.222321+0000","last_undegraded":"2024-09-12T03:18:13.222321+0000","last_fullsized":"2024-09-12T03:18:13.222321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224119+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224119+0000","last_peered":"2024-09-12T03:18:13.224119+0000","last_clean":"2024-09-12T03:18:13.224119+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224119+0000","last_undegraded":"2024-09-12T03:18:13.224119+0000","last_fullsized":"2024-09-12T03:18:13.224119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220558+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:13.220558+0000","last_peered":"2024-09-12T03:18:13.220558+0000","last_clean":"2024-09-12T03:18:13.220558+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:13.220558+0000","last_undegraded":"2024-09-12T03:18:13.220558+0000","last_fullsized":"2024-09-12T03:18:13.220558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222863+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:13.222863+0000","last_peered":"2024-09-12T03:18:13.222863+0000","last_clean":"2024-09-12T03:18:13.222863+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:13.222863+0000","last_undegraded":"2024-09-12T03:18:13.222863+0000","last_fullsized":"2024-09-12T03:18:13.222863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224342+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224342+0000","last_peered":"2024-09-12T03:18:13.224342+0000","last_clean":"2024-09-12T03:18:13.224342+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224342+0000","last_undegraded":"2024-09-12T03:18:13.224342+0000","last_fullsized":"2024-09-12T03:18:13.224342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226157+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:13.226157+0000","last_peered":"2024-09-12T03:18:13.226157+0000","last_clean":"2024-09-12T03:18:13.226157+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:13.226157+0000","last_undegraded":"2024-09-12T03:18:13.226157+0000","last_fullsized":"2024-09-12T03:18:13.226157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230600+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230600+0000","last_peered":"2024-09-12T03:18:13.230600+0000","last_clean":"2024-09-12T03:18:13.230600+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230600+0000","last_undegraded":"2024-09-12T03:18:13.230600+0000","last_fullsized":"2024-09-12T03:18:13.230600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223193+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:13.223193+0000","last_peered":"2024-09-12T03:18:13.223193+0000","last_clean":"2024-09-12T03:18:13.223193+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:13.223193+0000","last_undegraded":"2024-09-12T03:18:13.223193+0000","last_fullsized":"2024-09-12T03:18:13.223193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225267+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:13.225267+0000","last_peered":"2024-09-12T03:18:13.225267+0000","last_clean":"2024-09-12T03:18:13.225267+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:13.225267+0000","last_undegraded":"2024-09-12T03:18:13.225267+0000","last_fullsized":"2024-09-12T03:18:13.225267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228053+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:13.228053+0000","last_peered":"2024-09-12T03:18:13.228053+0000","last_clean":"2024-09-12T03:18:13.228053+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:13.228053+0000","last_undegraded":"2024-09-12T03:18:13.228053+0000","last_fullsized":"2024-09-12T03:18:13.228053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223554+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:13.223554+0000","last_peered":"2024-09-12T03:18:13.223554+0000","last_clean":"2024-09-12T03:18:13.223554+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:13.223554+0000","last_undegraded":"2024-09-12T03:18:13.223554+0000","last_fullsized":"2024-09-12T03:18:13.223554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.231726+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:13.231726+0000","last_peered":"2024-09-12T03:18:13.231726+0000","last_clean":"2024-09-12T03:18:13.231726+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:13.231726+0000","last_undegraded":"2024-09-12T03:18:13.231726+0000","last_fullsized":"2024-09-12T03:18:13.231726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225732+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:13.225732+0000","last_peered":"2024-09-12T03:18:13.225732+0000","last_clean":"2024-09-12T03:18:13.225732+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:13.225732+0000","last_undegraded":"2024-09-12T03:18:13.225732+0000","last_fullsized":"2024-09-12T03:18:13.225732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228427+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228427+0000","last_peered":"2024-09-12T03:18:13.228427+0000","last_clean":"2024-09-12T03:18:13.228427+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228427+0000","last_undegraded":"2024-09-12T03:18:13.228427+0000","last_fullsized":"2024-09-12T03:18:13.228427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228289+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:13.228289+0000","last_peered":"2024-09-12T03:18:13.228289+0000","last_clean":"2024-09-12T03:18:13.228289+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:13.228289+0000","last_undegraded":"2024-09-12T03:18:13.228289+0000","last_fullsized":"2024-09-12T03:18:13.228289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":95,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:18.919946+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:18:18.919946+0000","last_peered":"2024-09-12T03:18:18.919946+0000","last_clean":"2024-09-12T03:18:18.919946+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:18:18.919946+0000","last_undegraded":"2024-09-12T03:18:18.919946+0000","last_fullsized":"2024-09-12T03:18:18.919946+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":60,"num_read_kb":60,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":573,"num_read_kb":389,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200916,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808864,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":1089115,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972390,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1031150,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":437576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855419,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1030435,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444107,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497696,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":966656,"data_stored":498253,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61516,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60959,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61893,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:18:24.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:23 smithi135 ceph-mon[35474]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:24.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:23 smithi121 ceph-mon[34963]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:24.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:23 smithi019 ceph-mon[30953]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:24 smithi135 ceph-mon[35474]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4 op/s; 187 B/s, 1 objects/s recovering 2024-09-12T03:18:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:24 smithi121 ceph-mon[34963]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4 op/s; 187 B/s, 1 objects/s recovering 2024-09-12T03:18:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:24 smithi019 ceph-mon[30953]: pgmap v318: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4 op/s; 187 B/s, 1 objects/s recovering 2024-09-12T03:18:26.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:18:26.429 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:26.840 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:26 smithi019 ceph-mon[30953]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 -- 172.21.15.19:0/221476687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 msgr2=0x7f586010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 --2- 172.21.15.19:0/221476687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f586010d950 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5858009cd0 tx=0x7f585802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 -- 172.21.15.19:0/221476687 shutdown_connections 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 --2- 172.21.15.19:0/221476687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f586010d950 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 --2- 172.21.15.19:0/221476687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 0x7f5860100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 --2- 172.21.15.19:0/221476687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f5860106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 -- 172.21.15.19:0/221476687 >> 172.21.15.19:0/221476687 conn(0x7f58600fc7a0 msgr2=0x7f58600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 -- 172.21.15.19:0/221476687 shutdown_connections 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 -- 172.21.15.19:0/221476687 wait complete. 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.922+0000 7f58655a7640 1 Processor -- start 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 -- start start 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 0x7f58601992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f58601a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:26.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5860110ef0 con 0x7f5860101420 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5860110d70 con 0x7f5860106a80 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f58655a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5860111070 con 0x7f5860100a80 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59476/0 (socket says 172.21.15.19:59476) 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 -- 172.21.15.19:0/1993107240 learned_addr learned my addr 172.21.15.19:0/1993107240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585f7fe640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f58601a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585effd640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 0x7f58601992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 -- 172.21.15.19:0/1993107240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 msgr2=0x7f58601992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 0x7f58601992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 -- 172.21.15.19:0/1993107240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 msgr2=0x7f58601a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f58601a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:26.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 -- 172.21.15.19:0/1993107240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58601a4890 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585f7fe640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f58601a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.923+0000 7f585e7fc640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f585400b4a0 tx=0x7f585400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.924+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854014070 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.924+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58601a4a60 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.924+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5854010af0 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.924+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854017d10 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.924+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58601a4fa0 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.925+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5860100ee0 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.926+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5854017070 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.927+0000 7f5847fff640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 0x7f58480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.927+0000 7f585effd640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 0x7f58480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.927+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58540ce4d0 con 0x7f5860101420 2024-09-12T03:18:26.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.927+0000 7f585effd640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 0x7f58480a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5850005d90 tx=0x7f5850005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:26.931+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f585409b3f0 con 0x7f5860101420 2024-09-12T03:18:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:26 smithi135 ceph-mon[35474]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:27.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.104+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58600fd570 con 0x7f58480a2c10 2024-09-12T03:18:27.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.110+0000 7f5847fff640 1 -- 172.21.15.19:0/1993107240 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258994 (secure 0 0 0) 0x7f58600fd570 con 0x7f58480a2c10 2024-09-12T03:18:27.112 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:27.113 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.113+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 msgr2=0x7f58480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.113+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 0x7f58480a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5850005d90 tx=0x7f5850005d20 comp rx=0 tx=0).stop 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.113+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 msgr2=0x7f5860199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.113+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f585400b4a0 tx=0x7f585400b960 comp rx=0 tx=0).stop 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 shutdown_connections 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5860106a80 0x7f58601a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58480a2c10 0x7f58480a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5860101420 0x7f5860199810 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 --2- 172.21.15.19:0/1993107240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5860100a80 0x7f58601992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:27.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 >> 172.21.15.19:0/1993107240 conn(0x7f58600fc7a0 msgr2=0x7f58600fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:27.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 shutdown_connections 2024-09-12T03:18:27.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:27.114+0000 7f58655a7640 1 -- 172.21.15.19:0/1993107240 wait complete. 2024-09-12T03:18:27.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:26 smithi121 ceph-mon[34963]: pgmap v319: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:27.842 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:27 smithi019 ceph-mon[30953]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:27.843 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-12T03:18:25.296551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657574,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":749,"num_read_kb":523,"num_write":426,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356284,"kb_used_data":11124,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585540,"statfs":{"total":1151940427776,"available":1151575592960,"internally_reserved":0,"allocated":11390976,"data_stored":7213566,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003126"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225942+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:13.225942+0000","last_peered":"2024-09-12T03:18:13.225942+0000","last_clean":"2024-09-12T03:18:13.225942+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:13.225942+0000","last_undegraded":"2024-09-12T03:18:13.225942+0000","last_fullsized":"2024-09-12T03:18:13.225942+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226078+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:13.226078+0000","last_peered":"2024-09-12T03:18:13.226078+0000","last_clean":"2024-09-12T03:18:13.226078+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:13.226078+0000","last_undegraded":"2024-09-12T03:18:13.226078+0000","last_fullsized":"2024-09-12T03:18:13.226078+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225971+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:13.225971+0000","last_peered":"2024-09-12T03:18:13.225971+0000","last_clean":"2024-09-12T03:18:13.225971+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:13.225971+0000","last_undegraded":"2024-09-12T03:18:13.225971+0000","last_fullsized":"2024-09-12T03:18:13.225971+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227583+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:13.227583+0000","last_peered":"2024-09-12T03:18:13.227583+0000","last_clean":"2024-09-12T03:18:13.227583+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:13.227583+0000","last_undegraded":"2024-09-12T03:18:13.227583+0000","last_fullsized":"2024-09-12T03:18:13.227583+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223544+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:13.223544+0000","last_peered":"2024-09-12T03:18:13.223544+0000","last_clean":"2024-09-12T03:18:13.223544+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:13.223544+0000","last_undegraded":"2024-09-12T03:18:13.223544+0000","last_fullsized":"2024-09-12T03:18:13.223544+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221903+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:13.221903+0000","last_peered":"2024-09-12T03:18:13.221903+0000","last_clean":"2024-09-12T03:18:13.221903+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:13.221903+0000","last_undegraded":"2024-09-12T03:18:13.221903+0000","last_fullsized":"2024-09-12T03:18:13.221903+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223234+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:13.223234+0000","last_peered":"2024-09-12T03:18:13.223234+0000","last_clean":"2024-09-12T03:18:13.223234+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:13.223234+0000","last_undegraded":"2024-09-12T03:18:13.223234+0000","last_fullsized":"2024-09-12T03:18:13.223234+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227222+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:13.227222+0000","last_peered":"2024-09-12T03:18:13.227222+0000","last_clean":"2024-09-12T03:18:13.227222+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:13.227222+0000","last_undegraded":"2024-09-12T03:18:13.227222+0000","last_fullsized":"2024-09-12T03:18:13.227222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230458+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:13.230458+0000","last_peered":"2024-09-12T03:18:13.230458+0000","last_clean":"2024-09-12T03:18:13.230458+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:13.230458+0000","last_undegraded":"2024-09-12T03:18:13.230458+0000","last_fullsized":"2024-09-12T03:18:13.230458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222751+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:13.222751+0000","last_peered":"2024-09-12T03:18:13.222751+0000","last_clean":"2024-09-12T03:18:13.222751+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:13.222751+0000","last_undegraded":"2024-09-12T03:18:13.222751+0000","last_fullsized":"2024-09-12T03:18:13.222751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224323+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:13.224323+0000","last_peered":"2024-09-12T03:18:13.224323+0000","last_clean":"2024-09-12T03:18:13.224323+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:13.224323+0000","last_undegraded":"2024-09-12T03:18:13.224323+0000","last_fullsized":"2024-09-12T03:18:13.224323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224385+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:13.224385+0000","last_peered":"2024-09-12T03:18:13.224385+0000","last_clean":"2024-09-12T03:18:13.224385+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:13.224385+0000","last_undegraded":"2024-09-12T03:18:13.224385+0000","last_fullsized":"2024-09-12T03:18:13.224385+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220415+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:13.220415+0000","last_peered":"2024-09-12T03:18:13.220415+0000","last_clean":"2024-09-12T03:18:13.220415+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:13.220415+0000","last_undegraded":"2024-09-12T03:18:13.220415+0000","last_fullsized":"2024-09-12T03:18:13.220415+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225501+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:13.225501+0000","last_peered":"2024-09-12T03:18:13.225501+0000","last_clean":"2024-09-12T03:18:13.225501+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:13.225501+0000","last_undegraded":"2024-09-12T03:18:13.225501+0000","last_fullsized":"2024-09-12T03:18:13.225501+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225258+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:13.225258+0000","last_peered":"2024-09-12T03:18:13.225258+0000","last_clean":"2024-09-12T03:18:13.225258+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:13.225258+0000","last_undegraded":"2024-09-12T03:18:13.225258+0000","last_fullsized":"2024-09-12T03:18:13.225258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226050+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:13.226050+0000","last_peered":"2024-09-12T03:18:13.226050+0000","last_clean":"2024-09-12T03:18:13.226050+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:13.226050+0000","last_undegraded":"2024-09-12T03:18:13.226050+0000","last_fullsized":"2024-09-12T03:18:13.226050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222851+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:13.222851+0000","last_peered":"2024-09-12T03:18:13.222851+0000","last_clean":"2024-09-12T03:18:13.222851+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:13.222851+0000","last_undegraded":"2024-09-12T03:18:13.222851+0000","last_fullsized":"2024-09-12T03:18:13.222851+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221546+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:13.221546+0000","last_peered":"2024-09-12T03:18:13.221546+0000","last_clean":"2024-09-12T03:18:13.221546+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:13.221546+0000","last_undegraded":"2024-09-12T03:18:13.221546+0000","last_fullsized":"2024-09-12T03:18:13.221546+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227637+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:13.227637+0000","last_peered":"2024-09-12T03:18:13.227637+0000","last_clean":"2024-09-12T03:18:13.227637+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:13.227637+0000","last_undegraded":"2024-09-12T03:18:13.227637+0000","last_fullsized":"2024-09-12T03:18:13.227637+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220468+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:13.220468+0000","last_peered":"2024-09-12T03:18:13.220468+0000","last_clean":"2024-09-12T03:18:13.220468+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:13.220468+0000","last_undegraded":"2024-09-12T03:18:13.220468+0000","last_fullsized":"2024-09-12T03:18:13.220468+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230002+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:13.230002+0000","last_peered":"2024-09-12T03:18:13.230002+0000","last_clean":"2024-09-12T03:18:13.230002+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:13.230002+0000","last_undegraded":"2024-09-12T03:18:13.230002+0000","last_fullsized":"2024-09-12T03:18:13.230002+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227500+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:13.227500+0000","last_peered":"2024-09-12T03:18:13.227500+0000","last_clean":"2024-09-12T03:18:13.227500+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:13.227500+0000","last_undegraded":"2024-09-12T03:18:13.227500+0000","last_fullsized":"2024-09-12T03:18:13.227500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225592+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:13.225592+0000","last_peered":"2024-09-12T03:18:13.225592+0000","last_clean":"2024-09-12T03:18:13.225592+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:13.225592+0000","last_undegraded":"2024-09-12T03:18:13.225592+0000","last_fullsized":"2024-09-12T03:18:13.225592+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228434+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:13.228434+0000","last_peered":"2024-09-12T03:18:13.228434+0000","last_clean":"2024-09-12T03:18:13.228434+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:13.228434+0000","last_undegraded":"2024-09-12T03:18:13.228434+0000","last_fullsized":"2024-09-12T03:18:13.228434+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221771+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:13.221771+0000","last_peered":"2024-09-12T03:18:13.221771+0000","last_clean":"2024-09-12T03:18:13.221771+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:13.221771+0000","last_undegraded":"2024-09-12T03:18:13.221771+0000","last_fullsized":"2024-09-12T03:18:13.221771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225026+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:13.225026+0000","last_peered":"2024-09-12T03:18:13.225026+0000","last_clean":"2024-09-12T03:18:13.225026+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:13.225026+0000","last_undegraded":"2024-09-12T03:18:13.225026+0000","last_fullsized":"2024-09-12T03:18:13.225026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220571+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:13.220571+0000","last_peered":"2024-09-12T03:18:13.220571+0000","last_clean":"2024-09-12T03:18:13.220571+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:13.220571+0000","last_undegraded":"2024-09-12T03:18:13.220571+0000","last_fullsized":"2024-09-12T03:18:13.220571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225810+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:13.225810+0000","last_peered":"2024-09-12T03:18:13.225810+0000","last_clean":"2024-09-12T03:18:13.225810+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:13.225810+0000","last_undegraded":"2024-09-12T03:18:13.225810+0000","last_fullsized":"2024-09-12T03:18:13.225810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221553+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:13.221553+0000","last_peered":"2024-09-12T03:18:13.221553+0000","last_clean":"2024-09-12T03:18:13.221553+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:13.221553+0000","last_undegraded":"2024-09-12T03:18:13.221553+0000","last_fullsized":"2024-09-12T03:18:13.221553+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221469+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:13.221469+0000","last_peered":"2024-09-12T03:18:13.221469+0000","last_clean":"2024-09-12T03:18:13.221469+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:13.221469+0000","last_undegraded":"2024-09-12T03:18:13.221469+0000","last_fullsized":"2024-09-12T03:18:13.221469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225010+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:13.225010+0000","last_peered":"2024-09-12T03:18:13.225010+0000","last_clean":"2024-09-12T03:18:13.225010+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:13.225010+0000","last_undegraded":"2024-09-12T03:18:13.225010+0000","last_fullsized":"2024-09-12T03:18:13.225010+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223333+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:13.223333+0000","last_peered":"2024-09-12T03:18:13.223333+0000","last_clean":"2024-09-12T03:18:13.223333+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:13.223333+0000","last_undegraded":"2024-09-12T03:18:13.223333+0000","last_fullsized":"2024-09-12T03:18:13.223333+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"80'6","reported_seq":95,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:21.901188+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:18:21.901188+0000","last_peered":"2024-09-12T03:18:21.901188+0000","last_clean":"2024-09-12T03:18:21.901188+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:18:21.901188+0000","last_undegraded":"2024-09-12T03:18:21.901188+0000","last_fullsized":"2024-09-12T03:18:21.901188+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226600+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:13.226600+0000","last_peered":"2024-09-12T03:18:13.226600+0000","last_clean":"2024-09-12T03:18:13.226600+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:13.226600+0000","last_undegraded":"2024-09-12T03:18:13.226600+0000","last_fullsized":"2024-09-12T03:18:13.226600+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227325+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:13.227325+0000","last_peered":"2024-09-12T03:18:13.227325+0000","last_clean":"2024-09-12T03:18:13.227325+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:13.227325+0000","last_undegraded":"2024-09-12T03:18:13.227325+0000","last_fullsized":"2024-09-12T03:18:13.227325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222614+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:13.222614+0000","last_peered":"2024-09-12T03:18:13.222614+0000","last_clean":"2024-09-12T03:18:13.222614+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:13.222614+0000","last_undegraded":"2024-09-12T03:18:13.222614+0000","last_fullsized":"2024-09-12T03:18:13.222614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'4","reported_seq":155,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:24.246840+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:18:24.246840+0000","last_peered":"2024-09-12T03:18:24.246840+0000","last_clean":"2024-09-12T03:18:24.246840+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:18:24.246840+0000","last_undegraded":"2024-09-12T03:18:24.246840+0000","last_fullsized":"2024-09-12T03:18:24.246840+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230681+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:13.230681+0000","last_peered":"2024-09-12T03:18:13.230681+0000","last_clean":"2024-09-12T03:18:13.230681+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:13.230681+0000","last_undegraded":"2024-09-12T03:18:13.230681+0000","last_fullsized":"2024-09-12T03:18:13.230681+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225820+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:13.225820+0000","last_peered":"2024-09-12T03:18:13.225820+0000","last_clean":"2024-09-12T03:18:13.225820+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:13.225820+0000","last_undegraded":"2024-09-12T03:18:13.225820+0000","last_fullsized":"2024-09-12T03:18:13.225820+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230416+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:13.230416+0000","last_peered":"2024-09-12T03:18:13.230416+0000","last_clean":"2024-09-12T03:18:13.230416+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:13.230416+0000","last_undegraded":"2024-09-12T03:18:13.230416+0000","last_fullsized":"2024-09-12T03:18:13.230416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"80'30","reported_seq":192,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:24.904700+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:18:24.904700+0000","last_peered":"2024-09-12T03:18:24.904700+0000","last_clean":"2024-09-12T03:18:24.904700+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:18:24.904700+0000","last_undegraded":"2024-09-12T03:18:24.904700+0000","last_fullsized":"2024-09-12T03:18:24.904700+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59764,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":140,"num_read_kb":128,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223911+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:13.223911+0000","last_peered":"2024-09-12T03:18:13.223911+0000","last_clean":"2024-09-12T03:18:13.223911+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:13.223911+0000","last_undegraded":"2024-09-12T03:18:13.223911+0000","last_fullsized":"2024-09-12T03:18:13.223911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220250+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:13.220250+0000","last_peered":"2024-09-12T03:18:13.220250+0000","last_clean":"2024-09-12T03:18:13.220250+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:13.220250+0000","last_undegraded":"2024-09-12T03:18:13.220250+0000","last_fullsized":"2024-09-12T03:18:13.220250+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223375+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:13.223375+0000","last_peered":"2024-09-12T03:18:13.223375+0000","last_clean":"2024-09-12T03:18:13.223375+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:13.223375+0000","last_undegraded":"2024-09-12T03:18:13.223375+0000","last_fullsized":"2024-09-12T03:18:13.223375+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'6","reported_seq":94,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:20.899744+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:18:20.899744+0000","last_peered":"2024-09-12T03:18:20.899744+0000","last_clean":"2024-09-12T03:18:20.899744+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:18:20.899744+0000","last_undegraded":"2024-09-12T03:18:20.899744+0000","last_fullsized":"2024-09-12T03:18:20.899744+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":67,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225722+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:13.225722+0000","last_peered":"2024-09-12T03:18:13.225722+0000","last_clean":"2024-09-12T03:18:13.225722+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:13.225722+0000","last_undegraded":"2024-09-12T03:18:13.225722+0000","last_fullsized":"2024-09-12T03:18:13.225722+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225695+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:13.225695+0000","last_peered":"2024-09-12T03:18:13.225695+0000","last_clean":"2024-09-12T03:18:13.225695+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:13.225695+0000","last_undegraded":"2024-09-12T03:18:13.225695+0000","last_fullsized":"2024-09-12T03:18:13.225695+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225060+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:13.225060+0000","last_peered":"2024-09-12T03:18:13.225060+0000","last_clean":"2024-09-12T03:18:13.225060+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:13.225060+0000","last_undegraded":"2024-09-12T03:18:13.225060+0000","last_fullsized":"2024-09-12T03:18:13.225060+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224091+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:13.224091+0000","last_peered":"2024-09-12T03:18:13.224091+0000","last_clean":"2024-09-12T03:18:13.224091+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:13.224091+0000","last_undegraded":"2024-09-12T03:18:13.224091+0000","last_fullsized":"2024-09-12T03:18:13.224091+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235706+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:13.235706+0000","last_peered":"2024-09-12T03:18:13.235706+0000","last_clean":"2024-09-12T03:18:13.235706+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:13.235706+0000","last_undegraded":"2024-09-12T03:18:13.235706+0000","last_fullsized":"2024-09-12T03:18:13.235706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:20.242710+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:18:20.242710+0000","last_peered":"2024-09-12T03:18:20.242710+0000","last_clean":"2024-09-12T03:18:20.242710+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:18:20.242710+0000","last_undegraded":"2024-09-12T03:18:20.242710+0000","last_fullsized":"2024-09-12T03:18:20.242710+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":10,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224420+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:13.224420+0000","last_peered":"2024-09-12T03:18:13.224420+0000","last_clean":"2024-09-12T03:18:13.224420+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:13.224420+0000","last_undegraded":"2024-09-12T03:18:13.224420+0000","last_fullsized":"2024-09-12T03:18:13.224420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":338,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223799+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223799+0000","last_peered":"2024-09-12T03:18:13.223799+0000","last_clean":"2024-09-12T03:18:13.223799+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223799+0000","last_undegraded":"2024-09-12T03:18:13.223799+0000","last_fullsized":"2024-09-12T03:18:13.223799+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223939+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.223939+0000","last_peered":"2024-09-12T03:18:13.223939+0000","last_clean":"2024-09-12T03:18:13.223939+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.223939+0000","last_undegraded":"2024-09-12T03:18:13.223939+0000","last_fullsized":"2024-09-12T03:18:13.223939+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227978+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:13.227978+0000","last_peered":"2024-09-12T03:18:13.227978+0000","last_clean":"2024-09-12T03:18:13.227978+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:13.227978+0000","last_undegraded":"2024-09-12T03:18:13.227978+0000","last_fullsized":"2024-09-12T03:18:13.227978+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223720+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223720+0000","last_peered":"2024-09-12T03:18:13.223720+0000","last_clean":"2024-09-12T03:18:13.223720+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223720+0000","last_undegraded":"2024-09-12T03:18:13.223720+0000","last_fullsized":"2024-09-12T03:18:13.223720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220163+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:13.220163+0000","last_peered":"2024-09-12T03:18:13.220163+0000","last_clean":"2024-09-12T03:18:13.220163+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:13.220163+0000","last_undegraded":"2024-09-12T03:18:13.220163+0000","last_fullsized":"2024-09-12T03:18:13.220163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"80'5","reported_seq":92,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:21.900471+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:18:21.900471+0000","last_peered":"2024-09-12T03:18:21.900471+0000","last_clean":"2024-09-12T03:18:21.900471+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:18:21.900471+0000","last_undegraded":"2024-09-12T03:18:21.900471+0000","last_fullsized":"2024-09-12T03:18:21.900471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225647+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:13.225647+0000","last_peered":"2024-09-12T03:18:13.225647+0000","last_clean":"2024-09-12T03:18:13.225647+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:13.225647+0000","last_undegraded":"2024-09-12T03:18:13.225647+0000","last_fullsized":"2024-09-12T03:18:13.225647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"80'5","reported_seq":89,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:20.923011+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:18:20.923011+0000","last_peered":"2024-09-12T03:18:20.923011+0000","last_clean":"2024-09-12T03:18:20.923011+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:18:20.923011+0000","last_undegraded":"2024-09-12T03:18:20.923011+0000","last_fullsized":"2024-09-12T03:18:20.923011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221810+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:13.221810+0000","last_peered":"2024-09-12T03:18:13.221810+0000","last_clean":"2024-09-12T03:18:13.221810+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:13.221810+0000","last_undegraded":"2024-09-12T03:18:13.221810+0000","last_fullsized":"2024-09-12T03:18:13.221810+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223007+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223007+0000","last_peered":"2024-09-12T03:18:13.223007+0000","last_clean":"2024-09-12T03:18:13.223007+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223007+0000","last_undegraded":"2024-09-12T03:18:13.223007+0000","last_fullsized":"2024-09-12T03:18:13.223007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222321+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:13.222321+0000","last_peered":"2024-09-12T03:18:13.222321+0000","last_clean":"2024-09-12T03:18:13.222321+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:13.222321+0000","last_undegraded":"2024-09-12T03:18:13.222321+0000","last_fullsized":"2024-09-12T03:18:13.222321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224119+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224119+0000","last_peered":"2024-09-12T03:18:13.224119+0000","last_clean":"2024-09-12T03:18:13.224119+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224119+0000","last_undegraded":"2024-09-12T03:18:13.224119+0000","last_fullsized":"2024-09-12T03:18:13.224119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220558+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:13.220558+0000","last_peered":"2024-09-12T03:18:13.220558+0000","last_clean":"2024-09-12T03:18:13.220558+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:13.220558+0000","last_undegraded":"2024-09-12T03:18:13.220558+0000","last_fullsized":"2024-09-12T03:18:13.220558+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"80'15","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222863+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:13.222863+0000","last_peered":"2024-09-12T03:18:13.222863+0000","last_clean":"2024-09-12T03:18:13.222863+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:13.222863+0000","last_undegraded":"2024-09-12T03:18:13.222863+0000","last_fullsized":"2024-09-12T03:18:13.222863+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224342+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224342+0000","last_peered":"2024-09-12T03:18:13.224342+0000","last_clean":"2024-09-12T03:18:13.224342+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224342+0000","last_undegraded":"2024-09-12T03:18:13.224342+0000","last_fullsized":"2024-09-12T03:18:13.224342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.226157+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:13.226157+0000","last_peered":"2024-09-12T03:18:13.226157+0000","last_clean":"2024-09-12T03:18:13.226157+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:13.226157+0000","last_undegraded":"2024-09-12T03:18:13.226157+0000","last_fullsized":"2024-09-12T03:18:13.226157+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230600+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230600+0000","last_peered":"2024-09-12T03:18:13.230600+0000","last_clean":"2024-09-12T03:18:13.230600+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230600+0000","last_undegraded":"2024-09-12T03:18:13.230600+0000","last_fullsized":"2024-09-12T03:18:13.230600+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223193+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:13.223193+0000","last_peered":"2024-09-12T03:18:13.223193+0000","last_clean":"2024-09-12T03:18:13.223193+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:13.223193+0000","last_undegraded":"2024-09-12T03:18:13.223193+0000","last_fullsized":"2024-09-12T03:18:13.223193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225267+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:13.225267+0000","last_peered":"2024-09-12T03:18:13.225267+0000","last_clean":"2024-09-12T03:18:13.225267+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:13.225267+0000","last_undegraded":"2024-09-12T03:18:13.225267+0000","last_fullsized":"2024-09-12T03:18:13.225267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228053+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:13.228053+0000","last_peered":"2024-09-12T03:18:13.228053+0000","last_clean":"2024-09-12T03:18:13.228053+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:13.228053+0000","last_undegraded":"2024-09-12T03:18:13.228053+0000","last_fullsized":"2024-09-12T03:18:13.228053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223554+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:13.223554+0000","last_peered":"2024-09-12T03:18:13.223554+0000","last_clean":"2024-09-12T03:18:13.223554+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:13.223554+0000","last_undegraded":"2024-09-12T03:18:13.223554+0000","last_fullsized":"2024-09-12T03:18:13.223554+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.231726+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:13.231726+0000","last_peered":"2024-09-12T03:18:13.231726+0000","last_clean":"2024-09-12T03:18:13.231726+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:13.231726+0000","last_undegraded":"2024-09-12T03:18:13.231726+0000","last_fullsized":"2024-09-12T03:18:13.231726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.229860+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:13.229860+0000","last_peered":"2024-09-12T03:18:13.229860+0000","last_clean":"2024-09-12T03:18:13.229860+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:13.229860+0000","last_undegraded":"2024-09-12T03:18:13.229860+0000","last_fullsized":"2024-09-12T03:18:13.229860+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225732+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:13.225732+0000","last_peered":"2024-09-12T03:18:13.225732+0000","last_clean":"2024-09-12T03:18:13.225732+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:13.225732+0000","last_undegraded":"2024-09-12T03:18:13.225732+0000","last_fullsized":"2024-09-12T03:18:13.225732+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228427+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228427+0000","last_peered":"2024-09-12T03:18:13.228427+0000","last_clean":"2024-09-12T03:18:13.228427+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228427+0000","last_undegraded":"2024-09-12T03:18:13.228427+0000","last_fullsized":"2024-09-12T03:18:13.228427+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228289+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:13.228289+0000","last_peered":"2024-09-12T03:18:13.228289+0000","last_clean":"2024-09-12T03:18:13.228289+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:13.228289+0000","last_undegraded":"2024-09-12T03:18:13.228289+0000","last_fullsized":"2024-09-12T03:18:13.228289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"80'13","reported_seq":100,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:23.925957+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:18:23.925957+0000","last_peered":"2024-09-12T03:18:23.925957+0000","last_clean":"2024-09-12T03:18:23.925957+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:18:23.925957+0000","last_undegraded":"2024-09-12T03:18:23.925957+0000","last_fullsized":"2024-09-12T03:18:23.925957+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":65,"num_read_kb":65,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66972,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":623,"num_read_kb":414,"num_write":243,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":200916,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1052672,"data_stored":1089115,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972391,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462961,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":974848,"data_stored":1031150,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626487,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":909312,"data_stored":437576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855420,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":987136,"data_stored":1030435,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247877,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444107,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":497696,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":966656,"data_stored":498253,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":61516,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":60959,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":61893,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:18:27.847 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:18:27.848 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T03:18:27.848 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T03:18:27.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T03:18:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:27 smithi135 ceph-mon[35474]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:28.114 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:28.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:27 smithi121 ceph-mon[34963]: from='client.14931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 -- 172.21.15.19:0/1347296970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040106f10 msgr2=0x7fd04010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 --2- 172.21.15.19:0/1347296970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040106f10 0x7fd04010ea90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd038009ca0 tx=0x7fd03802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 -- 172.21.15.19:0/1347296970 shutdown_connections 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 --2- 172.21.15.19:0/1347296970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040106f10 0x7fd04010ea90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 --2- 172.21.15.19:0/1347296970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040102790 0x7fd0401067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 --2- 172.21.15.19:0/1347296970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040101de0 0x7fd0401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 -- 172.21.15.19:0/1347296970 >> 172.21.15.19:0/1347296970 conn(0x7fd0400fd8c0 msgr2=0x7fd0400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 -- 172.21.15.19:0/1347296970 shutdown_connections 2024-09-12T03:18:28.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.582+0000 7fd047106640 1 -- 172.21.15.19:0/1347296970 wait complete. 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 Processor -- start 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 -- start start 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 0x7fd0401985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 0x7fd04019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd040112010 con 0x7fd040102790 2024-09-12T03:18:28.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd040111e90 con 0x7fd040101de0 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd047106640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd040112190 con 0x7fd040106f10 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59492/0 (socket says 172.21.15.19:59492) 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 -- 172.21.15.19:0/2183668857 learned_addr learned my addr 172.21.15.19:0/2183668857 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd0456a3640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 0x7fd04019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd044ea2640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 0x7fd0401985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 -- 172.21.15.19:0/2183668857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 msgr2=0x7fd04019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 0x7fd04019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 -- 172.21.15.19:0/2183668857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 msgr2=0x7fd0401985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 0x7fd0401985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 -- 172.21.15.19:0/2183668857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04019d5a0 con 0x7fd040102790 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd0456a3640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 0x7fd04019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd044ea2640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 0x7fd0401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.583+0000 7fd03ffff640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00b4a0 tx=0x7fd02c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.584+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c014070 con 0x7fd040102790 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.584+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd02c010af0 con 0x7fd040102790 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.584+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04019d890 con 0x7fd040102790 2024-09-12T03:18:28.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.584+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c017d10 con 0x7fd040102790 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.584+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0401a5170 con 0x7fd040102790 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.585+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd040102240 con 0x7fd040102790 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.586+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd02c017070 con 0x7fd040102790 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.587+0000 7fd03dffb640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 0x7fd0300a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.587+0000 7fd044ea2640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 0x7fd0300a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.587+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd02c0ce4b0 con 0x7fd040102790 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.587+0000 7fd044ea2640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 0x7fd0300a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd034006fd0 tx=0x7fd034008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:28.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.591+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd02c09b370 con 0x7fd040102790 2024-09-12T03:18:28.805 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:28 smithi019 ceph-mon[30953]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.804+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd040109b60 con 0x7fd040102790 2024-09-12T03:18:28.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.805+0000 7fd03dffb640 1 -- 172.21.15.19:0/2183668857 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd02c09ac70 con 0x7fd040102790 2024-09-12T03:18:28.806 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:28.806 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T03:18:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 msgr2=0x7fd0300a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 0x7fd0300a5150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd034006fd0 tx=0x7fd034008040 comp rx=0 tx=0).stop 2024-09-12T03:18:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 msgr2=0x7fd040198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00b4a0 tx=0x7fd02c00b960 comp rx=0 tx=0).stop 2024-09-12T03:18:28.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 shutdown_connections 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd040106f10 0x7fd04019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0300a2c90 0x7fd0300a5150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd040102790 0x7fd040198b30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 --2- 172.21.15.19:0/2183668857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd040101de0 0x7fd0401985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 >> 172.21.15.19:0/2183668857 conn(0x7fd0400fd8c0 msgr2=0x7fd0400fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 shutdown_connections 2024-09-12T03:18:28.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:28.807+0000 7fd047106640 1 -- 172.21.15.19:0/2183668857 wait complete. 2024-09-12T03:18:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:28 smithi135 ceph-mon[35474]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:29.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:28 smithi121 ceph-mon[34963]: pgmap v320: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 5 op/s; 157 B/s, 0 objects/s recovering 2024-09-12T03:18:29.526 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T03:18:29.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:29.787 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:29.908 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:29 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2183668857' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:18:30.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:29 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2183668857' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:18:30.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:29 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2183668857' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T03:18:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.267+0000 7fd55feb3640 1 -- 172.21.15.19:0/3464807366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 msgr2=0x7fd5581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:30.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.267+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3464807366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd5581067e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd548009900 tx=0x7fd54802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 -- 172.21.15.19:0/3464807366 shutdown_connections 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3464807366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558106f10 0x7fd55810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3464807366 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd5581067e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3464807366 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558101de0 0x7fd5581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 -- 172.21.15.19:0/3464807366 >> 172.21.15.19:0/3464807366 conn(0x7fd5580fd8c0 msgr2=0x7fd5580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 -- 172.21.15.19:0/3464807366 shutdown_connections 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 -- 172.21.15.19:0/3464807366 wait complete. 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 Processor -- start 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 -- start start 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558101de0 0x7fd5581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.268+0000 7fd55feb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:30.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55feb3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 0x7fd55819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55feb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111f70 con 0x7fd558102790 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55feb3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd558111df0 con 0x7fd558101de0 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55feb3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd5581120f0 con 0x7fd558106f10 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59516/0 (socket says 172.21.15.19:59516) 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 -- 172.21.15.19:0/3907956457 learned_addr learned my addr 172.21.15.19:0/3907956457 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55dc4f640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558101de0 0x7fd5581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55e450640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 0x7fd55819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 -- 172.21.15.19:0/3907956457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 msgr2=0x7fd55819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 0x7fd55819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 -- 172.21.15.19:0/3907956457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558101de0 msgr2=0x7fd5581983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558101de0 0x7fd5581983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 -- 172.21.15.19:0/3907956457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd55819d380 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55e450640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 0x7fd55819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55d44e640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd548006010 tx=0x7fd5480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54803f030 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd548003ce0 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd548035070 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.269+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd55819d610 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.270+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5581a4f50 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.270+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5581021c0 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.272+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd548042030 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.272+0000 7fd54effd640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 0x7fd5440a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.272+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd5480ef310 con 0x7fd558102790 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.272+0000 7fd55dc4f640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 0x7fd5440a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:30.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.273+0000 7fd55dc4f640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 0x7fd5440a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd554005fd0 tx=0x7fd55400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:30.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.276+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5480bc1e0 con 0x7fd558102790 2024-09-12T03:18:30.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.489+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd55810a180 con 0x7fd558102790 2024-09-12T03:18:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.490+0000 7fd54effd640 1 -- 172.21.15.19:0/3907956457 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7fd5480bbae0 con 0x7fd558102790 2024-09-12T03:18:30.490 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:30.490 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:18:30.490 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 msgr2=0x7fd5440a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 0x7fd5440a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd554005fd0 tx=0x7fd55400b390 comp rx=0 tx=0).stop 2024-09-12T03:18:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 msgr2=0x7fd558198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd548006010 tx=0x7fd5480042f0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 shutdown_connections 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd558106f10 0x7fd55819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd5440a2c90 0x7fd5440a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd558102790 0x7fd558198910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 --2- 172.21.15.19:0/3907956457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd558101de0 0x7fd5581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 >> 172.21.15.19:0/3907956457 conn(0x7fd5580fd8c0 msgr2=0x7fd5580fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 shutdown_connections 2024-09-12T03:18:30.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:30.492+0000 7fd55feb3640 1 -- 172.21.15.19:0/3907956457 wait complete. 2024-09-12T03:18:30.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:30 smithi019 ceph-mon[30953]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:30.873 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:30 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3907956457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:30 smithi135 ceph-mon[35474]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:30 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3907956457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:31.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:30 smithi121 ceph-mon[34963]: pgmap v321: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:30 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3907956457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:31.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:31.466 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:31.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 -- 172.21.15.19:0/2788997232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14100a80 msgr2=0x7faa14100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 --2- 172.21.15.19:0/2788997232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14100a80 0x7faa14100ee0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faa04009930 tx=0x7faa0402a420 comp rx=0 tx=0).stop 2024-09-12T03:18:32.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 -- 172.21.15.19:0/2788997232 shutdown_connections 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 --2- 172.21.15.19:0/2788997232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 0x7faa1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 --2- 172.21.15.19:0/2788997232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14100a80 0x7faa14100ee0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 --2- 172.21.15.19:0/2788997232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14106a80 0x7faa14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 -- 172.21.15.19:0/2788997232 >> 172.21.15.19:0/2788997232 conn(0x7faa140fc7a0 msgr2=0x7faa140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 -- 172.21.15.19:0/2788997232 shutdown_connections 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.944+0000 7faa1c77e640 1 -- 172.21.15.19:0/2788997232 wait complete. 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 Processor -- start 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 -- start start 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14100a80 0x7faa141984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 0x7faa141989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa14110e80 con 0x7faa14106a80 2024-09-12T03:18:32.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa14110d00 con 0x7faa14100a80 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1c77e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa14111000 con 0x7faa14101420 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1ad1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1ad1b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38348/0 (socket says 172.21.15.19:38348) 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1ad1b640 1 -- 172.21.15.19:0/1090795701 learned_addr learned my addr 172.21.15.19:0/1090795701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa1a51a640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14100a80 0x7faa141984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.945+0000 7faa19d19640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 0x7faa141989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 -- 172.21.15.19:0/1090795701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 msgr2=0x7faa141989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 0x7faa141989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 -- 172.21.15.19:0/1090795701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14100a80 msgr2=0x7faa141984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14100a80 0x7faa141984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 -- 172.21.15.19:0/1090795701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1419d450 con 0x7faa14106a80 2024-09-12T03:18:32.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1ad1b640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cad0 tx=0x7faa0c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00ae20 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1419d740 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa0c005020 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0c00f6f0 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.946+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa141a5020 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.947+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa14100ee0 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.948+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faa0c0051d0 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.949+0000 7faa0b7fe640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 0x7faa000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.949+0000 7faa1a51a640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 0x7faa000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.949+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa0c0cf580 con 0x7faa14106a80 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.949+0000 7faa1a51a640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 0x7faa000a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa1000bb60 tx=0x7faa1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:32.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:31.952+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa0c014030 con 0x7faa14106a80 2024-09-12T03:18:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.168+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa14199800 con 0x7faa14106a80 2024-09-12T03:18:32.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.168+0000 7faa0b7fe640 1 -- 172.21.15.19:0/1090795701 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7faa14199800 con 0x7faa14106a80 2024-09-12T03:18:32.202 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:32.202 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 msgr2=0x7faa000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 0x7faa000a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7faa1000bb60 tx=0x7faa1000b390 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 msgr2=0x7faa1419cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa0c00cad0 tx=0x7faa0c00cf90 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 shutdown_connections 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa000a2c10 0x7faa000a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa14106a80 0x7faa1419cd70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa14101420 0x7faa141989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 --2- 172.21.15.19:0/1090795701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa14100a80 0x7faa141984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 >> 172.21.15.19:0/1090795701 conn(0x7faa140fc7a0 msgr2=0x7faa1406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:32.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 shutdown_connections 2024-09-12T03:18:32.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:32.171+0000 7faa1c77e640 1 -- 172.21.15.19:0/1090795701 wait complete. 2024-09-12T03:18:32.889 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:32 smithi019 ceph-mon[30953]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:32.890 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:32 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1090795701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:32.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:18:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:32 smithi135 ceph-mon[35474]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:32 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1090795701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:33.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:32 smithi121 ceph-mon[34963]: pgmap v322: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:32 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1090795701' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:33.150 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 -- 172.21.15.19:0/2985582674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc101420 msgr2=0x7f85dc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 --2- 172.21.15.19:0/2985582674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc101420 0x7f85dc10d950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f85d4009cd0 tx=0x7f85d402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 -- 172.21.15.19:0/2985582674 shutdown_connections 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 --2- 172.21.15.19:0/2985582674 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc101420 0x7f85dc10d950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 --2- 172.21.15.19:0/2985582674 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc100a80 0x7f85dc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 --2- 172.21.15.19:0/2985582674 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 -- 172.21.15.19:0/2985582674 >> 172.21.15.19:0/2985582674 conn(0x7f85dc0fc7a0 msgr2=0x7f85dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.628+0000 7f85e34cd640 1 -- 172.21.15.19:0/2985582674 shutdown_connections 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 -- 172.21.15.19:0/2985582674 wait complete. 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 Processor -- start 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 -- start start 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc101420 0x7f85dc102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc1030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc110ed0 con 0x7f85dc100a80 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc110d50 con 0x7f85dc101420 2024-09-12T03:18:33.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e34cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc111050 con 0x7f85dc106a80 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.629+0000 7f85e1269640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38372/0 (socket says 172.21.15.19:38372) 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 -- 172.21.15.19:0/559236933 learned_addr learned my addr 172.21.15.19:0/559236933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1a6a640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc1030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e0a68640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc101420 0x7f85dc102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 -- 172.21.15.19:0/559236933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 msgr2=0x7f85dc1030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc1030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 -- 172.21.15.19:0/559236933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc101420 msgr2=0x7f85dc102b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc101420 0x7f85dc102b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 -- 172.21.15.19:0/559236933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1a68a0 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1a6a640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e1269640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c970 tx=0x7f85d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d8007bb0 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc1a6b90 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85d8007d50 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d800f450 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.630+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a7080 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.631+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85dc100ee0 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.632+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f85d8004fe0 con 0x7f85dc100a80 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.633+0000 7f85d27fc640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 0x7f85c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:33.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.633+0000 7f85e0a68640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 0x7f85c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:33.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.633+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f85d80cf380 con 0x7f85dc100a80 2024-09-12T03:18:33.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.634+0000 7f85e0a68640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 0x7f85c80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005fd0 tx=0x7f85cc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:33.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.636+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85d8014030 con 0x7f85dc100a80 2024-09-12T03:18:33.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.815+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85dc103b80 con 0x7f85dc100a80 2024-09-12T03:18:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.816+0000 7f85d27fc640 1 -- 172.21.15.19:0/559236933 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7f85dc103b80 con 0x7f85dc100a80 2024-09-12T03:18:33.816 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:33.817 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 msgr2=0x7f85c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 0x7f85c80a50d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005fd0 tx=0x7f85cc005850 comp rx=0 tx=0).stop 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 msgr2=0x7f85dc102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f85d800c970 tx=0x7f85d800ce30 comp rx=0 tx=0).stop 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 shutdown_connections 2024-09-12T03:18:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85dc106a80 0x7f85dc1030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85dc101420 0x7f85dc102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85c80a2c10 0x7f85c80a50d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 --2- 172.21.15.19:0/559236933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85dc100a80 0x7f85dc102650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 >> 172.21.15.19:0/559236933 conn(0x7f85dc0fc7a0 msgr2=0x7f85dc06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 shutdown_connections 2024-09-12T03:18:33.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:33.819+0000 7f85e34cd640 1 -- 172.21.15.19:0/559236933 wait complete. 2024-09-12T03:18:34.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:34.803 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:34.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:34 smithi019 ceph-mon[30953]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:18:34.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:34 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/559236933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:35.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:34 smithi135 ceph-mon[35474]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:18:35.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:34 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/559236933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:35.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:34 smithi121 ceph-mon[34963]: pgmap v323: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:18:35.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:34 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/559236933' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 -- 172.21.15.19:0/3763734517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8101420 msgr2=0x7fa1a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 --2- 172.21.15.19:0/3763734517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8101420 0x7fa1a810d950 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009cd0 tx=0x7fa1a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 -- 172.21.15.19:0/3763734517 shutdown_connections 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 --2- 172.21.15.19:0/3763734517 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8101420 0x7fa1a810d950 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 --2- 172.21.15.19:0/3763734517 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 0x7fa1a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 --2- 172.21.15.19:0/3763734517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8106a80 0x7fa1a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 -- 172.21.15.19:0/3763734517 >> 172.21.15.19:0/3763734517 conn(0x7fa1a80fc7a0 msgr2=0x7fa1a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:35.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 -- 172.21.15.19:0/3763734517 shutdown_connections 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.280+0000 7fa1adf90640 1 -- 172.21.15.19:0/3763734517 wait complete. 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 Processor -- start 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 -- start start 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 0x7fa1a819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8101420 0x7fa1a819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8110e80 con 0x7fa1a8106a80 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8110d00 con 0x7fa1a8101420 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1adf90640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8111000 con 0x7fa1a8100a80 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38400/0 (socket says 172.21.15.19:38400) 2024-09-12T03:18:35.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.281+0000 7fa1a7fff640 1 -- 172.21.15.19:0/755767092 learned_addr learned my addr 172.21.15.19:0/755767092 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a6ffd640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8101420 0x7fa1a819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 0x7fa1a819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 -- 172.21.15.19:0/755767092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 msgr2=0x7fa1a819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 0x7fa1a819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 -- 172.21.15.19:0/755767092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8101420 msgr2=0x7fa1a819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8101420 0x7fa1a819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 -- 172.21.15.19:0/755767092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819f560 con 0x7fa1a8106a80 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a7fff640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005cc0 tx=0x7fa1a0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004040 con 0x7fa1a8106a80 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819f7f0 con 0x7fa1a8106a80 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1a00046c0 con 0x7fa1a8106a80 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0030e70 con 0x7fa1a8106a80 2024-09-12T03:18:35.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.282+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a7130 con 0x7fa1a8106a80 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.283+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a8100ee0 con 0x7fa1a8106a80 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.285+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa1a00305a0 con 0x7fa1a8106a80 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.285+0000 7fa1a4ff9640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 0x7fa1900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.285+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 0x7fa1900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.285+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1a002e030 con 0x7fa1a8106a80 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.286+0000 7fa1a77fe640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 0x7fa1900a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa19800ab60 tx=0x7fa19800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:35.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.288+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa1a00bd630 con 0x7fa1a8106a80 2024-09-12T03:18:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.501+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a819b910 con 0x7fa1a8106a80 2024-09-12T03:18:35.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.502+0000 7fa1a4ff9640 1 -- 172.21.15.19:0/755767092 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7fa1a819b910 con 0x7fa1a8106a80 2024-09-12T03:18:35.502 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:35.502 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:18:35.503 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 msgr2=0x7fa1900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:35.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 0x7fa1900a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa19800ab60 tx=0x7fa19800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 msgr2=0x7fa1a819ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0005cc0 tx=0x7fa1a0003970 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 shutdown_connections 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1900a2c10 0x7fa1900a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1a8106a80 0x7fa1a819ee80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1a8101420 0x7fa1a819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 --2- 172.21.15.19:0/755767092 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa1a8100a80 0x7fa1a819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 >> 172.21.15.19:0/755767092 conn(0x7fa1a80fc7a0 msgr2=0x7fa1a810c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 shutdown_connections 2024-09-12T03:18:35.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:35.504+0000 7fa1adf90640 1 -- 172.21.15.19:0/755767092 wait complete. 2024-09-12T03:18:35.885 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:35 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/755767092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:36.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:35 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/755767092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:36.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:35 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/755767092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:36.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:17:10.527386+0000', 'modified': '2024-09-12T03:17:21.825856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24467, 'mds_2': 24463, 'mds_3': 24421, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14577': {'gid': 14577, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/2689902455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 2689902455}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 2689902455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14589': {'gid': 14589, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/9500015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 9500015}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 9500015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.121:6833/3300883065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 3300883065}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 3300883065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.135:6833/990973415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6832', 'nonce': 990973415}, {'type': 'v1', 'addr': '172.21.15.135:6833', 'nonce': 990973415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/3114415604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3114415604}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 3114415604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/826498871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 826498871}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 826498871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3494935988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3494935988}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3494935988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.135:6837/1827195054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 1827195054}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 1827195054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6839/1305484715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 1305484715}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 1305484715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.135:6839/1449185077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 1449185077}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 1449185077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24487, 'qdb_cluster': [24487, 24467, 24463, 24421, 24407]} 2024-09-12T03:18:36.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T03:18:36.257 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-12T03:18:36.520 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:36.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:36 smithi019 ceph-mon[30953]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:36 smithi135 ceph-mon[35474]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 -- 172.21.15.19:0/3092906577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4028102d50 msgr2=0x7f4028103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 --2- 172.21.15.19:0/3092906577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4028102d50 0x7f4028103130 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4018009900 tx=0x7f401802a480 comp rx=0 tx=0).stop 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 -- 172.21.15.19:0/3092906577 shutdown_connections 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 --2- 172.21.15.19:0/3092906577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f402810c180 0x7f402810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 --2- 172.21.15.19:0/3092906577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f402810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 --2- 172.21.15.19:0/3092906577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4028102d50 0x7f4028103130 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 -- 172.21.15.19:0/3092906577 >> 172.21.15.19:0/3092906577 conn(0x7f40280fc7d0 msgr2=0x7f40280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 -- 172.21.15.19:0/3092906577 shutdown_connections 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 -- 172.21.15.19:0/3092906577 wait complete. 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.039+0000 7f402d2b7640 1 Processor -- start 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 -- start start 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4028102d50 0x7f4028198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f4028198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4028111f80 con 0x7f402810c180 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4028111e00 con 0x7f4028102d50 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f402d2b7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4028112100 con 0x7f4028103670 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38414/0 (socket says 172.21.15.19:38414) 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 -- 172.21.15.19:0/262018133 learned_addr learned my addr 172.21.15.19:0/262018133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f4026ffd640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4028102d50 0x7f4028198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40267fc640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f4028198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 -- 172.21.15.19:0/262018133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 msgr2=0x7f4028198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f4028198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 -- 172.21.15.19:0/262018133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4028102d50 msgr2=0x7f4028198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4028102d50 0x7f4028198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40277fe640 1 -- 172.21.15.19:0/262018133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f402819d3e0 con 0x7f402810c180 2024-09-12T03:18:37.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.040+0000 7f40267fc640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f4028198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f40277fe640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f402000eaa0 tx=0x7f402000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f401ffff640 1 -- 172.21.15.19:0/262018133 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402000cdb0 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f402819d6d0 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f401ffff640 1 -- 172.21.15.19:0/262018133 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4020005020 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f401ffff640 1 -- 172.21.15.19:0/262018133 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40200116b0 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.041+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40281a4fb0 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.043+0000 7f401ffff640 1 -- 172.21.15.19:0/262018133 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4020005560 con 0x7f402810c180 2024-09-12T03:18:37.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.043+0000 7f402f690740 1 Processor -- start 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.043+0000 7f402f690740 1 -- start start 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4e890 0x556975e4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 0x556975e603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556975db5090 con 0x556975e5dfc0 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556975d5ca70 con 0x556975e4f1b0 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f402f690740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556975db1050 con 0x556975e4e890 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 0x556975e603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 0x556975e603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38416/0 (socket says 172.21.15.19:38416) 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f40267fc640 1 -- 172.21.15.19:0/258648456 learned_addr learned my addr 172.21.15.19:0/258648456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f40267fc640 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4e890 0x556975e4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f401ffff640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 0x7f40140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 -- 172.21.15.19:0/258648456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4e890 msgr2=0x556975e4ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4e890 0x556975e4ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 -- 172.21.15.19:0/258648456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 msgr2=0x556975e603b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 0x556975e603b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 -- 172.21.15.19:0/258648456 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556975d69130 con 0x556975e4f1b0 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f40277fe640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 0x7f40140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f401ffff640 1 -- 172.21.15.19:0/262018133 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40200cf3b0 con 0x7f402810c180 2024-09-12T03:18:37.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.044+0000 7f4026ffd640 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5da80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4018005940 tx=0x7f4018005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0ef805f13ef7153 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f40277fe640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 0x7f40140a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40200092f0 tx=0x7f402006dad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f401d7fa640 1 -- 172.21.15.19:0/258648456 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018040070 con 0x556975e4f1b0 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f401d7fa640 1 -- 172.21.15.19:0/258648456 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40180319f0 con 0x556975e4f1b0 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f401d7fa640 1 -- 172.21.15.19:0/258648456 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018035040 con 0x556975e4f1b0 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f402f690740 1 -- 172.21.15.19:0/258648456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 msgr2=0x556975e5da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.045+0000 7f402f690740 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5da80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4018005940 tx=0x7f4018005e80 comp rx=0 tx=0).stop 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- 172.21.15.19:0/258648456 shutdown_connections 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e5dfc0 0x556975e603b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5da80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- 172.21.15.19:0/258648456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4e890 0x556975e4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- 172.21.15.19:0/258648456 >> 172.21.15.19:0/258648456 conn(0x556975e4af60 msgr2=0x556975e4b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- 172.21.15.19:0/258648456 shutdown_connections 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- 172.21.15.19:0/258648456 wait complete. 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- start start 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 Processor -- start 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975e5b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556975e60b30 con 0x556975e4e890 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556975d6ff20 con 0x556975e4f1b0 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556975e60cb0 con 0x556975e5dfc0 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f402f690740 10 client.0 _ll_register_callbacks cb 0x556975d39030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38420/0 (socket says 172.21.15.19:38420) 2024-09-12T03:18:37.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f40267fc640 1 -- 172.21.15.19:0/3651166345 learned_addr learned my addr 172.21.15.19:0/3651166345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.046+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975e5b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 -- 172.21.15.19:0/3651166345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 msgr2=0x556975e5b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975e5b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 -- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 msgr2=0x556975e5abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 -- 172.21.15.19:0/3651166345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556975b262b0 con 0x556975e4e890 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:37.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f40267fc640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f401000bee0 tx=0x7f4010016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbd4b91b665576d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010012ce0 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4010005020 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40180319f0 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4010005220 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f402f690740 10 client.14973 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556975e6c4f0 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40180319f0 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.047+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f401000f440 con 0x556975e4e890 2024-09-12T03:18:37.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 10 client.14973 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 10 client.14973 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 20 client.14973 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975e6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 4 client.14973 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556975dc2600 con 0x556975e6ead0 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975e6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f400e7fc640 10 client.14973 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.048+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975e6ff40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6b7a24e6123212f9 server_cookie=c59fa2ded0804b04 in_seq=0 out_seq=0 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.049+0000 7f400e7fc640 1 -- 172.21.15.19:0/3651166345 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556975dc2600 con 0x556975e6ead0 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.049+0000 7f400e7fc640 10 client.14973 handle_command_reply: tid=0 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.049+0000 7f402f690740 1 client.14973 shutdown 2024-09-12T03:18:37.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.049+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 msgr2=0x556975e5d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.049+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f401000bee0 tx=0x7f4010016040 comp rx=0 tx=0).stop 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 shutdown_connections 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975e6ff40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975e5b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f400e7fc640 0 client.14973 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4f1b0 0x556975e5abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 --2- 172.21.15.19:0/3651166345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e5d820 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 >> 172.21.15.19:0/3651166345 conn(0x556975e4af60 msgr2=0x556975e50cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 shutdown_connections 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 1 -- 172.21.15.19:0/3651166345 wait complete. 2024-09-12T03:18:37.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.050+0000 7f402f690740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T03:18:37.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 Processor -- start 2024-09-12T03:18:37.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 -- start start 2024-09-12T03:18:37.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4f1b0 0x556975e57e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e5dfc0 0x556975e58360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556975e6e280 con 0x556975e4e890 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556975e6e950 con 0x556975e5dfc0 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f402f690740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556975e6f020 con 0x556975e4f1b0 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38432/0 (socket says 172.21.15.19:38432) 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 -- 172.21.15.19:0/980421363 learned_addr learned my addr 172.21.15.19:0/980421363 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f4026ffd640 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4f1b0 0x556975e57e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 -- 172.21.15.19:0/980421363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4f1b0 msgr2=0x556975e57e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4f1b0 0x556975e57e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 -- 172.21.15.19:0/980421363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e5dfc0 msgr2=0x556975e58360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e5dfc0 0x556975e58360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.068+0000 7f40267fc640 1 -- 172.21.15.19:0/980421363 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556975e4ad90 con 0x556975e4e890 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f40267fc640 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4010016010 tx=0x7f4010014660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d42a5546fdf5dc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f401cff9640 1 -- 172.21.15.19:0/980421363 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010030020 con 0x556975e4e890 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f401cff9640 1 -- 172.21.15.19:0/980421363 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401000b8a0 con 0x556975e4e890 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f401cff9640 1 -- 172.21.15.19:0/980421363 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010004a00 con 0x556975e4e890 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f402f690740 1 -- 172.21.15.19:0/980421363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 msgr2=0x556975e6ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.069+0000 7f402f690740 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4010016010 tx=0x7f4010014660 comp rx=0 tx=0).stop 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- 172.21.15.19:0/980421363 shutdown_connections 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e5dfc0 0x556975e58360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e4f1b0 0x556975e57e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- 172.21.15.19:0/980421363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4e890 0x556975e6ff40 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- 172.21.15.19:0/980421363 >> 172.21.15.19:0/980421363 conn(0x556975e4af60 msgr2=0x556975e59490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- 172.21.15.19:0/980421363 shutdown_connections 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- 172.21.15.19:0/980421363 wait complete. 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- start start 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 Processor -- start 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 0x556975e712d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975ed71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556975e6fdc0 con 0x556975e4f1b0 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556975e6f6f0 con 0x556975e4e890 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556975e70490 con 0x556975e5dfc0 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f402f690740 10 client.0 _ll_register_callbacks cb 0x7f401000bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f4026ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38446/0 (socket says 172.21.15.19:38446) 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.070+0000 7f4026ffd640 1 -- 172.21.15.19:0/3450237889 learned_addr learned my addr 172.21.15.19:0/3450237889 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f40267fc640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 0x556975e712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f40267fc640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975ed71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 -- 172.21.15.19:0/3450237889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 msgr2=0x556975ed71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975ed71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 -- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 msgr2=0x556975e712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 0x556975e712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 -- 172.21.15.19:0/3450237889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556975e60ca0 con 0x556975e4f1b0 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f40267fc640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 0x556975e712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f4026ffd640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4018009850 tx=0x7f4018005890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd64c1dc8e8ceb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f400e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:37.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4018054060 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4018004950 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401000b8a0 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4018040040 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f402f690740 10 client.14979 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.071+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556975ed8660 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401000b8a0 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4018003b20 con 0x556975e4f1b0 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 10 client.14979 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 10 client.14979 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 20 client.14979 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975edbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 4 client.14979 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:18:37.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "status"}) -- 0x556975dbcb10 con 0x556975e6ead0 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f40267fc640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975edbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f400e7fc640 10 client.14979 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.072+0000 7f40267fc640 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975edbb50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8c0b582e49a41c0 server_cookie=dbb5bc1501ca8eed in_seq=0 out_seq=0 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f400e7fc640 1 -- 172.21.15.19:0/3450237889 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x556975dbcb10 con 0x556975e6ead0 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f400e7fc640 10 client.14979 handle_command_reply: tid=0 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 client.14979 shutdown 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 msgr2=0x556975e71810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4018009850 tx=0x7f4018005890 comp rx=0 tx=0).stop 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 shutdown_connections 2024-09-12T03:18:37.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556975e6ead0 0x556975edbb50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556975e5dfc0 0x556975ed71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f400e7fc640 0 client.14979 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556975e4f1b0 0x556975e71810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 --2- 172.21.15.19:0/3450237889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556975e4e890 0x556975e712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 >> 172.21.15.19:0/3450237889 conn(0x556975e4af60 msgr2=0x556975e56240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 shutdown_connections 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 1 -- 172.21.15.19:0/3450237889 wait complete. 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.073+0000 7f402f690740 20 client.14979 trim_cache size 0 max 16384 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:18:37.081 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24407, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 76.252543222, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 128.93954292500001 2024-09-12T03:18:37.082 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 msgr2=0x7f40140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 0x7f40140a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f40200092f0 tx=0x7f402006dad0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 msgr2=0x7f402819cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f402000eaa0 tx=0x7f402000ef60 comp rx=0 tx=0).stop 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 shutdown_connections 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40140a2c10 0x7f40140a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f402810c180 0x7f402819cd00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4028103670 0x7f4028198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 --2- 172.21.15.19:0/262018133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4028102d50 0x7f4028198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:37.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 >> 172.21.15.19:0/262018133 conn(0x7f40280fc7d0 msgr2=0x7f40280fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:37.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 shutdown_connections 2024-09-12T03:18:37.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:37.074+0000 7f402d2b7640 1 -- 172.21.15.19:0/262018133 wait complete. 2024-09-12T03:18:37.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:36 smithi121 ceph-mon[34963]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:37.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-12T03:18:38.074 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:38.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 -- 172.21.15.19:0/2616185208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 msgr2=0x7fbf5c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 --2- 172.21.15.19:0/2616185208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c10ec40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c009cd0 tx=0x7fbf4c02a910 comp rx=0 tx=0).stop 2024-09-12T03:18:38.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 -- 172.21.15.19:0/2616185208 shutdown_connections 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 --2- 172.21.15.19:0/2616185208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c10ec40 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 --2- 172.21.15.19:0/2616185208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 0x7fbf5c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 --2- 172.21.15.19:0/2616185208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 0x7fbf5c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.547+0000 7fbf60c06640 1 -- 172.21.15.19:0/2616185208 >> 172.21.15.19:0/2616185208 conn(0x7fbf5c0fc7d0 msgr2=0x7fbf5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.548+0000 7fbf60c06640 1 -- 172.21.15.19:0/2616185208 shutdown_connections 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.548+0000 7fbf60c06640 1 -- 172.21.15.19:0/2616185208 wait complete. 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.548+0000 7fbf60c06640 1 Processor -- start 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.548+0000 7fbf60c06640 1 -- start start 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 0x7fbf5c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 0x7fbf5c198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c112160 con 0x7fbf5c1082a0 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c111fe0 con 0x7fbf5c102d30 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf60c06640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5c1122e0 con 0x7fbf5c1036e0 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38466/0 (socket says 172.21.15.19:38466) 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 -- 172.21.15.19:0/1457368719 learned_addr learned my addr 172.21.15.19:0/1457368719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 0x7fbf5c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 0x7fbf5c198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 -- 172.21.15.19:0/1457368719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 msgr2=0x7fbf5c198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 0x7fbf5c198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 -- 172.21.15.19:0/1457368719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 msgr2=0x7fbf5c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 0x7fbf5c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 -- 172.21.15.19:0/1457368719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5c19d660 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.549+0000 7fbf5ad9d640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005cc0 tx=0x7fbf4c031460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.550+0000 7fbf537fe640 1 -- 172.21.15.19:0/1457368719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c031850 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.550+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5c19d8f0 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.550+0000 7fbf537fe640 1 -- 172.21.15.19:0/1457368719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf4c004520 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.550+0000 7fbf537fe640 1 -- 172.21.15.19:0/1457368719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c02fd90 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.550+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5c1a5230 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.552+0000 7fbf537fe640 1 -- 172.21.15.19:0/1457368719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fbf4c0046d0 con 0x7fbf5c1082a0 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.552+0000 7fbf62fdf740 1 Processor -- start 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.552+0000 7fbf62fdf740 1 -- start start 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bd6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9be1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9b3d090 con 0x55b7c9be1bb0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9ae4a70 con 0x55b7c9bd69d0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9b39050 con 0x55b7c9bd72f0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf537fe640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 0x7fbf480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38468/0 (socket says 172.21.15.19:38468) 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2902038068 learned_addr learned my addr 172.21.15.19:0/2902038068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bd6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf5ad9d640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 0x7fbf480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9be1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf537fe640 1 -- 172.21.15.19:0/1457368719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf4c0f07b0 con 0x7fbf5c1082a0 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2902038068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 msgr2=0x55b7c9be1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9be1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2902038068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 msgr2=0x55b7c9bd6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bd6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2902038068 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c9af1130 con 0x55b7c9be1bb0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400afb0 tx=0x7fbf4400a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b28a1de64956c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.553+0000 7fbf5ad9d640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 0x7fbf480a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0bcf00 tx=0x7fbf4c0b2450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf50ff9640 1 -- 172.21.15.19:0/2902038068 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44016e50 con 0x55b7c9be1bb0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf50ff9640 1 -- 172.21.15.19:0/2902038068 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44018910 con 0x55b7c9be1bb0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf50ff9640 1 -- 172.21.15.19:0/2902038068 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44017810 con 0x55b7c9be1bb0 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2902038068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 msgr2=0x55b7c9be3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400afb0 tx=0x7fbf4400a250 comp rx=0 tx=0).stop 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2902038068 shutdown_connections 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9be3fa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9be1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2902038068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bd6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2902038068 >> 172.21.15.19:0/2902038068 conn(0x55b7c9bd2f60 msgr2=0x55b7c9bd3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.554+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2902038068 shutdown_connections 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2902038068 wait complete. 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 -- start start 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 Processor -- start 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9be1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bde7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9be4780 con 0x55b7c9be1bb0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9af7f20 con 0x55b7c9bd69d0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9be4900 con 0x55b7c9bd72f0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf62fdf740 10 client.0 _ll_register_callbacks cb 0x55b7c9ac1030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38478/0 (socket says 172.21.15.19:38478) 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.555+0000 7fbf59d9b640 1 -- 172.21.15.19:0/271819430 learned_addr learned my addr 172.21.15.19:0/271819430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9be1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bde7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 -- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 msgr2=0x55b7c9bde7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bde7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 -- 172.21.15.19:0/271819430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 msgr2=0x55b7c9be1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9be1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 -- 172.21.15.19:0/271819430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c98ae2b0 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bde7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400a510 tx=0x7fbf44006a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d58d9803311287 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44018cd0 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf44018430 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf44018910 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf44018630 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf62fdf740 10 client.14991 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7c9bf00a0 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.556+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44018910 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf44012050 con 0x55b7c9be1bb0 2024-09-12T03:18:38.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 10 client.14991 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 10 client.14991 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 20 client.14991 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9bf36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 4 client.14991 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7c9b4a600 con 0x55b7c9bf2280 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.557+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9bf36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.558+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9bf36f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=312971596afc4619 server_cookie=cb1df20789852d81 in_seq=0 out_seq=0 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.558+0000 7fbf41ffb640 10 client.14991 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.558+0000 7fbf41ffb640 1 -- 172.21.15.19:0/271819430 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7c9b4a600 con 0x55b7c9bf2280 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.558+0000 7fbf41ffb640 10 client.14991 handle_command_reply: tid=0 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 client.14991 shutdown 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 msgr2=0x55b7c9bdecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400a510 tx=0x7fbf44006a30 comp rx=0 tx=0).stop 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 shutdown_connections 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9bf36f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdecf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf41ffb640 0 client.14991 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bde7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/271819430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd69d0 0x55b7c9be1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 >> 172.21.15.19:0/271819430 conn(0x55b7c9bd2f60 msgr2=0x55b7c9bd8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 shutdown_connections 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 1 -- 172.21.15.19:0/271819430 wait complete. 2024-09-12T03:18:38.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.559+0000 7fbf62fdf740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.577+0000 7fbf62fdf740 1 Processor -- start 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.577+0000 7fbf62fdf740 1 -- start start 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf1a30 con 0x55b7c9bd69d0 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf2100 con 0x55b7c9bd72f0 2024-09-12T03:18:38.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf27d0 con 0x55b7c9be1bb0 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38482/0 (socket says 172.21.15.19:38482) 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2871849621 learned_addr learned my addr 172.21.15.19:0/2871849621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2871849621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9be1bb0 msgr2=0x55b7c9bdd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2871849621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 msgr2=0x55b7c9bf2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 -- 172.21.15.19:0/2871849621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7c9bd3bf0 con 0x55b7c9bd69d0 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:38.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf44016ea0 tx=0x7fbf44024f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d562f89053d136a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf43fff640 1 -- 172.21.15.19:0/2871849621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44017450 con 0x55b7c9bd69d0 2024-09-12T03:18:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf43fff640 1 -- 172.21.15.19:0/2871849621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44018430 con 0x55b7c9bd69d0 2024-09-12T03:18:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.578+0000 7fbf43fff640 1 -- 172.21.15.19:0/2871849621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44016f90 con 0x55b7c9bd69d0 2024-09-12T03:18:38.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2871849621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 msgr2=0x55b7c9bf1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fbf44016ea0 tx=0x7fbf44024f70 comp rx=0 tx=0).stop 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2871849621 shutdown_connections 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9be1bb0 0x55b7c9bdd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/2871849621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf1bb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2871849621 >> 172.21.15.19:0/2871849621 conn(0x55b7c9bd2f60 msgr2=0x55b7c9bcdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2871849621 shutdown_connections 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.579+0000 7fbf62fdf740 1 -- 172.21.15.19:0/2871849621 wait complete. 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 -- start start 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 Processor -- start 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9be1bb0 0x55b7c9c5a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf3570 con 0x55b7c9bd72f0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf2ea0 con 0x55b7c9be1bb0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b7c9bf3c40 con 0x55b7c9bd69d0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf62fdf740 10 client.0 _ll_register_callbacks cb 0x7fbf4400ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38494/0 (socket says 172.21.15.19:38494) 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 -- 172.21.15.19:0/4041890240 learned_addr learned my addr 172.21.15.19:0/4041890240 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9be1bb0 0x55b7c9c5a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 -- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 msgr2=0x55b7c9bf6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 -- 172.21.15.19:0/4041890240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9be1bb0 msgr2=0x55b7c9c5a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9be1bb0 0x55b7c9c5a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf5a59c640 1 -- 172.21.15.19:0/4041890240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7c9c5aec0 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.580+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf5a59c640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b7c9bdfdf0 tx=0x7fbf54018450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbca5271f0af8c7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf54022020 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf5401e610 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf44018430 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbf5400f040 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf62fdf740 10 client.14997 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7c9c5c0d0 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf44018430 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fbf540158f0 con 0x55b7c9bd72f0 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf62fdf740 10 client.14997 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:38.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.581+0000 7fbf62fdf740 10 client.14997 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf62fdf740 20 client.14997 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9c5f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf62fdf740 4 client.14997 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "status"}) -- 0x55b7c9b44b10 con 0x55b7c9bf2280 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9c5f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf59d9b640 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9c5f990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=440416cec0db8f8e server_cookie=5fd727174d33bc33 in_seq=0 out_seq=0 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.582+0000 7fbf41ffb640 10 client.14997 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf41ffb640 1 -- 172.21.15.19:0/4041890240 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b7c9b44b10 con 0x55b7c9bf2280 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf41ffb640 10 client.14997 handle_command_reply: tid=0 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 client.14997 shutdown 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 msgr2=0x55b7c9bf72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55b7c9bdfdf0 tx=0x7fbf54018450 comp rx=0 tx=0).stop 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 shutdown_connections 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b7c9bf2280 0x55b7c9c5f990 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b7c9be1bb0 0x55b7c9c5a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b7c9bd72f0 0x55b7c9bf72a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 --2- 172.21.15.19:0/4041890240 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b7c9bd69d0 0x55b7c9bf6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 >> 172.21.15.19:0/4041890240 conn(0x55b7c9bd2f60 msgr2=0x55b7c9bdd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf41ffb640 0 client.14997 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 shutdown_connections 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 1 -- 172.21.15.19:0/4041890240 wait complete. 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf62fdf740 20 client.14997 trim_cache size 0 max 16384 2024-09-12T03:18:38.589 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24421, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 78.769051504999993, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 133.126536773 2024-09-12T03:18:38.590 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 msgr2=0x7fbf480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 0x7fbf480a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c0bcf00 tx=0x7fbf4c0b2450 comp rx=0 tx=0).stop 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 msgr2=0x7fbf5c19cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.583+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c005cc0 tx=0x7fbf4c031460 comp rx=0 tx=0).stop 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 shutdown_connections 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf480a2c10 0x7fbf480a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf5c1082a0 0x7fbf5c19cf80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf5c1036e0 0x7fbf5c198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 --2- 172.21.15.19:0/1457368719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf5c102d30 0x7fbf5c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 >> 172.21.15.19:0/1457368719 conn(0x7fbf5c0fc7d0 msgr2=0x7fbf5c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 shutdown_connections 2024-09-12T03:18:38.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:38.584+0000 7fbf60c06640 1 -- 172.21.15.19:0/1457368719 wait complete. 2024-09-12T03:18:38.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:38 smithi019 ceph-mon[30953]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:39.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:38 smithi135 ceph-mon[35474]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:39.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:38 smithi121 ceph-mon[34963]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:39.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T03:18:39.689 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:40.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- 172.21.15.19:0/1727103390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44108300 msgr2=0x7f7e4410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/1727103390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44108300 0x7f7e4410ec70 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009930 tx=0x7f7e3c02a420 comp rx=0 tx=0).stop 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- 172.21.15.19:0/1727103390 shutdown_connections 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/1727103390 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44108300 0x7f7e4410ec70 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/1727103390 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44103780 0x7f7e44107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/1727103390 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44102dd0 0x7f7e441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- 172.21.15.19:0/1727103390 >> 172.21.15.19:0/1727103390 conn(0x7f7e440fc7d0 msgr2=0x7f7e440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- 172.21.15.19:0/1727103390 shutdown_connections 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- 172.21.15.19:0/1727103390 wait complete. 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 Processor -- start 2024-09-12T03:18:40.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.167+0000 7f7e4ac39640 1 -- start start 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44103780 0x7f7e44100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 0x7f7e441014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44112110 con 0x7f7e44102dd0 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44111f90 con 0x7f7e44103780 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7e44112290 con 0x7f7e44108300 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38502/0 (socket says 172.21.15.19:38502) 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 -- 172.21.15.19:0/4099612656 learned_addr learned my addr 172.21.15.19:0/4099612656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e43fff640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44103780 0x7f7e44100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e491d6640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 0x7f7e441014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 -- 172.21.15.19:0/4099612656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 msgr2=0x7f7e441014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 0x7f7e441014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 -- 172.21.15.19:0/4099612656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44103780 msgr2=0x7f7e44100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44103780 0x7f7e44100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 -- 172.21.15.19:0/4099612656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e441a6a50 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e491d6640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 0x7f7e441014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e489d5640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b4a0 tx=0x7f7e2c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e41ffb640 1 -- 172.21.15.19:0/4099612656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c014070 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e441a6d40 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.168+0000 7f7e41ffb640 1 -- 172.21.15.19:0/4099612656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e2c010af0 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.169+0000 7f7e41ffb640 1 -- 172.21.15.19:0/4099612656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e2c017d10 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.169+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e441a7220 con 0x7f7e44102dd0 2024-09-12T03:18:40.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.171+0000 7f7e41ffb640 1 -- 172.21.15.19:0/4099612656 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7e2c017070 con 0x7f7e44102dd0 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.171+0000 7f7e4d012740 1 Processor -- start 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 -- start start 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e41ffb640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 0x7f7e300a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f1cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558494e76090 con 0x558494f102f0 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558494e1da70 con 0x558494f1abb0 2024-09-12T03:18:40.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e4d012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558494e72050 con 0x558494f0f9d0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e489d5640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 0x7f7e300a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e41ffb640 1 -- 172.21.15.19:0/4099612656 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7e2c0ce420 con 0x7f7e44102dd0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e491d6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38512/0 (socket says 172.21.15.19:38512) 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 -- 172.21.15.19:0/3658779875 learned_addr learned my addr 172.21.15.19:0/3658779875 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e491d6640 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f1cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 -- 172.21.15.19:0/3658779875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 msgr2=0x558494f0fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f0fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 -- 172.21.15.19:0/3658779875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 msgr2=0x558494f1cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f1cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e489d5640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 0x7f7e300a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0970a0 tx=0x7f7e2c06cc50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e43fff640 1 -- 172.21.15.19:0/3658779875 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558494e2a130 con 0x558494f102f0 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.172+0000 7f7e491d6640 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e43fff640 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ae90 tx=0x7f7e3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f303730102fe24b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e3affd640 1 -- 172.21.15.19:0/3658779875 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34003b70 con 0x558494f102f0 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e3affd640 1 -- 172.21.15.19:0/3658779875 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e34003d10 con 0x558494f102f0 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e3affd640 1 -- 172.21.15.19:0/3658779875 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e34010ba0 con 0x558494f102f0 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e4d012740 1 -- 172.21.15.19:0/3658779875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 msgr2=0x558494f1a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.173+0000 7f7e4d012740 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400ae90 tx=0x7f7e3400f790 comp rx=0 tx=0).stop 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- 172.21.15.19:0/3658779875 shutdown_connections 2024-09-12T03:18:40.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f1cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f1a670 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- 172.21.15.19:0/3658779875 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f0fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- 172.21.15.19:0/3658779875 >> 172.21.15.19:0/3658779875 conn(0x558494f0bf60 msgr2=0x558494f0c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- 172.21.15.19:0/3658779875 shutdown_connections 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- 172.21.15.19:0/3658779875 wait complete. 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- start start 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 Processor -- start 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f176a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f17be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558494f1d780 con 0x558494f0f9d0 2024-09-12T03:18:40.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558494e30f20 con 0x558494f1abb0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.174+0000 7f7e4d012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558494f1d900 con 0x558494f102f0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e4d012740 10 client.0 _ll_register_callbacks cb 0x558494dfa030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38528/0 (socket says 172.21.15.19:38528) 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 -- 172.21.15.19:0/1624456425 learned_addr learned my addr 172.21.15.19:0/1624456425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e43fff640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f176a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f17be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 -- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 msgr2=0x558494f176a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f176a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 -- 172.21.15.19:0/1624456425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 msgr2=0x558494f17be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f17be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 -- 172.21.15.19:0/1624456425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558494be72b0 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e491d6640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009230 tx=0x7f7e3c032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236e2da89f3713d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c035040 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7e3c004480 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e34003d10 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e3c005020 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e4d012740 10 client.15009 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.175+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558494f28fc0 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e34003d10 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e3c03a040 con 0x558494f0f9d0 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 10 client.15009 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 10 client.15009 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:18:40.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 20 client.15009 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f2b5d0 0x558494f2ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 4 client.15009 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558494e83600 con 0x558494f2b5d0 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.176+0000 7f7e43fff640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f2b5d0 0x558494f2ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.177+0000 7f7e23fff640 10 client.15009 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.177+0000 7f7e43fff640 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f2b5d0 0x558494f2ca40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=930db11c0398e87a server_cookie=7177a9a40867b20a in_seq=0 out_seq=0 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e23fff640 1 -- 172.21.15.19:0/1624456425 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558494e83600 con 0x558494f2b5d0 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e23fff640 10 client.15009 handle_command_reply: tid=0 2024-09-12T03:18:40.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 client.15009 shutdown 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 msgr2=0x558494f1a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c009230 tx=0x7f7e3c032d80 comp rx=0 tx=0).stop 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 shutdown_connections 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f2b5d0 0x558494f2ca40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f17be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f176a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 --2- 172.21.15.19:0/1624456425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f0f9d0 0x558494f1a350 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e23fff640 0 client.15009 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 >> 172.21.15.19:0/1624456425 conn(0x558494f0bf60 msgr2=0x558494f11c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 shutdown_connections 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 1 -- 172.21.15.19:0/1624456425 wait complete. 2024-09-12T03:18:40.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.178+0000 7f7e4d012740 20 client.15009 trim_cache size 0 max 16384 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 Processor -- start 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 -- start start 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f0f9d0 0x558494f2b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f17bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558494f2ad80 con 0x558494f1abb0 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558494f2b450 con 0x558494f0f9d0 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e4d012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558494f2bb20 con 0x558494f102f0 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38534/0 (socket says 172.21.15.19:38534) 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 -- 172.21.15.19:0/149973821 learned_addr learned my addr 172.21.15.19:0/149973821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f0f9d0 0x558494f2b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e43fff640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f17bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 -- 172.21.15.19:0/149973821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 msgr2=0x558494f17bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f17bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 -- 172.21.15.19:0/149973821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f0f9d0 msgr2=0x558494f2b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f0f9d0 0x558494f2b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e491d6640 1 -- 172.21.15.19:0/149973821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558494f0bd90 con 0x558494f1abb0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.196+0000 7f7e43fff640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f17bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e491d6640 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005dc0 tx=0x7f7e3c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b1031f2fc5737d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e3a7fc640 1 -- 172.21.15.19:0/149973821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c035070 con 0x558494f1abb0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e3a7fc640 1 -- 172.21.15.19:0/149973821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c004480 con 0x558494f1abb0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e3a7fc640 1 -- 172.21.15.19:0/149973821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3c004c20 con 0x558494f1abb0 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 -- 172.21.15.19:0/149973821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 msgr2=0x558494f34d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7e3c005dc0 tx=0x7f7e3c005d50 comp rx=0 tx=0).stop 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 -- 172.21.15.19:0/149973821 shutdown_connections 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f1abb0 0x558494f34d20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f102f0 0x558494f17bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 --2- 172.21.15.19:0/149973821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f0f9d0 0x558494f2b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 -- 172.21.15.19:0/149973821 >> 172.21.15.19:0/149973821 conn(0x558494f0bf60 msgr2=0x558494f15f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 -- 172.21.15.19:0/149973821 shutdown_connections 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.197+0000 7f7e4d012740 1 -- 172.21.15.19:0/149973821 wait complete. 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.198+0000 7f7e4d012740 1 -- start start 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.198+0000 7f7e4d012740 1 Processor -- start 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f30100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f93cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558494f2c8c0 con 0x558494f102f0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558494f2c1f0 con 0x558494f1abb0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558494f2cf90 con 0x558494f0f9d0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e4d012740 10 client.0 _ll_register_callbacks cb 0x7f7e3c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38540/0 (socket says 172.21.15.19:38540) 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 -- 172.21.15.19:0/2560961039 learned_addr learned my addr 172.21.15.19:0/2560961039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e491d6640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f30100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e491d6640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f93cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 -- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 msgr2=0x558494f30100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f30100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 -- 172.21.15.19:0/2560961039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 msgr2=0x558494f93cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f93cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 -- 172.21.15.19:0/2560961039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558494f94210 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e491d6640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e43fff640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e34004990 tx=0x7f7e3400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477125838ce9bab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.199+0000 7f7e23fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3400a990 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7e340108b0 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e3c004480 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7e34010ab0 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 10 client.15015 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558494f953e0 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e3c004480 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7e3402c020 con 0x558494f102f0 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 10 client.15015 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:40.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 10 client.15015 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:18:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 20 client.15015 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f99e80 0x558494f98b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 4 client.15015 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:18:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.200+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "status"}) -- 0x558494e7db10 con 0x558494f99e80 2024-09-12T03:18:40.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e491d6640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f99e80 0x558494f98b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e491d6640 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f99e80 0x558494f98b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9ca163c9dce7066 server_cookie=585c50a647f5e4dd in_seq=0 out_seq=0 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e23fff640 10 client.15015 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e23fff640 1 -- 172.21.15.19:0/2560961039 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558494e7db10 con 0x558494f99e80 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e23fff640 10 client.15015 handle_command_reply: tid=0 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.201+0000 7f7e4d012740 1 client.15015 shutdown 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 msgr2=0x558494f30640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7e34004990 tx=0x7f7e3400fd50 comp rx=0 tx=0).stop 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 shutdown_connections 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x558494f99e80 0x558494f98b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558494f1abb0 0x558494f93cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e23fff640 0 client.15015 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558494f102f0 0x558494f30640 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 --2- 172.21.15.19:0/2560961039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558494f0f9d0 0x558494f30100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 >> 172.21.15.19:0/2560961039 conn(0x558494f0bf60 msgr2=0x558494f18d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 shutdown_connections 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 1 -- 172.21.15.19:0/2560961039 wait complete. 2024-09-12T03:18:40.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4d012740 20 client.15015 trim_cache size 0 max 16384 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24463, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 81.246521748000006, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:18:40.221 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 117.194675786 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 msgr2=0x7f7e300a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 0x7f7e300a50d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c0970a0 tx=0x7f7e2c06cc50 comp rx=0 tx=0).stop 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 msgr2=0x7f7e44100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.202+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00b4a0 tx=0x7f7e2c00b960 comp rx=0 tx=0).stop 2024-09-12T03:18:40.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 shutdown_connections 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7e44108300 0x7f7e441014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7e44103780 0x7f7e44100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7e300a2c10 0x7f7e300a50d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 --2- 172.21.15.19:0/4099612656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7e44102dd0 0x7f7e44100a30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 >> 172.21.15.19:0/4099612656 conn(0x7f7e440fc7d0 msgr2=0x7f7e441063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 shutdown_connections 2024-09-12T03:18:40.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:40.203+0000 7f7e4ac39640 1 -- 172.21.15.19:0/4099612656 wait complete. 2024-09-12T03:18:40.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:40 smithi019 ceph-mon[30953]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:41.011 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:18:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:40 smithi135 ceph-mon[35474]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:41.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:40 smithi121 ceph-mon[34963]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:18:41.302 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 -- 172.21.15.19:0/275425058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 msgr2=0x7fb78810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/275425058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb78810d950 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb780009cd0 tx=0x7fb78002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 -- 172.21.15.19:0/275425058 shutdown_connections 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/275425058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb78810d950 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/275425058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/275425058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 0x7fb788106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 -- 172.21.15.19:0/275425058 >> 172.21.15.19:0/275425058 conn(0x7fb7880fc7a0 msgr2=0x7fb7880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 -- 172.21.15.19:0/275425058 shutdown_connections 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.824+0000 7fb78fa8d640 1 -- 172.21.15.19:0/275425058 wait complete. 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 Processor -- start 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 -- start start 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 0x7fb7881a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb788110ec0 con 0x7fb788101420 2024-09-12T03:18:41.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb788110d40 con 0x7fb788100a80 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78fa8d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb788111040 con 0x7fb788106a80 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.825+0000 7fb78d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56718/0 (socket says 172.21.15.19:56718) 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 -- 172.21.15.19:0/427804527 learned_addr learned my addr 172.21.15.19:0/427804527 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78e02a640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 0x7fb7881a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d829640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 -- 172.21.15.19:0/427804527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 msgr2=0x7fb7881a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 0x7fb7881a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 -- 172.21.15.19:0/427804527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 msgr2=0x7fb788199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 -- 172.21.15.19:0/427804527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7881a48d0 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d829640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78d028640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b4a0 tx=0x7fb77800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb77effd640 1 -- 172.21.15.19:0/427804527 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778014070 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb77effd640 1 -- 172.21.15.19:0/427804527 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb778010af0 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7881a4aa0 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb77effd640 1 -- 172.21.15.19:0/427804527 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778017d10 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.826+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881a4fe0 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.828+0000 7fb77effd640 1 -- 172.21.15.19:0/427804527 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb778017070 con 0x7fb788101420 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 Processor -- start 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 -- start start 2024-09-12T03:18:41.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb77effd640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 0x7fb7740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb78e02a640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 0x7fb7740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c78d0 0x5565b57c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb77effd640 1 -- 172.21.15.19:0/427804527 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb7780ce320 con 0x7fb788101420 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565b572e090 con 0x5565b57d6fc0 2024-09-12T03:18:41.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565b56d5a70 con 0x5565b57c78d0 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.829+0000 7fb791e66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565b572a050 con 0x5565b57c81f0 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78e02a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78e02a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56732/0 (socket says 172.21.15.19:56732) 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78e02a640 1 -- 172.21.15.19:0/1767715098 learned_addr learned my addr 172.21.15.19:0/1767715098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d829640 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c78d0 0x5565b57c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 -- 172.21.15.19:0/1767715098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c78d0 msgr2=0x5565b57c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c78d0 0x5565b57c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 -- 172.21.15.19:0/1767715098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 msgr2=0x5565b57d93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78e02a640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 0x7fb7740a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb780009cd0 tx=0x7fb7800319a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 -- 172.21.15.19:0/1767715098 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b56e2130 con 0x5565b57c81f0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb78d028640 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d6a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb778009420 tx=0x7fb77800bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2db06400785f746 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.830+0000 7fb773fff640 1 -- 172.21.15.19:0/1767715098 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77801e070 con 0x5565b57c81f0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb773fff640 1 -- 172.21.15.19:0/1767715098 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77809ef40 con 0x5565b57c81f0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb773fff640 1 -- 172.21.15.19:0/1767715098 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778097200 con 0x5565b57c81f0 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 -- 172.21.15.19:0/1767715098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 msgr2=0x5565b57d6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d6a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb778009420 tx=0x7fb77800bf90 comp rx=0 tx=0).stop 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 -- 172.21.15.19:0/1767715098 shutdown_connections 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d6a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 --2- 172.21.15.19:0/1767715098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c78d0 0x5565b57c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 -- 172.21.15.19:0/1767715098 >> 172.21.15.19:0/1767715098 conn(0x5565b57c3f60 msgr2=0x5565b57c4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 -- 172.21.15.19:0/1767715098 shutdown_connections 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.831+0000 7fb791e66740 1 -- 172.21.15.19:0/1767715098 wait complete. 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 -- start start 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 Processor -- start 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c81f0 0x5565b57d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565b57d9bf0 con 0x5565b57d6fc0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565b56e8f20 con 0x5565b57c81f0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565b57d9d70 con 0x5565b57c78d0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb791e66740 10 client.0 _ll_register_callbacks cb 0x5565b56b2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb78e02a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb78e02a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56736/0 (socket says 172.21.15.19:56736) 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb78e02a640 1 -- 172.21.15.19:0/1850392588 learned_addr learned my addr 172.21.15.19:0/1850392588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb78d829640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.832+0000 7fb78d028640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c81f0 0x5565b57d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 -- 172.21.15.19:0/1850392588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 msgr2=0x5565b57d68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 -- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c81f0 msgr2=0x5565b57d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c81f0 0x5565b57d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 -- 172.21.15.19:0/1850392588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b549f2b0 con 0x5565b57d6fc0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb78e02a640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ca0 tx=0x7fb78002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152e5d46d922a2e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb7717fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7800309b0 con 0x5565b57d6fc0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77809ef40 con 0x5565b57d6fc0 2024-09-12T03:18:41.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb780030b50 con 0x5565b57d6fc0 2024-09-12T03:18:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb780003e20 con 0x5565b57d6fc0 2024-09-12T03:18:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb791e66740 10 client.15027 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565b57e5630 con 0x5565b57d6fc0 2024-09-12T03:18:41.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.833+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77809ef40 con 0x5565b57d6fc0 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb78005c020 con 0x5565b57d6fc0 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 10 client.15027 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 10 client.15027 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 20 client.15027 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b57e7bb0 0x5565b57e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 4 client.15027 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565b573b600 con 0x5565b57e7bb0 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb78d829640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b57e7bb0 0x5565b57e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb78d829640 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b57e7bb0 0x5565b57e9020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8dc169bbf306214b server_cookie=c5f8f7faa0e03978 in_seq=0 out_seq=0 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.834+0000 7fb7717fa640 10 client.15027 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:18:41.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb7717fa640 1 -- 172.21.15.19:0/1850392588 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565b573b600 con 0x5565b57e7bb0 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb7717fa640 10 client.15027 handle_command_reply: tid=0 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 client.15027 shutdown 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 msgr2=0x5565b57d4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ca0 tx=0x7fb78002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 shutdown_connections 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b57e7bb0 0x5565b57e9020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57d6fc0 0x5565b57d4180 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57c81f0 0x5565b57d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 --2- 172.21.15.19:0/1850392588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb7717fa640 0 client.15027 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:18:41.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 >> 172.21.15.19:0/1850392588 conn(0x5565b57c3f60 msgr2=0x5565b57c9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 shutdown_connections 2024-09-12T03:18:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 1 -- 172.21.15.19:0/1850392588 wait complete. 2024-09-12T03:18:41.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.835+0000 7fb791e66740 20 client.15027 trim_cache size 0 max 16384 2024-09-12T03:18:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.853+0000 7fb791e66740 1 Processor -- start 2024-09-12T03:18:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.853+0000 7fb791e66740 1 -- start start 2024-09-12T03:18:41.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e7360 con 0x5565b57c78d0 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e7a30 con 0x5565b57d6fc0 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb791e66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e8100 con 0x5565b57c81f0 2024-09-12T03:18:41.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56738/0 (socket says 172.21.15.19:56738) 2024-09-12T03:18:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 -- 172.21.15.19:0/2382438639 learned_addr learned my addr 172.21.15.19:0/2382438639 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78e02a640 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 -- 172.21.15.19:0/2382438639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 msgr2=0x5565b57d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 -- 172.21.15.19:0/2382438639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 msgr2=0x5565b57cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 -- 172.21.15.19:0/2382438639 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565b57d6510 con 0x5565b57c78d0 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb78d829640 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5565b57d4850 tx=0x7fb78400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e6c78fd4d51e51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb7737fe640 1 -- 172.21.15.19:0/2382438639 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78401b620 con 0x5565b57c78d0 2024-09-12T03:18:41.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb7737fe640 1 -- 172.21.15.19:0/2382438639 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78400fe50 con 0x5565b57c78d0 2024-09-12T03:18:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.854+0000 7fb7737fe640 1 -- 172.21.15.19:0/2382438639 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78401abf0 con 0x5565b57c78d0 2024-09-12T03:18:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 -- 172.21.15.19:0/2382438639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 msgr2=0x5565b57d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5565b57d4850 tx=0x7fb78400f7f0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 -- 172.21.15.19:0/2382438639 shutdown_connections 2024-09-12T03:18:41.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c81f0 0x5565b57d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 --2- 172.21.15.19:0/2382438639 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c78d0 0x5565b57d0ec0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 -- 172.21.15.19:0/2382438639 >> 172.21.15.19:0/2382438639 conn(0x5565b57c3f60 msgr2=0x5565b57d2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 -- 172.21.15.19:0/2382438639 shutdown_connections 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.855+0000 7fb791e66740 1 -- 172.21.15.19:0/2382438639 wait complete. 2024-09-12T03:18:41.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 -- start start 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 Processor -- start 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e8ea0 con 0x5565b57c81f0 2024-09-12T03:18:41.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e87d0 con 0x5565b57d6fc0 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5565b57e9570 con 0x5565b57c78d0 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb78d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb791e66740 10 client.0 _ll_register_callbacks cb 0x7fb7800099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb78d028640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56752/0 (socket says 172.21.15.19:56752) 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb78d028640 1 -- 172.21.15.19:0/2452972228 learned_addr learned my addr 172.21.15.19:0/2452972228 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:41.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb78d829640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.856+0000 7fb78e02a640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 -- 172.21.15.19:0/2452972228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 msgr2=0x5565b57d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 -- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 msgr2=0x5565b57d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.877 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 -- 172.21.15.19:0/2452972228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565b57ef7f0 con 0x5565b57c81f0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb78d028640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b470 tx=0x7fb77809a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163410ee8fb748de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb7717fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77801e040 con 0x5565b57c81f0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb778010ca0 con 0x5565b57c81f0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78400fe50 con 0x5565b57c81f0 2024-09-12T03:18:41.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb77809ef40 con 0x5565b57c81f0 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb791e66740 10 client.15033 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5565b5852560 con 0x5565b57c81f0 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.857+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78400fe50 con 0x5565b57c81f0 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb778082080 con 0x5565b57c81f0 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 10 client.15033 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 10 client.15033 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:18:41.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 20 client.15033 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b5856c00 0x5565b5855910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 4 client.15033 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:18:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "status"}) -- 0x5565b5735b10 con 0x5565b5856c00 2024-09-12T03:18:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb78d829640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b5856c00 0x5565b5855910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:41.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb78d829640 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b5856c00 0x5565b5855910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f201c4525bfcc1b9 server_cookie=c3e530d6e8413aee in_seq=0 out_seq=0 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.858+0000 7fb7717fa640 10 client.15033 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb7717fa640 1 -- 172.21.15.19:0/2452972228 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5565b5735b10 con 0x5565b5856c00 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb7717fa640 10 client.15033 handle_command_reply: tid=0 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 client.15033 shutdown 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 msgr2=0x5565b57d1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b470 tx=0x7fb77809a7e0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 shutdown_connections 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5565b5856c00 0x5565b5855910 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5565b57d6fc0 0x5565b57d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5565b57c81f0 0x5565b57d1260 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb7717fa640 0 client.15033 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:18:41.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 --2- 172.21.15.19:0/2452972228 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5565b57c78d0 0x5565b57d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 >> 172.21.15.19:0/2452972228 conn(0x5565b57c3f60 msgr2=0x5565b57cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 shutdown_connections 2024-09-12T03:18:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 1 -- 172.21.15.19:0/2452972228 wait complete. 2024-09-12T03:18:41.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb791e66740 20 client.15033 trim_cache size 0 max 16384 2024-09-12T03:18:41.884 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:18:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:18:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-12T03:18:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24467, 2024-09-12T03:18:41.884 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 83.912107219000006, 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:18:41.885 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:18:41.886 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 103.261957302 2024-09-12T03:18:41.886 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:18:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 msgr2=0x7fb7740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 0x7fb7740a50c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb780009cd0 tx=0x7fb7800319a0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 msgr2=0x7fb788199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.859+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b4a0 tx=0x7fb77800b960 comp rx=0 tx=0).stop 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 shutdown_connections 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb788106a80 0x7fb7881a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb7740a2c00 0x7fb7740a50c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb788101420 0x7fb788199850 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 --2- 172.21.15.19:0/427804527 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb788100a80 0x7fb788199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:41.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 >> 172.21.15.19:0/427804527 conn(0x7fb7880fc7a0 msgr2=0x7fb7880fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:41.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 shutdown_connections 2024-09-12T03:18:41.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:41.860+0000 7fb78fa8d640 1 -- 172.21.15.19:0/427804527 wait complete. 2024-09-12T03:18:42.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:18:42.996 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:42 smithi135 ceph-mon[35474]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:42 smithi135 ceph-mon[35474]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:18:43.117 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:42 smithi019 ceph-mon[30953]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:43.118 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:42 smithi019 ceph-mon[30953]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:18:43.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:42 smithi121 ceph-mon[34963]: pgmap v327: 97 pgs: 97 active+clean; 642 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:18:43.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:42 smithi121 ceph-mon[34963]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 -- 172.21.15.19:0/3502482921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024101de0 msgr2=0x7f60241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 --2- 172.21.15.19:0/3502482921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024101de0 0x7f60241021c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6014009900 tx=0x7f601402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 -- 172.21.15.19:0/3502482921 shutdown_connections 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 --2- 172.21.15.19:0/3502482921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024106f10 0x7f602410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 --2- 172.21.15.19:0/3502482921 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024102790 0x7f60241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 --2- 172.21.15.19:0/3502482921 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024101de0 0x7f60241021c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 -- 172.21.15.19:0/3502482921 >> 172.21.15.19:0/3502482921 conn(0x7f60240fd8c0 msgr2=0x7f60240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 -- 172.21.15.19:0/3502482921 shutdown_connections 2024-09-12T03:18:43.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.463+0000 7f602a5e8640 1 -- 172.21.15.19:0/3502482921 wait complete. 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 Processor -- start 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- start start 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024101de0 0x7f60241983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024102790 0x7f6024198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6024111f70 con 0x7f6024106f10 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6024111df0 con 0x7f6024102790 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60241120f0 con 0x7f6024101de0 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56760/0 (socket says 172.21.15.19:56760) 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 -- 172.21.15.19:0/1536672425 learned_addr learned my addr 172.21.15.19:0/1536672425 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:43.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6023fff640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024101de0 0x7f60241983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f60237fe640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024102790 0x7f6024198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 -- 172.21.15.19:0/1536672425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024101de0 msgr2=0x7f60241983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024101de0 0x7f60241983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 -- 172.21.15.19:0/1536672425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024102790 msgr2=0x7f6024198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024102790 0x7f6024198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 -- 172.21.15.19:0/1536672425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602419d380 con 0x7f6024106f10 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f6028b85640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cce0 tx=0x7f601c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f60217fa640 1 -- 172.21.15.19:0/1536672425 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c00f450 con 0x7f6024106f10 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602419d670 con 0x7f6024106f10 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f60217fa640 1 -- 172.21.15.19:0/1536672425 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c00fad0 con 0x7f6024106f10 2024-09-12T03:18:43.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f60217fa640 1 -- 172.21.15.19:0/1536672425 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018c10 con 0x7f6024106f10 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.465+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60241a4f50 con 0x7f6024106f10 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.467+0000 7f60217fa640 1 -- 172.21.15.19:0/1536672425 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f601c00f5f0 con 0x7f6024106f10 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 Processor -- start 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 -- start start 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfaedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfbbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fcdf15090 con 0x559fcdfaf2f0 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fcdebca70 con 0x559fcdfae9d0 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f602c9c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fcdf11050 con 0x559fcdfb9bb0 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56762/0 (socket says 172.21.15.19:56762) 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 -- 172.21.15.19:0/2780688734 learned_addr learned my addr 172.21.15.19:0/2780688734 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f60237fe640 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfaedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 -- 172.21.15.19:0/2780688734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 msgr2=0x559fcdfbbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfbbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 -- 172.21.15.19:0/2780688734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 msgr2=0x559fcdfaedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfaedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f6023fff640 1 -- 172.21.15.19:0/2780688734 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcdec9130 con 0x559fcdfaf2f0 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f60237fe640 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfaedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.468+0000 7f60217fa640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 0x7f600c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f60237fe640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 0x7f600c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f6023fff640 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6014009900 tx=0x7f6014005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca69db08adb458f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f60217fa640 1 -- 172.21.15.19:0/1536672425 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601c0cf610 con 0x7f6024106f10 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f600a7fc640 1 -- 172.21.15.19:0/2780688734 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014040070 con 0x559fcdfaf2f0 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f600a7fc640 1 -- 172.21.15.19:0/2780688734 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60140319f0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.469+0000 7f600a7fc640 1 -- 172.21.15.19:0/2780688734 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6014035040 con 0x559fcdfaf2f0 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- 172.21.15.19:0/2780688734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 msgr2=0x559fcdfb9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6014009900 tx=0x7f6014005e80 comp rx=0 tx=0).stop 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f60237fe640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 0x7f600c0a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559fcdfac2e0 tx=0x7f6018009940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- 172.21.15.19:0/2780688734 shutdown_connections 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfbbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb9670 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2780688734 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfaedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- 172.21.15.19:0/2780688734 >> 172.21.15.19:0/2780688734 conn(0x559fcdfaaf60 msgr2=0x559fcdfab340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- 172.21.15.19:0/2780688734 shutdown_connections 2024-09-12T03:18:43.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- 172.21.15.19:0/2780688734 wait complete. 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 -- start start 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.470+0000 7f602c9c1740 1 Processor -- start 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 0x559fcdfb9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfbc780 con 0x559fcdfae9d0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fcdecff20 con 0x559fcdfaf2f0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfbc900 con 0x559fcdfb9bb0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f602c9c1740 10 client.0 _ll_register_callbacks cb 0x559fcde99030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f6028b85640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:46258/0 (socket says 172.21.15.19:46258) 2024-09-12T03:18:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f6023fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 0x559fcdfb9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 -- 172.21.15.19:0/2672623108 learned_addr learned my addr 172.21.15.19:0/2672623108 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 -- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 msgr2=0x559fcdfb6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 -- 172.21.15.19:0/2672623108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 msgr2=0x559fcdfb9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 0x559fcdfb9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f60237fe640 1 -- 172.21.15.19:0/2672623108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcdc862b0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.471+0000 7f6023fff640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 0x559fcdfb9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60237fe640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6018005620 tx=0x7f6018005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c21b3cba0c01df3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018027440 con 0x559fcdfaf2f0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60140319f0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601801d430 con 0x559fcdfaf2f0 2024-09-12T03:18:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f601801d630 con 0x559fcdfaf2f0 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f602c9c1740 10 client.24728 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fcdfc80c0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60140319f0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.472+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f6018025070 con 0x559fcdfaf2f0 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 10 client.24728 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 10 client.24728 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 20 client.24728 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fcdfca740 0x559fcdfcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 4 client.24728 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcdf22600 con 0x559fcdfca740 2024-09-12T03:18:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f6023fff640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fcdfca740 0x559fcdfcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f6023fff640 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fcdfca740 0x559fcdfcbbb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8dfe787c436f443 server_cookie=68b4efa9daf2b724 in_seq=0 out_seq=0 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.473+0000 7f60037fe640 10 client.24728 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f60037fe640 1 -- 172.21.15.19:0/2672623108 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fcdf22600 con 0x559fcdfca740 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f60037fe640 10 client.24728 handle_command_reply: tid=0 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 client.24728 shutdown 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 msgr2=0x559fcdfb67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6018005620 tx=0x7f6018005f90 comp rx=0 tx=0).stop 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 shutdown_connections 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fcdfca740 0x559fcdfcbbb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfb67a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 --2- 172.21.15.19:0/2672623108 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfae9d0 0x559fcdfb9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f60037fe640 0 client.24728 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:18:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 >> 172.21.15.19:0/2672623108 conn(0x559fcdfaaf60 msgr2=0x559fcdfb0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 shutdown_connections 2024-09-12T03:18:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.474+0000 7f602c9c1740 1 -- 172.21.15.19:0/2672623108 wait complete. 2024-09-12T03:18:43.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.475+0000 7f602c9c1740 20 client.24728 trim_cache size 0 max 16384 2024-09-12T03:18:43.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.492+0000 7f602c9c1740 1 Processor -- start 2024-09-12T03:18:43.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.492+0000 7f602c9c1740 1 -- start start 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfcbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfc9ef0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfca5c0 con 0x559fcdfae9d0 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f602c9c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfcac90 con 0x559fcdfb9bb0 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56776/0 (socket says 172.21.15.19:56776) 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 -- 172.21.15.19:0/675539402 learned_addr learned my addr 172.21.15.19:0/675539402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 -- 172.21.15.19:0/675539402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 msgr2=0x559fcdfb0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f6023fff640 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfcbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 -- 172.21.15.19:0/675539402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 msgr2=0x559fcdfcbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfcbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 -- 172.21.15.19:0/675539402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcdfb4cf0 con 0x559fcdfaf2f0 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f6023fff640 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfcbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f60237fe640 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60180055f0 tx=0x7f6018004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d69a1d38e5ab04b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f6009ffb640 1 -- 172.21.15.19:0/675539402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6018035030 con 0x559fcdfaf2f0 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f6009ffb640 1 -- 172.21.15.19:0/675539402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6018005b90 con 0x559fcdfaf2f0 2024-09-12T03:18:43.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.493+0000 7f6009ffb640 1 -- 172.21.15.19:0/675539402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601801dd60 con 0x559fcdfaf2f0 2024-09-12T03:18:43.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- 172.21.15.19:0/675539402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 msgr2=0x559fcdfab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f60180055f0 tx=0x7f6018004f50 comp rx=0 tx=0).stop 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- 172.21.15.19:0/675539402 shutdown_connections 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfb9bb0 0x559fcdfb0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfaf2f0 0x559fcdfab340 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 --2- 172.21.15.19:0/675539402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfae9d0 0x559fcdfcbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- 172.21.15.19:0/675539402 >> 172.21.15.19:0/675539402 conn(0x559fcdfaaf60 msgr2=0x559fcdfb5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- 172.21.15.19:0/675539402 shutdown_connections 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- 172.21.15.19:0/675539402 wait complete. 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.494+0000 7f602c9c1740 1 -- start start 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 Processor -- start 2024-09-12T03:18:43.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 0x559fcdfcd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfcdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfcba30 con 0x559fcdfb9bb0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfcb360 con 0x559fcdfaf2f0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fcdfcc100 con 0x559fcdfae9d0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f602c9c1740 10 client.0 _ll_register_callbacks cb 0x7f6018010f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f6028b85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f6028b85640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56786/0 (socket says 172.21.15.19:56786) 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f6028b85640 1 -- 172.21.15.19:0/3609449480 learned_addr learned my addr 172.21.15.19:0/3609449480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f60237fe640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfcdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.495+0000 7f6023fff640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 0x559fcdfcd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 -- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 msgr2=0x559fcdfcd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 0x559fcdfcd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 -- 172.21.15.19:0/3609449480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 msgr2=0x559fcdfcdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfcdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 -- 172.21.15.19:0/3609449480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fce0304d0 con 0x559fcdfb9bb0 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60237fe640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfcdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6023fff640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 0x559fcdfcd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f6028b85640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f601c09bdb0 tx=0x7f601c09ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faf0ce73c36a0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c0b9020 con 0x559fcdfb9bb0 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601c0a4600 con 0x559fcdfb9bb0 2024-09-12T03:18:43.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6018005b90 con 0x559fcdfb9bb0 2024-09-12T03:18:43.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f601c081050 con 0x559fcdfb9bb0 2024-09-12T03:18:43.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 10 client.15051 fetch_fsmap learned FSMap version 27 2024-09-12T03:18:43.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fce0389b0 con 0x559fcdfb9bb0 2024-09-12T03:18:43.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6018005b90 con 0x559fcdfb9bb0 2024-09-12T03:18:43.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f601c0cf0e0 con 0x559fcdfb9bb0 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 10 client.15051 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 10 client.15051 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.496+0000 7f602c9c1740 20 client.15051 populate_metadata read hostname 'smithi019' 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fce03d4e0 0x559fce03c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 4 client.15051 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "status"}) -- 0x559fcdf1cb10 con 0x559fce03d4e0 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f6023fff640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fce03d4e0 0x559fce03c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f60037fe640 10 client.15051 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f6023fff640 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fce03d4e0 0x559fce03c1f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f082525035f09f4 server_cookie=aea740c2127fc51 in_seq=0 out_seq=0 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f60037fe640 1 -- 172.21.15.19:0/3609449480 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559fcdf1cb10 con 0x559fce03d4e0 2024-09-12T03:18:43.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f60037fe640 10 client.15051 handle_command_reply: tid=0 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 1 client.15051 shutdown 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 msgr2=0x559fce02fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.497+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f601c09bdb0 tx=0x7f601c09ba20 comp rx=0 tx=0).stop 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 shutdown_connections 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fcdfb9bb0 0x559fce02fe30 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559fce03d4e0 0x559fce03c1f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fcdfaf2f0 0x559fcdfcdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 --2- 172.21.15.19:0/3609449480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fcdfae9d0 0x559fcdfcd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 >> 172.21.15.19:0/3609449480 conn(0x559fcdfaaf60 msgr2=0x559fcdfb14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f60037fe640 0 client.15051 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 shutdown_connections 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 1 -- 172.21.15.19:0/3609449480 wait complete. 2024-09-12T03:18:43.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602c9c1740 20 client.15051 trim_cache size 0 max 16384 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 92.590881261999996, 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:18:43.507 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:18:43.510 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:18:43.510 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 115.03565267899999 2024-09-12T03:18:43.510 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:18:43.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 msgr2=0x7f600c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 0x7f600c0a5040 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x559fcdfac2e0 tx=0x7f6018009940 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 msgr2=0x7f602419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f601c00cce0 tx=0x7f601c00a620 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 shutdown_connections 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2b80 0x7f600c0a5040 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6024106f10 0x7f602419cca0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6024102790 0x7f6024198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 --2- 172.21.15.19:0/1536672425 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6024101de0 0x7f60241983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.498+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 >> 172.21.15.19:0/1536672425 conn(0x7f60240fd8c0 msgr2=0x7f60240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.499+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 shutdown_connections 2024-09-12T03:18:43.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:43.499+0000 7f602a5e8640 1 -- 172.21.15.19:0/1536672425 wait complete. 2024-09-12T03:18:44.182 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T03:18:44.182 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:18:44.195 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T03:18:44.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T03:18:44.452 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:44.848 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:44 smithi019 ceph-mon[30953]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- 172.21.15.19:0/3073126826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2718100b10 msgr2=0x7f2718100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 --2- 172.21.15.19:0/3073126826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2718100b10 0x7f2718100f10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2708009900 tx=0x7f270802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- 172.21.15.19:0/3073126826 shutdown_connections 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 --2- 172.21.15.19:0/3073126826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27181026d0 0x7f2718108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 --2- 172.21.15.19:0/3073126826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 0x7f2718102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:44.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 --2- 172.21.15.19:0/3073126826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2718100b10 0x7f2718100f10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- 172.21.15.19:0/3073126826 >> 172.21.15.19:0/3073126826 conn(0x7f27180fc2a0 msgr2=0x7f27180fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- 172.21.15.19:0/3073126826 shutdown_connections 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- 172.21.15.19:0/3073126826 wait complete. 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 Processor -- start 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.931+0000 7f271e796640 1 -- start start 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 0x7f2718197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 0x7f27181984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f271810c840 con 0x7f27181026d0 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f271810c6c0 con 0x7f2718101d10 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271e796640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f271810c9c0 con 0x7f2718100b10 2024-09-12T03:18:44.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56792/0 (socket says 172.21.15.19:56792) 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 -- 172.21.15.19:0/513493084 learned_addr learned my addr 172.21.15.19:0/513493084 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f270ffff640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 0x7f27181984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f2717fff640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 0x7f2718197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 -- 172.21.15.19:0/513493084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 msgr2=0x7f2718197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 0x7f2718197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 -- 172.21.15.19:0/513493084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 msgr2=0x7f27181984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 0x7f27181984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 -- 172.21.15.19:0/513493084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271819fbb0 con 0x7f27181026d0 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f2717fff640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 0x7f2718197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f271cd33640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f271000b970 tx=0x7f271000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:44.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.932+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710019070 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.933+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f271000cb40 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.933+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271819fea0 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.933+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2710007650 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.933+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27181a03d0 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.934+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27181a0660 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.935+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f271001c070 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.935+0000 7f2715ffb640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 0x7f27040a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.936+0000 7f2717fff640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 0x7f27040a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.936+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27100ce550 con 0x7f27181026d0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.936+0000 7f2717fff640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 0x7f27040a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f270802a930 tx=0x7f2708000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:44.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:44.940+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f271009ae70 con 0x7f27181026d0 2024-09-12T03:18:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:44 smithi135 ceph-mon[35474]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:18:45.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.132+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f27181a0830 con 0x7f27040a2bb0 2024-09-12T03:18:45.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.133+0000 7f2715ffb640 1 -- 172.21.15.19:0/513493084 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f27181a0830 con 0x7f27040a2bb0 2024-09-12T03:18:45.133 INFO:teuthology.orchestra.run.smithi019.stdout:Backend: cephadm 2024-09-12T03:18:45.133 INFO:teuthology.orchestra.run.smithi019.stdout:Available: Yes 2024-09-12T03:18:45.133 INFO:teuthology.orchestra.run.smithi019.stdout:Paused: No 2024-09-12T03:18:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 msgr2=0x7f27040a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 0x7f27040a5070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f270802a930 tx=0x7f2708000bc0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 msgr2=0x7f271819f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:45.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f271000b970 tx=0x7f271000be30 comp rx=0 tx=0).stop 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 shutdown_connections 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27040a2bb0 0x7f27040a5070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27181026d0 0x7f271819f540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2718101d10 0x7f27181984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 --2- 172.21.15.19:0/513493084 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2718100b10 0x7f2718197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 >> 172.21.15.19:0/513493084 conn(0x7f27180fc2a0 msgr2=0x7f271806a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 shutdown_connections 2024-09-12T03:18:45.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:45.135+0000 7f271e796640 1 -- 172.21.15.19:0/513493084 wait complete. 2024-09-12T03:18:45.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:44 smithi121 ceph-mon[34963]: pgmap v328: 97 pgs: 97 active+clean; 642 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:18:45.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T03:18:46.035 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:46.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:45 smithi135 ceph-mon[35474]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:46.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:45 smithi121 ceph-mon[34963]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:46.157 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:45 smithi019 ceph-mon[30953]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.530+0000 7f284857e640 1 -- 172.21.15.19:0/3938037200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840107960 msgr2=0x7f2840109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.530+0000 7f284857e640 1 --2- 172.21.15.19:0/3938037200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840107960 0x7f2840109d50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2838009900 tx=0x7f283802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 -- 172.21.15.19:0/3938037200 shutdown_connections 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 --2- 172.21.15.19:0/3938037200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840107960 0x7f2840109d50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 --2- 172.21.15.19:0/3938037200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840105030 0x7f2840107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 --2- 172.21.15.19:0/3938037200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840102700 0x7f2840104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 -- 172.21.15.19:0/3938037200 >> 172.21.15.19:0/3938037200 conn(0x7f28400fc280 msgr2=0x7f28400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 -- 172.21.15.19:0/3938037200 shutdown_connections 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 -- 172.21.15.19:0/3938037200 wait complete. 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 Processor -- start 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.531+0000 7f284857e640 1 -- start start 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840102700 0x7f2840102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840107960 0x7f2840100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d8b0 con 0x7f2840105030 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f284010d730 con 0x7f2840102700 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f284010da30 con 0x7f2840107960 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:46.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56798/0 (socket says 172.21.15.19:56798) 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 -- 172.21.15.19:0/70414376 learned_addr learned my addr 172.21.15.19:0/70414376 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284631a640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840102700 0x7f2840102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2846b1b640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840107960 0x7f2840100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 -- 172.21.15.19:0/70414376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840107960 msgr2=0x7f2840100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840107960 0x7f2840100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 -- 172.21.15.19:0/70414376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840102700 msgr2=0x7f2840102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840102700 0x7f2840102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 -- 172.21.15.19:0/70414376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28401014b0 con 0x7f2840105030 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f2845b19640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f283400d770 tx=0x7f283400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2834017070 con 0x7f2840105030 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2834009c30 con 0x7f2840105030 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28401a1ff0 con 0x7f2840105030 2024-09-12T03:18:46.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f283400f620 con 0x7f2840105030 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.532+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28401a24a0 con 0x7f2840105030 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.533+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2840105410 con 0x7f2840105030 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.534+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f283401a070 con 0x7f2840105030 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.535+0000 7f283f7fe640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 0x7f28280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.535+0000 7f284631a640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 0x7f28280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.535+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f28340cebe0 con 0x7f2840105030 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.536+0000 7f284631a640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 0x7f28280a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2830005c40 tx=0x7f283000a780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:46.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.539+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2834009de0 con 0x7f2840105030 2024-09-12T03:18:46.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.722+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f28400626f0 con 0x7f28280a2b80 2024-09-12T03:18:46.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.735+0000 7f283f7fe640 1 -- 172.21.15.19:0/70414376 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7f28400626f0 con 0x7f28280a2b80 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.a smithi019 running (112s) 31s ago 110s 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 e3086e8d8825 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.b smithi121 running (108s) 41s ago 108s 20.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 02df0f7f297b 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.c smithi135 running (2m) 41s ago 2m 20.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 3ecb57a1b6bd 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.d smithi019 running (2m) 31s ago 2m 16.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 2bbd538c6b87 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.e smithi121 running (2m) 41s ago 2m 16.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 e0ff72c82541 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.f smithi135 running (2m) 41s ago 2m 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 8ec1c147ce0a 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.g smithi019 running (115s) 31s ago 114s 19.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 32c08a4540d5 2024-09-12T03:18:46.736 INFO:teuthology.orchestra.run.smithi019.stdout:mds.h smithi121 running (2m) 41s ago 2m 16.1M - 19.1.1-298-gb53b6d3e 54467abc55f8 a1b14c53624f 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mds.i smithi135 running (118s) 41s ago 118s 28.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 1e37e1e7664a 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mds.j smithi019 running (2m) 31s ago 2m 13.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 3c5242e2b975 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mds.k smithi121 running (2m) 41s ago 2m 20.3M - 19.1.1-298-gb53b6d3e 54467abc55f8 1ff83e0bca6e 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mds.l smithi135 running (2m) 41s ago 2m 17.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 5ebb9a7c6977 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.x smithi019 *:9283,8765 running (9m) 31s ago 9m 508M - 19.1.1-298-gb53b6d3e 54467abc55f8 ec30d3fdcb6f 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.y smithi121 *:8443,8765 running (7m) 41s ago 7m 444M - 19.1.1-298-gb53b6d3e 54467abc55f8 9db0429741c2 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mgr.z smithi135 *:8443,8765 running (7m) 41s ago 7m 443M - 19.1.1-298-gb53b6d3e 54467abc55f8 f0b2323b8e0d 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a smithi019 running (9m) 31s ago 9m 51.3M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 4b7bab35e51b 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b smithi121 running (7m) 41s ago 7m 38.2M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 9d07ff711727 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c smithi135 running (8m) 41s ago 8m 42.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 a1047c0d1c5f 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 smithi019 running (7m) 31s ago 7m 55.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 d0e1b6c98c0e 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 smithi121 running (6m) 41s ago 6m 56.8M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 30fcce2823ad 2024-09-12T03:18:46.737 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 smithi135 running (6m) 41s ago 6m 46.4M 1453M 19.1.1-298-gb53b6d3e 54467abc55f8 52fac6e3f03b 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 smithi019 running (6m) 31s ago 6m 43.1M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 36572ebd5aab 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 smithi121 running (5m) 41s ago 5m 42.7M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 6f6641e7bbe0 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 smithi135 running (5m) 41s ago 5m 43.5M 1453M 19.1.1-298-gb53b6d3e 54467abc55f8 85706fda8d60 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 smithi019 running (4m) 31s ago 4m 41.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 e6dc5fe2895d 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 smithi121 running (4m) 41s ago 4m 54.6M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 d9950e174d18 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 smithi135 running (3m) 41s ago 3m 53.5M 1453M 19.1.1-298-gb53b6d3e 54467abc55f8 973b6e7447c5 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 smithi019 running (3m) 31s ago 3m 42.7M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 a9e1cc426d92 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 smithi121 running (3m) 41s ago 3m 43.9M 4271M 19.1.1-298-gb53b6d3e 54467abc55f8 e6ef3f0975a0 2024-09-12T03:18:46.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 smithi135 running (2m) 41s ago 2m 50.8M 1453M 19.1.1-298-gb53b6d3e 54467abc55f8 583b37a384a5 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 msgr2=0x7f28280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 0x7f28280a5040 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2830005c40 tx=0x7f283000a780 comp rx=0 tx=0).stop 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 msgr2=0x7f28401006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f283400d770 tx=0x7f283400dc30 comp rx=0 tx=0).stop 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 shutdown_connections 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2840107960 0x7f2840100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f28280a2b80 0x7f28280a5040 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2840105030 0x7f28401006b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 --2- 172.21.15.19:0/70414376 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2840102700 0x7f2840102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.737+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 >> 172.21.15.19:0/70414376 conn(0x7f28400fc280 msgr2=0x7f2840102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.738+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 shutdown_connections 2024-09-12T03:18:46.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:46.738+0000 7f284857e640 1 -- 172.21.15.19:0/70414376 wait complete. 2024-09-12T03:18:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:46 smithi135 ceph-mon[35474]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:47.102 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:46 smithi019 ceph-mon[30953]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:47.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:46 smithi121 ceph-mon[34963]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:47.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T03:18:47.729 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:47.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:47 smithi019 ceph-mon[30953]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:48.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:47 smithi135 ceph-mon[35474]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:48.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:47 smithi121 ceph-mon[34963]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 -- 172.21.15.19:0/1135827805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 msgr2=0x7f753c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 --2- 172.21.15.19:0/1135827805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c102000 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02a410 comp rx=0 tx=0).stop 2024-09-12T03:18:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 -- 172.21.15.19:0/1135827805 shutdown_connections 2024-09-12T03:18:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 --2- 172.21.15.19:0/1135827805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c1037c0 0x7f753c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 --2- 172.21.15.19:0/1135827805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c102e00 0x7f753c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 --2- 172.21.15.19:0/1135827805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c102000 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.190+0000 7f7541f3c640 1 -- 172.21.15.19:0/1135827805 >> 172.21.15.19:0/1135827805 conn(0x7f753c0fd390 msgr2=0x7f753c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- 172.21.15.19:0/1135827805 shutdown_connections 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- 172.21.15.19:0/1135827805 wait complete. 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 Processor -- start 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- start start 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c102e00 0x7f753c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 0x7f753c19f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10d8e0 con 0x7f753c101c00 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10d760 con 0x7f753c102e00 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.191+0000 7f7541f3c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f753c10da60 con 0x7f753c1037c0 2024-09-12T03:18:48.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56816/0 (socket says 172.21.15.19:56816) 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 -- 172.21.15.19:0/619090353 learned_addr learned my addr 172.21.15.19:0/619090353 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753affd640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c102e00 0x7f753c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753bfff640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 0x7f753c19f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 -- 172.21.15.19:0/619090353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 msgr2=0x7f753c19f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 0x7f753c19f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 -- 172.21.15.19:0/619090353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c102e00 msgr2=0x7f753c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c102e00 0x7f753c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 -- 172.21.15.19:0/619090353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c19fb90 con 0x7f753c101c00 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753bfff640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 0x7f753c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f753b7fe640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f752c031610 tx=0x7f752c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:48.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c03f030 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c19fe20 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f752c003ce0 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c035070 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.192+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c1a02a0 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.193+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c109070 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.194+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f752c042030 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.195+0000 7f7538ff9640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.195+0000 7f753affd640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.195+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f752c0ef7b0 con 0x7f753c101c00 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.196+0000 7f753affd640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 0x7f75240a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f753c199480 tx=0x7f7530009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:48.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.198+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f752c0bc720 con 0x7f753c101c00 2024-09-12T03:18:48.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.382+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f753c062630 con 0x7f75240a2bb0 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.389+0000 7f7538ff9640 1 -- 172.21.15.19:0/619090353 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f753c062630 con 0x7f75240a2bb0 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stdout:mds.all 12/12 43s ago 2m smithi019=a;smithi019=d;smithi019=g;smithi019=j;smithi121=b;smithi121=e;smithi121=h;smithi121=k;smithi135=c;smithi135=f;smithi135=i;smithi135=l;count:12 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stdout:mgr 3/3 43s ago 7m smithi019=x;smithi121=y;smithi135=z;count:3 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stdout:mon 3/3 43s ago 8m smithi019:172.21.15.19=a;smithi121:172.21.15.121=b;smithi135:172.21.15.135=c;count:3 2024-09-12T03:18:48.389 INFO:teuthology.orchestra.run.smithi019.stdout:osd 12 43s ago - 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 msgr2=0x7f75240a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 0x7f75240a5070 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f753c199480 tx=0x7f7530009110 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 msgr2=0x7f753c197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f752c031610 tx=0x7f752c0043b0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 shutdown_connections 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f753c1037c0 0x7f753c19f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f753c102e00 0x7f753c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75240a2bb0 0x7f75240a5070 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 --2- 172.21.15.19:0/619090353 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f753c101c00 0x7f753c197f60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:48.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 >> 172.21.15.19:0/619090353 conn(0x7f753c0fd390 msgr2=0x7f753c106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 shutdown_connections 2024-09-12T03:18:48.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:48.391+0000 7f7541f3c640 1 -- 172.21.15.19:0/619090353 wait complete. 2024-09-12T03:18:49.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:48 smithi135 ceph-mon[35474]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:49.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:48 smithi019 ceph-mon[30953]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:49.074 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T03:18:49.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:48 smithi121 ceph-mon[34963]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:49.332 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.892+0000 7f6d05ee5640 1 -- 172.21.15.19:0/3858929200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00105030 msgr2=0x7f6d00107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.892+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/3858929200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00105030 0x7f6d00107420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf402a410 comp rx=0 tx=0).stop 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.892+0000 7f6d05ee5640 1 -- 172.21.15.19:0/3858929200 shutdown_connections 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/3858929200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/3858929200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00105030 0x7f6d00107420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/3858929200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00102700 0x7f6d00104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- 172.21.15.19:0/3858929200 >> 172.21.15.19:0/3858929200 conn(0x7f6d000fc280 msgr2=0x7f6d000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- 172.21.15.19:0/3858929200 shutdown_connections 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- 172.21.15.19:0/3858929200 wait complete. 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 Processor -- start 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- start start 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00105030 0x7f6d001006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d8d0 con 0x7f6d00102700 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010d750 con 0x7f6d00105030 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.893+0000 7f6d05ee5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010da50 con 0x7f6d00107960 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56834/0 (socket says 172.21.15.19:56834) 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 -- 172.21.15.19:0/2110141424 learned_addr learned my addr 172.21.15.19:0/2110141424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfffff640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfeffd640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00105030 0x7f6d001006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 -- 172.21.15.19:0/2110141424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 msgr2=0x7f6d00100bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00100bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 -- 172.21.15.19:0/2110141424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00105030 msgr2=0x7f6d001006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00105030 0x7f6d001006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 -- 172.21.15.19:0/2110141424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d001014b0 con 0x7f6d00102700 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfffff640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cff7fe640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000b4a0 tx=0x7f6cf000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0014070 con 0x7f6d00102700 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf0010af0 con 0x7f6d00102700 2024-09-12T03:18:49.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d001a2110 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf0017d10 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.894+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d001a26a0 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.895+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d000fdd70 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.896+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cf0017070 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.897+0000 7f6cfcff9640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.897+0000 7f6cfeffd640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.897+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6cf00ce450 con 0x7f6d00102700 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.898+0000 7f6cfeffd640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 0x7f6ce80a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:49.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:49.902+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cf009b3c0 con 0x7f6d00102700 2024-09-12T03:18:50.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:49 smithi135 ceph-mon[35474]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:50.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.082+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f6d00062630 con 0x7f6ce80a2b80 2024-09-12T03:18:50.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:49 smithi019 ceph-mon[30953]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.084+0000 7f6cfcff9640 1 -- 172.21.15.19:0/2110141424 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f6d00062630 con 0x7f6ce80a2b80 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stdout:HOST ADDR LABELS STATUS 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 172.21.15.19 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stdout:smithi121 172.21.15.121 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stdout:smithi135 172.21.15.135 2024-09-12T03:18:50.084 INFO:teuthology.orchestra.run.smithi019.stdout:3 hosts in cluster 2024-09-12T03:18:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 msgr2=0x7f6ce80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 0x7f6ce80a5040 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 msgr2=0x7f6d00102060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6cf000b4a0 tx=0x7f6cf000b960 comp rx=0 tx=0).stop 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 shutdown_connections 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d00107960 0x7f6d00100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d00105030 0x7f6d001006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ce80a2b80 0x7f6ce80a5040 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 --2- 172.21.15.19:0/2110141424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d00102700 0x7f6d00102060 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 >> 172.21.15.19:0/2110141424 conn(0x7f6d000fc280 msgr2=0x7f6d00103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 shutdown_connections 2024-09-12T03:18:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:50.086+0000 7f6d05ee5640 1 -- 172.21.15.19:0/2110141424 wait complete. 2024-09-12T03:18:50.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:49 smithi121 ceph-mon[34963]: from='client.15069 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:50.891 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T03:18:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:50 smithi135 ceph-mon[35474]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:50 smithi135 ceph-mon[35474]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:51.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:50 smithi121 ceph-mon[34963]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:51.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:50 smithi121 ceph-mon[34963]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:51.156 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:51.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:50 smithi019 ceph-mon[30953]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:18:51.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:50 smithi019 ceph-mon[30953]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:51.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 -- 172.21.15.19:0/1251659307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4102690 msgr2=0x7f8ad4108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 --2- 172.21.15.19:0/1251659307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4102690 0x7f8ad4108f20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009ca0 tx=0x7f8acc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 -- 172.21.15.19:0/1251659307 shutdown_connections 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 --2- 172.21.15.19:0/1251659307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4102690 0x7f8ad4108f20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 --2- 172.21.15.19:0/1251659307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 --2- 172.21.15.19:0/1251659307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 -- 172.21.15.19:0/1251659307 >> 172.21.15.19:0/1251659307 conn(0x7f8ad40fc280 msgr2=0x7f8ad40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 -- 172.21.15.19:0/1251659307 shutdown_connections 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.585+0000 7f8adad99640 1 -- 172.21.15.19:0/1251659307 wait complete. 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 Processor -- start 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 -- start start 2024-09-12T03:18:51.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4102690 0x7f8ad419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c750 con 0x7f8ad4100ad0 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c5d0 con 0x7f8ad4102690 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8adad99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad410c8d0 con 0x7f8ad4101cd0 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39902/0 (socket says 172.21.15.19:39902) 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 -- 172.21.15.19:0/1358258643 learned_addr learned my addr 172.21.15.19:0/1358258643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad9336640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4102690 0x7f8ad419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8acbfff640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 -- 172.21.15.19:0/1358258643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 msgr2=0x7f8ad4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:51.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 -- 172.21.15.19:0/1358258643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4102690 msgr2=0x7f8ad419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4102690 0x7f8ad419f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 -- 172.21.15.19:0/1358258643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad419fb50 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8acbfff640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.586+0000 7f8ad8b35640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000d950 tx=0x7f8ad000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.587+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0017070 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.587+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ad0009c30 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.587+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad419fe40 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.587+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000f600 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.587+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad41a0290 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.588+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad41a04c0 con 0x7f8ad4100ad0 2024-09-12T03:18:51.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.589+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ad001a070 con 0x7f8ad4100ad0 2024-09-12T03:18:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.590+0000 7f8ac9ffb640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 0x7f8abc0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.590+0000 7f8acbfff640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 0x7f8abc0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.590+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8ad00ceba0 con 0x7f8ad4100ad0 2024-09-12T03:18:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.590+0000 7f8acbfff640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 0x7f8abc0a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4005d00 tx=0x7f8ac4009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:51.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.594+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8ad009bab0 con 0x7f8ad4100ad0 2024-09-12T03:18:51.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.777+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f8ad406a710 con 0x7f8abc0a2bb0 2024-09-12T03:18:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.779+0000 7f8ac9ffb640 1 -- 172.21.15.19:0/1358258643 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f8ad406a710 con 0x7f8abc0a2bb0 2024-09-12T03:18:51.779 INFO:teuthology.orchestra.run.smithi019.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T03:18:51.779 INFO:teuthology.orchestra.run.smithi019.stdout:smithi019 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330002G400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:18:51.779 INFO:teuthology.orchestra.run.smithi019.stdout:smithi121 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400ZF400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-12T03:18:51.779 INFO:teuthology.orchestra.run.smithi019.stdout:smithi135 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620401BA400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.781+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 msgr2=0x7f8abc0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.781+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 0x7f8abc0a5070 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4005d00 tx=0x7f8ac4009550 comp rx=0 tx=0).stop 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.781+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 msgr2=0x7f8ad4197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.781+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000d950 tx=0x7f8ad000de10 comp rx=0 tx=0).stop 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 shutdown_connections 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8ad4102690 0x7f8ad419f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8ad4101cd0 0x7f8ad4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8abc0a2bb0 0x7f8abc0a5070 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 --2- 172.21.15.19:0/1358258643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8ad4100ad0 0x7f8ad4197f20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 >> 172.21.15.19:0/1358258643 conn(0x7f8ad40fc280 msgr2=0x7f8ad406a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 shutdown_connections 2024-09-12T03:18:51.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:51.782+0000 7f8adad99640 1 -- 172.21.15.19:0/1358258643 wait complete. 2024-09-12T03:18:52.624 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T03:18:52.637 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T03:18:52.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T03:18:52.900 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:52 smithi019 ceph-mon[30953]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:52 smithi019 ceph-mon[30953]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:52 smithi135 ceph-mon[35474]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:52 smithi135 ceph-mon[35474]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:53.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:52 smithi121 ceph-mon[34963]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:53.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:52 smithi121 ceph-mon[34963]: from='client.15081 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T03:18:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/746773441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 msgr2=0x7f3ea8100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/746773441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea8100ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009980 tx=0x7f3ea402c620 comp rx=0 tx=0).stop 2024-09-12T03:18:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/746773441 shutdown_connections 2024-09-12T03:18:53.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/746773441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8102690 0x7f3ea8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/746773441 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ea8101cd0 0x7f3ea8102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/746773441 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea8100ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/746773441 >> 172.21.15.19:0/746773441 conn(0x7f3ea80fc280 msgr2=0x7f3ea80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/746773441 shutdown_connections 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/746773441 wait complete. 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.420+0000 7f3eaf9c1640 1 Processor -- start 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 -- start start 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8101cd0 0x7f3ea819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ea8102690 0x7f3ea81a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810b6a0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810b520 con 0x7f3ea8101cd0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eaf9c1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea810b820 con 0x7f3ea8102690 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39922/0 (socket says 172.21.15.19:39922) 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 -- 172.21.15.19:0/2109447409 learned_addr learned my addr 172.21.15.19:0/2109447409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3eacf5c640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8101cd0 0x7f3ea819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 -- 172.21.15.19:0/2109447409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ea8102690 msgr2=0x7f3ea81a1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ea8102690 0x7f3ea81a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 -- 172.21.15.19:0/2109447409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8101cd0 msgr2=0x7f3ea819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8101cd0 0x7f3ea819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 -- 172.21.15.19:0/2109447409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea81a1df0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3ead75d640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4033610 tx=0x7f3ea4033b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.421+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4041070 con 0x7f3ea8100ad0 2024-09-12T03:18:53.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.422+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea81a2080 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.422+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ea4003cd0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.422+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4037050 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.422+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea81a2630 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.422+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea8193bb0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.424+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ea4044070 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.424+0000 7f3e9e7fc640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 0x7f3e940a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.424+0000 7f3eacf5c640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 0x7f3e940a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.425+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3ea40efcd0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.425+0000 7f3eacf5c640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 0x7f3e940a5070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ea819b6e0 tx=0x7f3e98008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:53.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.427+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ea40bcc40 con 0x7f3ea8100ad0 2024-09-12T03:18:53.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.640+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3ea802d7c0 con 0x7f3ea8100ad0 2024-09-12T03:18:53.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.641+0000 7f3e9e7fc640 1 -- 172.21.15.19:0/2109447409 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3129 (secure 0 0 0) 0x7f3ea40bc540 con 0x7f3ea8100ad0 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:e27 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:btime 2024-09-12T03:17:21:825879+0000 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:legacy client fscid: 1 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:53.641 INFO:teuthology.orchestra.run.smithi019.stdout:Filesystem 'cephfs' (1) 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:fs_name cephfs 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 27 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-12T03:17:10.527386+0000 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-09-12T03:17:21.825856+0000 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:tableserver 0 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:root 0 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:session_timeout 300 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:session_autoclose 300 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:max_file_size 1099511627776 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:max_xattr_size 65536 2024-09-12T03:18:53.642 INFO:teuthology.orchestra.run.smithi019.stdout:required_client_features {} 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure 0 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:last_failure_osd_epoch 0 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:max_mds 5 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:in 0,1,2,3,4 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:up {0=24487,1=24467,2=24463,3=24421,4=24407} 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:failed 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:damaged 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:stopped 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:data_pools [3] 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:metadata_pool 2 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:inline_data disabled 2024-09-12T03:18:53.643 INFO:teuthology.orchestra.run.smithi019.stdout:balancer 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:bal_rank_mask -1 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:standby_count_wanted 0 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:qdb_cluster leader: 24487 members: 24487,24467,24463,24421,24407 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.i{0:24487} state up:active seq 7 addr [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.f{0:24457} state up:standby-replay seq 1 addr [v2:172.21.15.135:6834/3494935988,v1:172.21.15.135:6835/3494935988] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.b{1:24467} state up:active seq 6 addr [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.h{1:24443} state up:standby-replay seq 1 addr [v2:172.21.15.121:6836/826498871,v1:172.21.15.121:6837/826498871] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.c{2:24463} state up:active seq 10 addr [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.e{2:24422} state up:standby-replay seq 1 addr [v2:172.21.15.121:6834/3114415604,v1:172.21.15.121:6835/3114415604] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.l{3:24421} state up:active seq 15 addr [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.a{3:14589} state up:standby-replay seq 1 addr [v2:172.21.15.19:6840/9500015,v1:172.21.15.19:6841/9500015] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.644 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.k{4:24407} state up:active seq 15 addr [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.g{4:14577} state up:standby-replay seq 1 addr [v2:172.21.15.19:6838/2689902455,v1:172.21.15.19:6839/2689902455] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout:Standby daemons: 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.j{-1:14511} state up:standby seq 1 addr [v2:172.21.15.19:6834/1760992520,v1:172.21.15.19:6835/1760992520] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.645 INFO:teuthology.orchestra.run.smithi019.stdout:[mds.d{-1:14523} state up:standby seq 1 addr [v2:172.21.15.19:6836/2911685711,v1:172.21.15.19:6837/2911685711] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 msgr2=0x7f3e940a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 0x7f3e940a5070 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ea819b6e0 tx=0x7f3e98008040 comp rx=0 tx=0).stop 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 msgr2=0x7f3ea819a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4033610 tx=0x7f3ea4033b60 comp rx=0 tx=0).stop 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 shutdown_connections 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ea8102690 0x7f3ea81a1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ea8101cd0 0x7f3ea819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e940a2bb0 0x7f3e940a5070 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 --2- 172.21.15.19:0/2109447409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ea8100ad0 0x7f3ea819a1c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:53.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 >> 172.21.15.19:0/2109447409 conn(0x7f3ea80fc280 msgr2=0x7f3ea8190d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:53.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 shutdown_connections 2024-09-12T03:18:53.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:53.643+0000 7f3eaf9c1640 1 -- 172.21.15.19:0/2109447409 wait complete. 2024-09-12T03:18:53.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:53 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2109447409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:18:54.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:53 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2109447409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:18:54.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:53 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2109447409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T03:18:54.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T03:18:54.651 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:55.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:54 smithi019 ceph-mon[30953]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:55.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:54 smithi135 ceph-mon[35474]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 -- 172.21.15.19:0/836090325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 msgr2=0x7f2974108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 --2- 172.21.15.19:0/836090325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f2974108f60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f296c009ca0 tx=0x7f296c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 -- 172.21.15.19:0/836090325 shutdown_connections 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 --2- 172.21.15.19:0/836090325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f2974108f60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 --2- 172.21.15.19:0/836090325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974101d10 0x7f2974102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 --2- 172.21.15.19:0/836090325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974100b10 0x7f2974100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 -- 172.21.15.19:0/836090325 >> 172.21.15.19:0/836090325 conn(0x7f29740fc2a0 msgr2=0x7f29740fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 -- 172.21.15.19:0/836090325 shutdown_connections 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.104+0000 7f297b46b640 1 -- 172.21.15.19:0/836090325 wait complete. 2024-09-12T03:18:55.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 Processor -- start 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 -- start start 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974100b10 0x7f297410b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 0x7f2974109ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d880 con 0x7f29741026d0 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d700 con 0x7f2974101d10 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f297b46b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f297410da00 con 0x7f2974100b10 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f2979a08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f2979a08640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39952/0 (socket says 172.21.15.19:39952) 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f2979a08640 1 -- 172.21.15.19:0/1824763724 learned_addr learned my addr 172.21.15.19:0/1824763724 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.105+0000 7f2979207640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974100b10 0x7f297410b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2978a06640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 0x7f2974109ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:55.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 -- 172.21.15.19:0/1824763724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974100b10 msgr2=0x7f297410b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974100b10 0x7f297410b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 -- 172.21.15.19:0/1824763724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 msgr2=0x7f2974109ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 0x7f2974109ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 -- 172.21.15.19:0/1824763724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297410a8e0 con 0x7f29741026d0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2978a06640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 0x7f2974109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f2979a08640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f296c005e30 tx=0x7f296c031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c003c30 con 0x7f29741026d0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f296c003dd0 con 0x7f29741026d0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29741a1fd0 con 0x7f29741026d0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c02fde0 con 0x7f29741026d0 2024-09-12T03:18:55.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.106+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741a2480 con 0x7f29741026d0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.107+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974107ee0 con 0x7f29741026d0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.108+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f296c004050 con 0x7f29741026d0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.109+0000 7f296a7fc640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 0x7f29600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.109+0000 7f2979207640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 0x7f29600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.109+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f296c0f0440 con 0x7f29741026d0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.110+0000 7f2979207640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 0x7f29600a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2974101b70 tx=0x7f297000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:55.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.114+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f296c0bd350 con 0x7f29741026d0 2024-09-12T03:18:55.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:54 smithi121 ceph-mon[34963]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:18:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.290+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2974062e90 con 0x7f29741026d0 2024-09-12T03:18:55.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.291+0000 7f296a7fc640 1 -- 172.21.15.19:0/1824763724 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v82) ==== 56+0+5286 (secure 0 0 0) 0x7f296c0bcc50 con 0x7f29741026d0 2024-09-12T03:18:55.294 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 82 2024-09-12T03:18:55.294 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:18:55.294 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-12T03:09:21.564122+0000 2024-09-12T03:18:55.295 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-09-12T03:18:13.217217+0000 2024-09-12T03:18:55.295 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T03:18:55.295 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 26 2024-09-12T03:18:55.295 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-09-12T03:18:55.295 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client luminous 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-09-12T03:18:55.296 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T03:18:55.297 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T03:18:55.297 INFO:teuthology.orchestra.run.smithi019.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T03:18:55.297 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 12 2024-09-12T03:18:55.297 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 8 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6802/3926650955,v1:172.21.15.19:6803/3926650955] [v2:172.21.15.19:6804/3926650955,v1:172.21.15.19:6805/3926650955] exists,up 3c60c7b8-10fe-4d61-8f59-09d8bb612f89 2024-09-12T03:18:55.298 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 13 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6800/4159912273,v1:172.21.15.121:6801/4159912273] [v2:172.21.15.121:6802/4159912273,v1:172.21.15.121:6803/4159912273] exists,up 963d3ee4-bad4-4896-8b30-38c5d33c0333 2024-09-12T03:18:55.298 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 17 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6800/2430507446,v1:172.21.15.135:6801/2430507446] [v2:172.21.15.135:6802/2430507446,v1:172.21.15.135:6803/2430507446] exists,up aca96f86-8aca-451b-8be2-0d18ed46afae 2024-09-12T03:18:55.298 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 23 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6810/3191237589,v1:172.21.15.19:6811/3191237589] [v2:172.21.15.19:6812/3191237589,v1:172.21.15.19:6813/3191237589] exists,up 4eccbc69-1f18-4b12-be71-e115af1f187f 2024-09-12T03:18:55.298 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 30 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6808/1363833707,v1:172.21.15.121:6809/1363833707] [v2:172.21.15.121:6810/1363833707,v1:172.21.15.121:6811/1363833707] exists,up bf3b091d-b06c-466a-b160-49acb3bb2917 2024-09-12T03:18:55.299 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 35 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6808/3468418134,v1:172.21.15.135:6809/3468418134] [v2:172.21.15.135:6810/3468418134,v1:172.21.15.135:6811/3468418134] exists,up 3ac0c634-1484-497b-bee3-58e37c79b212 2024-09-12T03:18:55.299 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 42 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6818/2104639727,v1:172.21.15.19:6819/2104639727] [v2:172.21.15.19:6820/2104639727,v1:172.21.15.19:6821/2104639727] exists,up 10489254-1ec9-4c5d-acf1-abc03c7e1167 2024-09-12T03:18:55.299 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 47 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6816/1039653795,v1:172.21.15.121:6817/1039653795] [v2:172.21.15.121:6818/1039653795,v1:172.21.15.121:6819/1039653795] exists,up 17cbf654-fee4-4cfd-a5f3-32a8e312fabc 2024-09-12T03:18:55.299 INFO:teuthology.orchestra.run.smithi019.stdout:osd.8 up in weight 1 up_from 54 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6816/693996615,v1:172.21.15.135:6817/693996615] [v2:172.21.15.135:6818/693996615,v1:172.21.15.135:6819/693996615] exists,up d7263722-4663-4725-8f31-833f02d73b8e 2024-09-12T03:18:55.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.9 up in weight 1 up_from 62 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6826/892182666,v1:172.21.15.19:6827/892182666] [v2:172.21.15.19:6828/892182666,v1:172.21.15.19:6829/892182666] exists,up 93a2fba3-453f-42ab-ac18-23faec974ede 2024-09-12T03:18:55.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.121:6824/1462491703,v1:172.21.15.121:6825/1462491703] [v2:172.21.15.121:6826/1462491703,v1:172.21.15.121:6827/1462491703] exists,up 17c4e2a4-2369-4f99-b966-e9deb821b99d 2024-09-12T03:18:55.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.11 up in weight 1 up_from 74 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.135:6824/3648701186,v1:172.21.15.135:6825/3648701186] [v2:172.21.15.135:6826/3648701186,v1:172.21.15.135:6827/3648701186] exists,up a58b445d-ac43-4862-9926-1bcd0a63537d 2024-09-12T03:18:55.300 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.b [2,1] 2024-09-12T03:18:55.300 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.c [2,4] 2024-09-12T03:18:55.301 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.f [8,11] 2024-09-12T03:18:55.301 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 2.1c [8,11] 2024-09-12T03:18:55.301 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.c [3,9] 2024-09-12T03:18:55.301 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.10 [5,9] 2024-09-12T03:18:55.301 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.1d [4,8] 2024-09-12T03:18:55.302 INFO:teuthology.orchestra.run.smithi019.stdout:pg_upmap_items 3.3c [4,9] 2024-09-12T03:18:55.302 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3786063501 expires 2024-09-13T03:10:11.180029+0000 2024-09-12T03:18:55.302 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/262011260 expires 2024-09-13T03:10:11.180029+0000 2024-09-12T03:18:55.302 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/1406182514 expires 2024-09-13T03:10:11.180029+0000 2024-09-12T03:18:55.303 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/1268895810 expires 2024-09-13T03:10:11.180029+0000 2024-09-12T03:18:55.303 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/1268895810 expires 2024-09-13T03:10:11.180029+0000 2024-09-12T03:18:55.303 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/153701601 expires 2024-09-13T03:09:47.038261+0000 2024-09-12T03:18:55.303 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6801/2981478510 expires 2024-09-13T03:09:47.038261+0000 2024-09-12T03:18:55.303 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:6800/2981478510 expires 2024-09-13T03:09:47.038261+0000 2024-09-12T03:18:55.304 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3143338534 expires 2024-09-13T03:09:47.038261+0000 2024-09-12T03:18:55.304 INFO:teuthology.orchestra.run.smithi019.stdout:blocklist 172.21.15.19:0/3087383304 expires 2024-09-13T03:09:47.038261+0000 2024-09-12T03:18:55.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 msgr2=0x7f29600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:55.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 0x7f29600a5040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2974101b70 tx=0x7f297000b390 comp rx=0 tx=0).stop 2024-09-12T03:18:55.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 msgr2=0x7f297410a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:55.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f296c005e30 tx=0x7f296c031ee0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 shutdown_connections 2024-09-12T03:18:55.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f29600a2b80 0x7f29600a5040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f29741026d0 0x7f297410a020 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2974101d10 0x7f2974109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 --2- 172.21.15.19:0/1824763724 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2974100b10 0x7f297410b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 >> 172.21.15.19:0/1824763724 conn(0x7f29740fc2a0 msgr2=0x7f29741050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 shutdown_connections 2024-09-12T03:18:55.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:55.294+0000 7f297b46b640 1 -- 172.21.15.19:0/1824763724 wait complete. 2024-09-12T03:18:55.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:55 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1824763724' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:18:56.002 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T03:18:56.017 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:56.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:55 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1824763724' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:18:56.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:55 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1824763724' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T03:18:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 -- 172.21.15.19:0/4288376056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 msgr2=0x7fb59c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/4288376056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c10bb50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb584009900 tx=0x7fb58402a480 comp rx=0 tx=0).stop 2024-09-12T03:18:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 -- 172.21.15.19:0/4288376056 shutdown_connections 2024-09-12T03:18:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/4288376056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 0x7fb59c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/4288376056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c10bb50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/4288376056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 0x7fb59c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 -- 172.21.15.19:0/4288376056 >> 172.21.15.19:0/4288376056 conn(0x7fb59c0fc7a0 msgr2=0x7fb59c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 -- 172.21.15.19:0/4288376056 shutdown_connections 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.763+0000 7fb5a1c34640 1 -- 172.21.15.19:0/4288376056 wait complete. 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 Processor -- start 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 -- start start 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 0x7fb59c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 0x7fb59c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c111fc0 con 0x7fb59c103660 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c111e40 con 0x7fb59c10c170 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb5a1c34640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb59c112140 con 0x7fb59c102d40 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39958/0 (socket says 172.21.15.19:39958) 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb592ffd640 1 -- 172.21.15.19:0/473584339 learned_addr learned my addr 172.21.15.19:0/473584339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.764+0000 7fb59b7fe640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 0x7fb59c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb59bfff640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 0x7fb59c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 -- 172.21.15.19:0/473584339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 msgr2=0x7fb59c198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 0x7fb59c198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 -- 172.21.15.19:0/473584339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 msgr2=0x7fb59c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 0x7fb59c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 -- 172.21.15.19:0/473584339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb59c19d430 con 0x7fb59c103660 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb592ffd640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb584004770 tx=0x7fb584003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58403f070 con 0x7fb59c103660 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb584031ca0 con 0x7fb59c103660 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb59c19d6c0 con 0x7fb59c103660 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb584035030 con 0x7fb59c103660 2024-09-12T03:18:56.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.765+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb59c1a5000 con 0x7fb59c103660 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.766+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb59c06b070 con 0x7fb59c103660 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.767+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb584042070 con 0x7fb59c103660 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.768+0000 7fb5997fa640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 0x7fb5880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.768+0000 7fb59b7fe640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 0x7fb5880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.768+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb5840ef710 con 0x7fb59c103660 2024-09-12T03:18:56.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.769+0000 7fb59b7fe640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 0x7fb5880a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0058c0 tx=0x7fb58c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:56.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.772+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb5840bbfd0 con 0x7fb59c103660 2024-09-12T03:18:56.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.983+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb59c107210 con 0x7fb59c103660 2024-09-12T03:18:56.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:56 smithi019 ceph-mon[30953]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:18:56.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.984+0000 7fb5997fa640 1 -- 172.21.15.19:0/473584339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7fb5840c02d0 con 0x7fb59c103660 2024-09-12T03:18:56.984 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:56.984 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:18:56.985 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 msgr2=0x7fb5880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 0x7fb5880a5040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb58c0058c0 tx=0x7fb58c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 msgr2=0x7fb59c1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb584004770 tx=0x7fb584003c10 comp rx=0 tx=0).stop 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 shutdown_connections 2024-09-12T03:18:56.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb59c10c170 0x7fb59c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb5880a2b80 0x7fb5880a5040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb59c103660 0x7fb59c1989c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 --2- 172.21.15.19:0/473584339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb59c102d40 0x7fb59c198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 >> 172.21.15.19:0/473584339 conn(0x7fb59c0fc7a0 msgr2=0x7fb59c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 shutdown_connections 2024-09-12T03:18:56.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:56.986+0000 7fb5a1c34640 1 -- 172.21.15.19:0/473584339 wait complete. 2024-09-12T03:18:57.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:56 smithi135 ceph-mon[35474]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:18:57.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:56 smithi121 ceph-mon[34963]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:18:57.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:18:57.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:57 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/473584339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:57.989 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:18:58.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:57 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/473584339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:58.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:57 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/473584339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:58.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 -- 172.21.15.19:0/2527160467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08101de0 msgr2=0x7fef081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/2527160467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08101de0 0x7fef081021c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feef8009900 tx=0x7feef802a410 comp rx=0 tx=0).stop 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 -- 172.21.15.19:0/2527160467 shutdown_connections 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/2527160467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08106f10 0x7fef0810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/2527160467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08102790 0x7fef081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/2527160467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08101de0 0x7fef081021c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 -- 172.21.15.19:0/2527160467 >> 172.21.15.19:0/2527160467 conn(0x7fef080fd8c0 msgr2=0x7fef080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 -- 172.21.15.19:0/2527160467 shutdown_connections 2024-09-12T03:18:58.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.488+0000 7fef0e5fb640 1 -- 172.21.15.19:0/2527160467 wait complete. 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 Processor -- start 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 -- start start 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 0x7fef08198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08106f10 0x7fef0819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fef08111f90 con 0x7fef08102790 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fef08111e10 con 0x7fef08106f10 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.489+0000 7fef0e5fb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fef08112110 con 0x7fef08101de0 2024-09-12T03:18:58.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39968/0 (socket says 172.21.15.19:39968) 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 -- 172.21.15.19:0/1938180729 learned_addr learned my addr 172.21.15.19:0/1938180729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef07fff640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 0x7fef08198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef0cb98640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08106f10 0x7fef0819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 -- 172.21.15.19:0/1938180729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 msgr2=0x7fef08198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 0x7fef08198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 -- 172.21.15.19:0/1938180729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08106f10 msgr2=0x7fef0819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08106f10 0x7fef0819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 -- 172.21.15.19:0/1938180729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0819d430 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef07fff640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 0x7fef08198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef077fe640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feefc00d950 tx=0x7feefc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc017070 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feefc009c30 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef0819d720 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc00f6b0 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.490+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef081a5000 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.491+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef081021c0 con 0x7fef08102790 2024-09-12T03:18:58.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.492+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feefc0109d0 con 0x7fef08102790 2024-09-12T03:18:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.493+0000 7fef057fa640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 0x7feef00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:18:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.493+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feefc0ced30 con 0x7fef08102790 2024-09-12T03:18:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.493+0000 7fef07fff640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 0x7feef00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:18:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.493+0000 7fef07fff640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 0x7feef00a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feef8005ab0 tx=0x7feef8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:18:58.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.497+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7feefc009de0 con 0x7fef08102790 2024-09-12T03:18:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.710+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef0802d7c0 con 0x7fef08102790 2024-09-12T03:18:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.711+0000 7fef057fa640 1 -- 172.21.15.19:0/1938180729 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11974 (secure 0 0 0) 0x7feefc09bc40 con 0x7fef08102790 2024-09-12T03:18:58.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:18:58.712 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":27,"btime":"2024-09-12T03:17:21:825879+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:17:21.825856+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:18:58.713 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 27 2024-09-12T03:18:58.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 msgr2=0x7feef00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 0x7feef00a50c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feef8005ab0 tx=0x7feef8005a20 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 msgr2=0x7fef081989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feefc00d950 tx=0x7feefc00de10 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 shutdown_connections 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fef08106f10 0x7fef0819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feef00a2c00 0x7feef00a50c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fef08102790 0x7fef081989c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 --2- 172.21.15.19:0/1938180729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fef08101de0 0x7fef08198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 >> 172.21.15.19:0/1938180729 conn(0x7fef080fd8c0 msgr2=0x7fef080fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 shutdown_connections 2024-09-12T03:18:58.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:18:58.713+0000 7fef0e5fb640 1 -- 172.21.15.19:0/1938180729 wait complete. 2024-09-12T03:18:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:58 smithi135 ceph-mon[35474]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:18:58 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1938180729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:58 smithi019 ceph-mon[30953]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:59.147 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:18:58 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1938180729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:58 smithi121 ceph-mon[34963]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:18:59.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:18:58 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1938180729' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:18:59.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:18:59.745 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.422+0000 7fc03be80640 1 -- 172.21.15.19:0/2177351039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03410e480 msgr2=0x7fc034110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.422+0000 7fc03be80640 1 --2- 172.21.15.19:0/2177351039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03410e480 0x7fc034110870 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc02c009ca0 tx=0x7fc02c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 -- 172.21.15.19:0/2177351039 shutdown_connections 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 --2- 172.21.15.19:0/2177351039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc03410e480 0x7fc034110870 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 --2- 172.21.15.19:0/2177351039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc034101280 0x7fc03410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 --2- 172.21.15.19:0/2177351039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0341008d0 0x7fc034100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 -- 172.21.15.19:0/2177351039 >> 172.21.15.19:0/2177351039 conn(0x7fc0340fc7a0 msgr2=0x7fc0340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 -- 172.21.15.19:0/2177351039 shutdown_connections 2024-09-12T03:19:00.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 -- 172.21.15.19:0/2177351039 wait complete. 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 Processor -- start 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.423+0000 7fc03be80640 1 -- start start 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 0x7fc03419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03410e480 0x7fc03419eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc034114180 con 0x7fc0341008d0 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc034114000 con 0x7fc034101280 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03be80640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc034114300 con 0x7fc03410e480 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39986/0 (socket says 172.21.15.19:39986) 2024-09-12T03:19:00.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 -- 172.21.15.19:0/2289320881 learned_addr learned my addr 172.21.15.19:0/2289320881 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03941b640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 0x7fc03419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 -- 172.21.15.19:0/2289320881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03410e480 msgr2=0x7fc03419eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03410e480 0x7fc03419eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 -- 172.21.15.19:0/2289320881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 msgr2=0x7fc03419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 0x7fc03419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 -- 172.21.15.19:0/2289320881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc03419f5d0 con 0x7fc0341008d0 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc03941b640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 0x7fc03419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc039c1c640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c920 tx=0x7fc03000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.424+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030007c70 con 0x7fc0341008d0 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.425+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc03419f8c0 con 0x7fc0341008d0 2024-09-12T03:19:00.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.425+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc030007e10 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.425+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc030018890 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.425+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0341a71a0 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.425+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc034100cb0 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.427+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc030018a70 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.427+0000 7fc02affd640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 0x7fc0200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.428+0000 7fc03941b640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 0x7fc0200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.428+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc0300cf230 con 0x7fc0341008d0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.428+0000 7fc03941b640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 0x7fc0200a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc024005fd0 tx=0x7fc0240094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:00.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.432+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc030014030 con 0x7fc0341008d0 2024-09-12T03:19:00.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.607+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc034109080 con 0x7fc0341008d0 2024-09-12T03:19:00.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.609+0000 7fc02affd640 1 -- 172.21.15.19:0/2289320881 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7fc03000de20 con 0x7fc0341008d0 2024-09-12T03:19:00.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:00.610 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:00.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 msgr2=0x7fc0200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 0x7fc0200a5040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc024005fd0 tx=0x7fc0240094d0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 msgr2=0x7fc03419a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc03000c920 tx=0x7fc03000cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 shutdown_connections 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc03410e480 0x7fc03419eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc034101280 0x7fc03419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0200a2b80 0x7fc0200a5040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 --2- 172.21.15.19:0/2289320881 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0341008d0 0x7fc03419a620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 >> 172.21.15.19:0/2289320881 conn(0x7fc0340fc7a0 msgr2=0x7fc0340feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:00.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 shutdown_connections 2024-09-12T03:19:00.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:00.611+0000 7fc03be80640 1 -- 172.21.15.19:0/2289320881 wait complete. 2024-09-12T03:19:00.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:00 smithi019 ceph-mon[30953]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:19:00.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:00 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2289320881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:00 smithi135 ceph-mon[35474]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:19:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:00 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2289320881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:01.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:00 smithi121 ceph-mon[34963]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:19:01.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:00 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2289320881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:01.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:17:10.527386+0000', 'modified': '2024-09-12T03:17:21.825856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24467, 'mds_2': 24463, 'mds_3': 24421, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14577': {'gid': 14577, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/2689902455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 2689902455}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 2689902455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14589': {'gid': 14589, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/9500015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 9500015}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 9500015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.121:6833/3300883065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 3300883065}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 3300883065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.135:6833/990973415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6832', 'nonce': 990973415}, {'type': 'v1', 'addr': '172.21.15.135:6833', 'nonce': 990973415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/3114415604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3114415604}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 3114415604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/826498871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 826498871}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 826498871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3494935988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3494935988}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3494935988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.135:6837/1827195054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 1827195054}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 1827195054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6839/1305484715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 1305484715}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 1305484715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.135:6839/1449185077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 1449185077}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 1449185077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24487, 'qdb_cluster': [24487, 24467, 24463, 24421, 24407]} 2024-09-12T03:19:01.412 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T03:19:01.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-12T03:19:01.673 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:02.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.181+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/135687581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0101de0 msgr2=0x7fc0d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.181+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/135687581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0101de0 0x7fc0d01021c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009ca0 tx=0x7fc0c002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/135687581 shutdown_connections 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/135687581 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0106f10 0x7fc0d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/135687581 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0102790 0x7fc0d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/135687581 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0101de0 0x7fc0d01021c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/135687581 >> 172.21.15.19:0/135687581 conn(0x7fc0d00fd8c0 msgr2=0x7fc0d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/135687581 shutdown_connections 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/135687581 wait complete. 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 Processor -- start 2024-09-12T03:19:02.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.182+0000 7fc0d5a1a640 1 -- start start 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0101de0 0x7fc0d0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 0x7fc0d019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0111f70 con 0x7fc0d0102790 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d0111df0 con 0x7fc0d0101de0 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d5a1a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d01120f0 con 0x7fc0d0106f10 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42820/0 (socket says 172.21.15.19:42820) 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/1821407649 learned_addr learned my addr 172.21.15.19:0/1821407649 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:02.185 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0101de0 0x7fc0d0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 0x7fc0d019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/1821407649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 msgr2=0x7fc0d019cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 0x7fc0d019cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/1821407649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0101de0 msgr2=0x7fc0d0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0101de0 0x7fc0d0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/1821407649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d019d410 con 0x7fc0d0102790 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 0x7fc0d019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d770 tx=0x7fc0c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.183+0000 7fc0d4a18640 1 -- 172.21.15.19:0/1821407649 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4017070 con 0x7fc0d0102790 2024-09-12T03:19:02.186 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.184+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d019d700 con 0x7fc0d0102790 2024-09-12T03:19:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.184+0000 7fc0d4a18640 1 -- 172.21.15.19:0/1821407649 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c4005500 con 0x7fc0d0102790 2024-09-12T03:19:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.184+0000 7fc0d4a18640 1 -- 172.21.15.19:0/1821407649 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004e00 con 0x7fc0d0102790 2024-09-12T03:19:02.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.184+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d01a4fe0 con 0x7fc0d0102790 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 Processor -- start 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 -- start start 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d4a18640 1 -- 172.21.15.19:0/1821407649 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0c4004fa0 con 0x7fc0d0102790 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b8fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b907670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e26b863090 con 0x55e26b907bb0 2024-09-12T03:19:02.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e26b80aa70 con 0x55e26b8fd2f0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.186+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e26b85f050 con 0x55e26b8fc9d0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ceffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b907670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42832/0 (socket says 172.21.15.19:42832) 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b8fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/2661698905 learned_addr learned my addr 172.21.15.19:0/2661698905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/2661698905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 msgr2=0x55e26b907670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b907670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/2661698905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 msgr2=0x55e26b909fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/2661698905 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26b817130 con 0x55e26b8fc9d0 2024-09-12T03:19:02.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0d4a18640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 0x7fc0b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b8fcdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b9a0 tx=0x7fc0c800be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=412db2463970a3b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 0x7fc0b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0d4a18640 1 -- 172.21.15.19:0/1821407649 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc0c40ceac0 con 0x7fc0d0102790 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.187+0000 7fc0b5ffb640 1 -- 172.21.15.19:0/2661698905 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800ddf0 con 0x55e26b8fc9d0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0b5ffb640 1 -- 172.21.15.19:0/2661698905 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8017900 con 0x55e26b8fc9d0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0b5ffb640 1 -- 172.21.15.19:0/2661698905 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c800fa90 con 0x55e26b8fc9d0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2661698905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 msgr2=0x55e26b8fcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b8fcdb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800b9a0 tx=0x7fc0c800be60 comp rx=0 tx=0).stop 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 0x7fc0b80a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006010 tx=0x7fc0c003d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2661698905 shutdown_connections 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b909fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b907670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2661698905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b8fcdb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2661698905 >> 172.21.15.19:0/2661698905 conn(0x55e26b8f8f60 msgr2=0x55e26b8f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2661698905 shutdown_connections 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.188+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2661698905 wait complete. 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 -- start start 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 Processor -- start 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fc9d0 0x55e26b907410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fd2f0 0x55e26b904760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e26b90eb90 con 0x55e26b907bb0 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e26b81df20 con 0x55e26b8fc9d0 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e26b90ed10 con 0x55e26b8fd2f0 2024-09-12T03:19:02.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0d7df3740 10 client.0 _ll_register_callbacks cb 0x55e26b7e7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0cf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42836/0 (socket says 172.21.15.19:42836) 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fd2f0 0x55e26b904760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.189+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/3551614895 learned_addr learned my addr 172.21.15.19:0/3551614895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fc9d0 0x55e26b907410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/3551614895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fd2f0 msgr2=0x55e26b904760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fd2f0 0x55e26b904760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fc9d0 msgr2=0x55e26b907410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fc9d0 0x55e26b907410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 -- 172.21.15.19:0/3551614895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26b5d42b0 con 0x55e26b907bb0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800f630 tx=0x7fc0c8017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17a14c645ee581e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:02.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8027020 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c8017900 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc0c800f790 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0c800f990 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0d7df3740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.190+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e26b91a4e0 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c8017900 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0c8023950 con 0x55e26b907bb0 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 10 client.15123 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 20 client.15123 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b91c700 0x55e26b91db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 4 client.15123 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:02.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26b870600 con 0x55e26b91c700 2024-09-12T03:19:02.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.191+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b91c700 0x55e26b91db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b91c700 0x55e26b91db70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a9620ac3a1dbe1f2 server_cookie=59c7fdaecaa14447 in_seq=0 out_seq=0 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0aeffd640 10 client.15123 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3551614895 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e26b870600 con 0x55e26b91c700 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0aeffd640 10 client.15123 handle_command_reply: tid=0 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0d7df3740 1 client.15123 shutdown 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 msgr2=0x55e26b904ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.192+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800f630 tx=0x7fc0c8017650 comp rx=0 tx=0).stop 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 shutdown_connections 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b907bb0 0x55e26b904ca0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b91c700 0x55e26b91db70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fd2f0 0x55e26b904760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3551614895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fc9d0 0x55e26b907410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 >> 172.21.15.19:0/3551614895 conn(0x55e26b8f8f60 msgr2=0x55e26b8fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0aeffd640 0 client.15123 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 shutdown_connections 2024-09-12T03:19:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3551614895 wait complete. 2024-09-12T03:19:02.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.193+0000 7fc0d7df3740 20 client.15123 trim_cache size 0 max 16384 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.210+0000 7fc0d7df3740 1 Processor -- start 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 -- start start 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b91db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 0x55e26b9019e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91beb0 con 0x55e26b8fd2f0 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91c580 con 0x55e26b907bb0 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91cc50 con 0x55e26b8fc9d0 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42842/0 (socket says 172.21.15.19:42842) 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/2363567424 learned_addr learned my addr 172.21.15.19:0/2363567424 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 0x55e26b9019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b91db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/2363567424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 msgr2=0x55e26b91db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b91db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/2363567424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 msgr2=0x55e26b9019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 0x55e26b9019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 -- 172.21.15.19:0/2363567424 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e26b905280 con 0x55e26b8fd2f0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 0x55e26b9019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.211+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d740 tx=0x7fc0c40d25a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda1c91f9ab3598d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0b57fa640 1 -- 172.21.15.19:0/2363567424 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4020040 con 0x55e26b8fd2f0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0b57fa640 1 -- 172.21.15.19:0/2363567424 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0c40046c0 con 0x55e26b8fd2f0 2024-09-12T03:19:02.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0b57fa640 1 -- 172.21.15.19:0/2363567424 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40a3fa0 con 0x55e26b8fd2f0 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2363567424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 msgr2=0x55e26b9065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d740 tx=0x7fc0c40d25a0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2363567424 shutdown_connections 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b907bb0 0x55e26b9019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fd2f0 0x55e26b9065c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/2363567424 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b8fc9d0 0x55e26b91db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2363567424 >> 172.21.15.19:0/2363567424 conn(0x55e26b8f8f60 msgr2=0x55e26b905eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2363567424 shutdown_connections 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.212+0000 7fc0d7df3740 1 -- 172.21.15.19:0/2363567424 wait complete. 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 -- start start 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 Processor -- start 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b9266e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b907bb0 0x55e26b921720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91d9f0 con 0x55e26b8fc9d0 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91d320 con 0x55e26b8fd2f0 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e26b91e0c0 con 0x55e26b907bb0 2024-09-12T03:19:02.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0d7df3740 10 client.0 _ll_register_callbacks cb 0x7fc0c40d2470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0ceffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42856/0 (socket says 172.21.15.19:42856) 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.213+0000 7fc0ceffd640 1 -- 172.21.15.19:0/3547573098 learned_addr learned my addr 172.21.15.19:0/3547573098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0cf7fe640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b907bb0 0x55e26b921720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 -- 172.21.15.19:0/3547573098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b907bb0 msgr2=0x55e26b921720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b907bb0 0x55e26b921720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b9266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 -- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 msgr2=0x55e26b9266e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b9266e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 -- 172.21.15.19:0/3547573098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e26b921f50 con 0x55e26b8fc9d0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0ceffd640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009bf0 tx=0x7fc0c002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d488d55574c5d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0057070 con 0x55e26b8fc9d0 2024-09-12T03:19:02.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc0c0053430 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e26b86ab10 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc0c0053630 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0d7df3740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e26b986ab0 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.214+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e26b86ab10 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc0c00309e0 con 0x55e26b8fc9d0 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 10 client.15129 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 20 client.15129 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b98b6f0 0x55e26b98a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:02.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 4 client.15129 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "status"}) -- 0x55e26b7e70e0 con 0x55e26b98b6f0 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b98b6f0 0x55e26b98a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0aeffd640 10 client.15129 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0ce7fc640 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b98b6f0 0x55e26b98a400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=abc7d738eaeaa8d server_cookie=3656ca769eb1e4ca in_seq=0 out_seq=0 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0aeffd640 1 -- 172.21.15.19:0/3547573098 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55e26b7e70e0 con 0x55e26b98b6f0 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.215+0000 7fc0aeffd640 10 client.15129 handle_command_reply: tid=0 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 client.15129 shutdown 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 msgr2=0x55e26b9261a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0009bf0 tx=0x7fc0c002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 shutdown_connections 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55e26b98b6f0 0x55e26b98a400 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e26b907bb0 0x55e26b921720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e26b8fd2f0 0x55e26b9266e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0aeffd640 0 client.15129 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 --2- 172.21.15.19:0/3547573098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e26b8fc9d0 0x55e26b9261a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 >> 172.21.15.19:0/3547573098 conn(0x55e26b8f8f60 msgr2=0x55e26b8f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 shutdown_connections 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 1 -- 172.21.15.19:0/3547573098 wait complete. 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d7df3740 20 client.15129 trim_cache size 0 max 16384 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-12T03:19:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24407, 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 101.394308388, 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 154.08130842 2024-09-12T03:19:02.223 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 msgr2=0x7fc0b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 0x7fc0b80a5040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0006010 tx=0x7fc0c003d040 comp rx=0 tx=0).stop 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 msgr2=0x7fc0d01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.216+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d770 tx=0x7fc0c400dc30 comp rx=0 tx=0).stop 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 shutdown_connections 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc0d0106f10 0x7fc0d019cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0b80a2b80 0x7fc0b80a5040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0d0102790 0x7fc0d01989a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 --2- 172.21.15.19:0/1821407649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0d0101de0 0x7fc0d0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 >> 172.21.15.19:0/1821407649 conn(0x7fc0d00fd8c0 msgr2=0x7fc0d00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 shutdown_connections 2024-09-12T03:19:02.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:02.217+0000 7fc0d5a1a640 1 -- 172.21.15.19:0/1821407649 wait complete. 2024-09-12T03:19:03.005 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:02 smithi019 ceph-mon[30953]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:19:03.038 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-12T03:19:03.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:02 smithi135 ceph-mon[35474]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:19:03.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:02 smithi121 ceph-mon[34963]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:19:03.300 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 -- 172.21.15.19:0/1358386698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc1036c0 msgr2=0x7f15bc10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 --2- 172.21.15.19:0/1358386698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc1036c0 0x7f15bc10bbb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f15b0009900 tx=0x7f15b002a480 comp rx=0 tx=0).stop 2024-09-12T03:19:03.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 -- 172.21.15.19:0/1358386698 shutdown_connections 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 --2- 172.21.15.19:0/1358386698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 --2- 172.21.15.19:0/1358386698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc1036c0 0x7f15bc10bbb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 --2- 172.21.15.19:0/1358386698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc102da0 0x7f15bc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 -- 172.21.15.19:0/1358386698 >> 172.21.15.19:0/1358386698 conn(0x7f15bc0fc7a0 msgr2=0x7f15bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 -- 172.21.15.19:0/1358386698 shutdown_connections 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.793+0000 7f15c21a6640 1 -- 172.21.15.19:0/1358386698 wait complete. 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 Processor -- start 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 -- start start 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc1036c0 0x7f15bc198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc112020 con 0x7f15bc102da0 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc111ea0 con 0x7f15bc1036c0 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15c21a6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1121a0 con 0x7f15bc10c180 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42870/0 (socket says 172.21.15.19:42870) 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15bb7fe640 1 -- 172.21.15.19:0/2210325009 learned_addr learned my addr 172.21.15.19:0/2210325009 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.794+0000 7f15bbfff640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 -- 172.21.15.19:0/2210325009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 msgr2=0x7f15bc19ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc19ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 -- 172.21.15.19:0/2210325009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc1036c0 msgr2=0x7f15bc198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15baffd640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc1036c0 0x7f15bc198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc1036c0 0x7f15bc198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 -- 172.21.15.19:0/2210325009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19d550 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bbfff640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00b4a0 tx=0x7f15ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15b8ff9640 1 -- 172.21.15.19:0/2210325009 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac014070 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19d840 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15b8ff9640 1 -- 172.21.15.19:0/2210325009 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac010af0 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15b8ff9640 1 -- 172.21.15.19:0/2210325009 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017d10 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.795+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a5120 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.797+0000 7f15b8ff9640 1 -- 172.21.15.19:0/2210325009 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15bc102da0 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 Processor -- start 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 -- start start 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15b8ff9640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 0x7f15a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a62924ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a6292513b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6291a6090 con 0x55a62923f8b0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a62914da70 con 0x55a62924efc0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15c457f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6291a2050 con 0x55a6292401d0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bbfff640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 0x7f15a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42880/0 (socket says 172.21.15.19:42880) 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 -- 172.21.15.19:0/693316948 learned_addr learned my addr 172.21.15.19:0/693316948 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15b8ff9640 1 -- 172.21.15.19:0/2210325009 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f15ac0ce510 con 0x7f15bc102da0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15baffd640 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a62924ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bbfff640 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a6292513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 -- 172.21.15.19:0/693316948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 msgr2=0x55a62924ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a62924ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 -- 172.21.15.19:0/693316948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 msgr2=0x55a6292513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a6292513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.798+0000 7f15bb7fe640 1 -- 172.21.15.19:0/693316948 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62915a130 con 0x55a62923f8b0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15bbfff640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 0x7f15a40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15b40136c0 tx=0x7f15b4013ac0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15ac097180 tx=0x7f15ac017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10b8b9795c8a9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15a1ffb640 1 -- 172.21.15.19:0/693316948 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0b3250 con 0x55a62923f8b0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15a1ffb640 1 -- 172.21.15.19:0/693316948 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac09b420 con 0x55a62923f8b0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15a1ffb640 1 -- 172.21.15.19:0/693316948 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac010ca0 con 0x55a62923f8b0 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15c457f740 1 -- 172.21.15.19:0/693316948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 msgr2=0x55a62923fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15c457f740 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f15ac097180 tx=0x7f15ac017730 comp rx=0 tx=0).stop 2024-09-12T03:19:03.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15c457f740 1 -- 172.21.15.19:0/693316948 shutdown_connections 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.799+0000 7f15c457f740 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a6292513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a62924ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 --2- 172.21.15.19:0/693316948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a62923fc90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- 172.21.15.19:0/693316948 >> 172.21.15.19:0/693316948 conn(0x55a62923bf60 msgr2=0x55a62923c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- 172.21.15.19:0/693316948 shutdown_connections 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- 172.21.15.19:0/693316948 wait complete. 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- start start 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 Processor -- start 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a62924e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a62924c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a629251b90 con 0x55a6292401d0 2024-09-12T03:19:03.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a629160f20 con 0x55a62924efc0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a629251d10 con 0x55a62923f8b0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15c457f740 10 client.0 _ll_register_callbacks cb 0x55a62912a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.800+0000 7f15baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42884/0 (socket says 172.21.15.19:42884) 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 -- 172.21.15.19:0/3457451112 learned_addr learned my addr 172.21.15.19:0/3457451112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a62924e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15bbfff640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a62924c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 -- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 msgr2=0x55a62924e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a62924e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 -- 172.21.15.19:0/3457451112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 msgr2=0x55a62924c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a62924c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 -- 172.21.15.19:0/3457451112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a628f172b0 con 0x55a6292401d0 2024-09-12T03:19:03.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15bbfff640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a62924c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15baffd640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15b0031550 tx=0x7f15b0031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a43577acd3d0036 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f159affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b003f070 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f15b0004450 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac09b420 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15b0035070 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15c457f740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.801+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a62925d520 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac09b420 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15b0004bc0 con 0x55a6292401d0 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 10 client.15141 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:19:03.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 20 client.15141 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a62925fb80 0x55a629260ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 4 client.15141 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6291b3600 con 0x55a62925fb80 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.802+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a62925fb80 0x55a629260ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.803+0000 7f159affd640 10 client.15141 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.803+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a62925fb80 0x55a629260ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c5e33bb47b4f9fc server_cookie=174aaaf5f111a6a7 in_seq=0 out_seq=0 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.803+0000 7f159affd640 1 -- 172.21.15.19:0/3457451112 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6291b3600 con 0x55a62925fb80 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.803+0000 7f159affd640 10 client.15141 handle_command_reply: tid=0 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.803+0000 7f15c457f740 1 client.15141 shutdown 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 msgr2=0x55a62924bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f15b0031550 tx=0x7f15b0031860 comp rx=0 tx=0).stop 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 shutdown_connections 2024-09-12T03:19:03.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a62925fb80 0x55a629260ff0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a62924c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6292401d0 0x55a62924bbd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 --2- 172.21.15.19:0/3457451112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a62924e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 >> 172.21.15.19:0/3457451112 conn(0x55a62923bf60 msgr2=0x55a629241cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f159affd640 0 client.15141 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 shutdown_connections 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 1 -- 172.21.15.19:0/3457451112 wait complete. 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.804+0000 7f15c457f740 20 client.15141 trim_cache size 0 max 16384 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.821+0000 7f15c457f740 1 Processor -- start 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.821+0000 7f15c457f740 1 -- start start 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a629248e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a629249370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a62925f330 con 0x55a62923f8b0 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a62925fa00 con 0x55a62924efc0 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15c457f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6292600d0 con 0x55a6292401d0 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42888/0 (socket says 172.21.15.19:42888) 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 -- 172.21.15.19:0/310692296 learned_addr learned my addr 172.21.15.19:0/310692296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 -- 172.21.15.19:0/310692296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 msgr2=0x55a629248e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a629248e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 -- 172.21.15.19:0/310692296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 msgr2=0x55a629249370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a629249370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 -- 172.21.15.19:0/310692296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62924a120 con 0x55a62923f8b0 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00be60 tx=0x7f15ac0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ce0fe6f5b631f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15a17fa640 1 -- 172.21.15.19:0/310692296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac01e040 con 0x55a62923f8b0 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15a17fa640 1 -- 172.21.15.19:0/310692296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac09ad20 con 0x55a62923f8b0 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.822+0000 7f15a17fa640 1 -- 172.21.15.19:0/310692296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac097390 con 0x55a62923f8b0 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- 172.21.15.19:0/310692296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 msgr2=0x55a629260250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00be60 tx=0x7f15ac0047d0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- 172.21.15.19:0/310692296 shutdown_connections 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a62924efc0 0x55a629249370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6292401d0 0x55a629248e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 --2- 172.21.15.19:0/310692296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62923f8b0 0x55a629260250 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- 172.21.15.19:0/310692296 >> 172.21.15.19:0/310692296 conn(0x55a62923bf60 msgr2=0x55a62924a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- 172.21.15.19:0/310692296 shutdown_connections 2024-09-12T03:19:03.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- 172.21.15.19:0/310692296 wait complete. 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 -- start start 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.823+0000 7f15c457f740 1 Processor -- start 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a629262750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 0x55a629262c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a629260e70 con 0x55a62924efc0 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6292607a0 con 0x55a6292401d0 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a629261540 con 0x55a62923f8b0 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 10 client.0 _ll_register_callbacks cb 0x7f15b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42894/0 (socket says 172.21.15.19:42894) 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 -- 172.21.15.19:0/3960217818 learned_addr learned my addr 172.21.15.19:0/3960217818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:03.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a629262750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15baffd640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 0x55a629262c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 -- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 msgr2=0x55a629262750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a629262750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 -- 172.21.15.19:0/3960217818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 msgr2=0x55a629262c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 0x55a629262c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 -- 172.21.15.19:0/3960217818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6292c87c0 con 0x55a62924efc0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15baffd640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 0x55a629262c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15bbfff640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a62923d2e0 tx=0x7f15b4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd1b9aaa6a59f6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f159affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b4025070 con 0x55a62924efc0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f15b401f470 con 0x55a62924efc0 2024-09-12T03:19:03.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.824+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15ac09ad20 con 0x55a62924efc0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f15b4031050 con 0x55a62924efc0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a6292c99f0 con 0x55a62924efc0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15ac09ad20 con 0x55a62924efc0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f15b401d040 con 0x55a62924efc0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 10 client.15147 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 20 client.15147 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a6292ce590 0x55a6292cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 4 client.15147 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "status"}) -- 0x55a6291adb10 con 0x55a6292ce590 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.825+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a6292ce590 0x55a6292cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:03.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f159affd640 10 client.15147 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15bb7fe640 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a6292ce590 0x55a6292cd2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=209e2baede1390b2 server_cookie=e0f77558a897edfa in_seq=0 out_seq=0 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f159affd640 1 -- 172.21.15.19:0/3960217818 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x7f15ac0b8150 con 0x55a6292ce590 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f159affd640 10 client.15147 handle_command_reply: tid=0 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 client.15147 shutdown 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 msgr2=0x55a6292c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55a62923d2e0 tx=0x7f15b4004b50 comp rx=0 tx=0).stop 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:03.850 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24421, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 104.01133697, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 158.36882242799999 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:03.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 shutdown_connections 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a6292ce590 0x55a6292cd2a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a62924efc0 0x55a6292c8280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6292401d0 0x55a629262c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 --2- 172.21.15.19:0/3960217818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a62923f8b0 0x55a629262750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f159affd640 0 client.15147 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 >> 172.21.15.19:0/3960217818 conn(0x55a62923bf60 msgr2=0x55a629246a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 shutdown_connections 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 1 -- 172.21.15.19:0/3960217818 wait complete. 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.826+0000 7f15c457f740 20 client.15147 trim_cache size 0 max 16384 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 msgr2=0x7f15a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 0x7f15a40a5040 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f15b40136c0 tx=0x7f15b4013ac0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 msgr2=0x7f15bc1985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00b4a0 tx=0x7f15ac00b960 comp rx=0 tx=0).stop 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 shutdown_connections 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f15bc10c180 0x7f15bc19ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f15bc1036c0 0x7f15bc198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f15a40a2b80 0x7f15a40a5040 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 --2- 172.21.15.19:0/2210325009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f15bc102da0 0x7f15bc1985a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 >> 172.21.15.19:0/2210325009 conn(0x7f15bc0fc7a0 msgr2=0x7f15bc0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 shutdown_connections 2024-09-12T03:19:03.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:03.827+0000 7f15c21a6640 1 -- 172.21.15.19:0/2210325009 wait complete. 2024-09-12T03:19:04.670 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T03:19:04.929 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:04 smithi135 ceph-mon[35474]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:05.050 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:04 smithi019 ceph-mon[30953]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:05.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:04 smithi121 ceph-mon[34963]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:05.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 -- 172.21.15.19:0/1016957465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464100e20 msgr2=0x7f0464101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 --2- 172.21.15.19:0/1016957465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464100e20 0x7f0464101200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0460009900 tx=0x7f046002a480 comp rx=0 tx=0).stop 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 -- 172.21.15.19:0/1016957465 shutdown_connections 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 --2- 172.21.15.19:0/1016957465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464102ef0 0x7f046410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 --2- 172.21.15.19:0/1016957465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f04641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 --2- 172.21.15.19:0/1016957465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464100e20 0x7f0464101200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 -- 172.21.15.19:0/1016957465 >> 172.21.15.19:0/1016957465 conn(0x7f04640fc7d0 msgr2=0x7f04640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 -- 172.21.15.19:0/1016957465 shutdown_connections 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.378+0000 7f046ad18640 1 -- 172.21.15.19:0/1016957465 wait complete. 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 Processor -- start 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 -- start start 2024-09-12T03:19:05.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 0x7f0464199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f0464199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0464110fa0 con 0x7f0464102ef0 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0464110e20 con 0x7f0464100e20 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f046ad18640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0464111120 con 0x7f0464102570 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42900/0 (socket says 172.21.15.19:42900) 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 -- 172.21.15.19:0/1429986586 learned_addr learned my addr 172.21.15.19:0/1429986586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f0468ab4640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 0x7f0464199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f045bfff640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f0464199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 -- 172.21.15.19:0/1429986586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 msgr2=0x7f0464199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f0464199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 -- 172.21.15.19:0/1429986586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 msgr2=0x7f0464199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 0x7f0464199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f04692b5640 1 -- 172.21.15.19:0/1429986586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04641a49f0 con 0x7f0464102ef0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f0468ab4640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 0x7f0464199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.379+0000 7f045bfff640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f0464199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f04692b5640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f045c00eaa0 tx=0x7f045c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f0459ffb640 1 -- 172.21.15.19:0/1429986586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c00cdb0 con 0x7f0464102ef0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04641a4bc0 con 0x7f0464102ef0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f0459ffb640 1 -- 172.21.15.19:0/1429986586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045c005020 con 0x7f0464102ef0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f0459ffb640 1 -- 172.21.15.19:0/1429986586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c0116b0 con 0x7f0464102ef0 2024-09-12T03:19:05.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.380+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04641a50d0 con 0x7f0464102ef0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.382+0000 7f0459ffb640 1 -- 172.21.15.19:0/1429986586 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f045c005560 con 0x7f0464102ef0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.382+0000 7f046d0f1740 1 Processor -- start 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.382+0000 7f046d0f1740 1 -- start start 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549b2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d95490e090 con 0x55d9549a79d0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9548b5a70 con 0x55d9549a82f0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f046d0f1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d95490a050 con 0x55d9549b2bb0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f0459ffb640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 0x7f044c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42916/0 (socket says 172.21.15.19:42916) 2024-09-12T03:19:05.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 -- 172.21.15.19:0/4171665744 learned_addr learned my addr 172.21.15.19:0/4171665744 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f0468ab4640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f04692b5640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 0x7f044c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f0459ffb640 1 -- 172.21.15.19:0/1429986586 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f045c0cf330 con 0x7f0464102ef0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 -- 172.21.15.19:0/4171665744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 msgr2=0x55d9549b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 -- 172.21.15.19:0/4171665744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 msgr2=0x55d9549b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 -- 172.21.15.19:0/4171665744 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9548c2130 con 0x55d9549a79d0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f0468ab4640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f045bfff640 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f045400af10 tx=0x7f045400a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da633158e2f03f3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.383+0000 7f04692b5640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 0x7f044c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f045c09bb30 tx=0x7f045c06ec60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f044affd640 1 -- 172.21.15.19:0/4171665744 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045401b070 con 0x55d9549a79d0 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f044affd640 1 -- 172.21.15.19:0/4171665744 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f045400ee40 con 0x55d9549a79d0 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f044affd640 1 -- 172.21.15.19:0/4171665744 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400f040 con 0x55d9549a79d0 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 -- 172.21.15.19:0/4171665744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 msgr2=0x55d9549a7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f045400af10 tx=0x7f045400a410 comp rx=0 tx=0).stop 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 -- 172.21.15.19:0/4171665744 shutdown_connections 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 --2- 172.21.15.19:0/4171665744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549a7db0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 -- 172.21.15.19:0/4171665744 >> 172.21.15.19:0/4171665744 conn(0x55d9549a3f60 msgr2=0x55d9549a4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 -- 172.21.15.19:0/4171665744 shutdown_connections 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.384+0000 7f046d0f1740 1 -- 172.21.15.19:0/4171665744 wait complete. 2024-09-12T03:19:05.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 -- start start 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 Processor -- start 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a82f0 0x55d9549af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d9549b5780 con 0x55d9549a79d0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9548c8f20 con 0x55d9549b2bb0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9549b5900 con 0x55d9549a82f0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f046d0f1740 10 client.0 _ll_register_callbacks cb 0x55d954892030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42930/0 (socket says 172.21.15.19:42930) 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.385+0000 7f045bfff640 1 -- 172.21.15.19:0/232982402 learned_addr learned my addr 172.21.15.19:0/232982402 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0468ab4640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a82f0 0x55d9549af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 -- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a82f0 msgr2=0x55d9549af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a82f0 0x55d9549af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 -- 172.21.15.19:0/232982402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 msgr2=0x55d9549afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 -- 172.21.15.19:0/232982402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d95467f2b0 con 0x55d9549a79d0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f045bfff640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0454013480 tx=0x7f0454004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58e02222627d1d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0443fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454023070 con 0x55d9549a79d0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0454016920 con 0x55d9549a79d0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d95491b600 con 0x55d9549a79d0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f045401b040 con 0x55d9549a79d0 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f046d0f1740 10 client.15159 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:05.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d9549c1100 con 0x55d9549a79d0 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.386+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d95491b600 con 0x55d9549a79d0 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f045401e050 con 0x55d9549a79d0 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 10 client.15159 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 10 client.15159 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 20 client.15159 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d9549c3440 0x55d9549c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 4 client.15159 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d954915b10 con 0x55d9549c3440 2024-09-12T03:19:05.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.387+0000 7f0468ab4640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d9549c3440 0x55d9549c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.388+0000 7f0468ab4640 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d9549c3440 0x55d9549c48b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3c73befab464b73 server_cookie=3fc8f7f3c6fa4d42 in_seq=0 out_seq=0 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.388+0000 7f0443fff640 10 client.15159 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.388+0000 7f0443fff640 1 -- 172.21.15.19:0/232982402 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d954915b10 con 0x55d9549c3440 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.388+0000 7f0443fff640 10 client.15159 handle_command_reply: tid=0 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.388+0000 7f046d0f1740 1 client.15159 shutdown 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 msgr2=0x55d9549b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0454013480 tx=0x7f0454004660 comp rx=0 tx=0).stop 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 shutdown_connections 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d9549c3440 0x55d9549c48b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a82f0 0x55d9549af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f0443fff640 0 client.15159 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 --2- 172.21.15.19:0/232982402 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549b2480 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 >> 172.21.15.19:0/232982402 conn(0x55d9549a3f60 msgr2=0x55d9549a9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 shutdown_connections 2024-09-12T03:19:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 1 -- 172.21.15.19:0/232982402 wait complete. 2024-09-12T03:19:05.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.389+0000 7f046d0f1740 20 client.15159 trim_cache size 0 max 16384 2024-09-12T03:19:05.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.408+0000 7f046d0f1740 1 Processor -- start 2024-09-12T03:19:05.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.408+0000 7f046d0f1740 1 -- start start 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a79d0 0x55d9549c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c2bf0 con 0x55d9549a82f0 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c32c0 con 0x55d9549b2bb0 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f046d0f1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c3990 con 0x55d9549a79d0 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42946/0 (socket says 172.21.15.19:42946) 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 -- 172.21.15.19:0/3044642436 learned_addr learned my addr 172.21.15.19:0/3044642436 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f045bfff640 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 -- 172.21.15.19:0/3044642436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a79d0 msgr2=0x55d9549c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a79d0 0x55d9549c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 -- 172.21.15.19:0/3044642436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 msgr2=0x55d9549ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549ae0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 -- 172.21.15.19:0/3044642436 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9549add20 con 0x55d9549a82f0 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f0468ab4640 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0460009230 tx=0x7f0460031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70cef86f84f69ab0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f044a7fc640 1 -- 172.21.15.19:0/3044642436 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460035040 con 0x55d9549a82f0 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f044a7fc640 1 -- 172.21.15.19:0/3044642436 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460049070 con 0x55d9549a82f0 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.409+0000 7f044a7fc640 1 -- 172.21.15.19:0/3044642436 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600035f0 con 0x55d9549a82f0 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- 172.21.15.19:0/3044642436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 msgr2=0x55d95499ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0460009230 tx=0x7f0460031400 comp rx=0 tx=0).stop 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- 172.21.15.19:0/3044642436 shutdown_connections 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549b2bb0 0x55d9549ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a82f0 0x55d95499ece0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 --2- 172.21.15.19:0/3044642436 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549a79d0 0x55d9549c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- 172.21.15.19:0/3044642436 >> 172.21.15.19:0/3044642436 conn(0x55d9549a3f60 msgr2=0x55d9549af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- 172.21.15.19:0/3044642436 shutdown_connections 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- 172.21.15.19:0/3044642436 wait complete. 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.410+0000 7f046d0f1740 1 -- start start 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 Processor -- start 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c4730 con 0x55d9549a79d0 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c4060 con 0x55d9549a82f0 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d9549c4e00 con 0x55d9549b2bb0 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f046d0f1740 10 client.0 _ll_register_callbacks cb 0x7f0454019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:05.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42950/0 (socket says 172.21.15.19:42950) 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 -- 172.21.15.19:0/2171753293 learned_addr learned my addr 172.21.15.19:0/2171753293 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f0468ab4640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 -- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 msgr2=0x55d9549c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 -- 172.21.15.19:0/2171753293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 msgr2=0x55d9549c82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549c82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f045bfff640 1 -- 172.21.15.19:0/2171753293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d954a2be10 con 0x55d9549a79d0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.411+0000 7f0468ab4640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f045bfff640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0454006780 tx=0x7f0454013410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7711f013990b2284 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:05.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454023040 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f045400a4c0 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0460049070 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f045400ad60 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 10 client.15165 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d954a2d110 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0460049070 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f045401e040 con 0x55d9549a79d0 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 10 client.15165 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 10 client.15165 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 20 client.15165 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d954a31c20 0x55d954a30930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 4 client.15165 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:05.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.412+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "status"}) -- 0x55d9548920e0 con 0x55d954a31c20 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0468ab4640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d954a31c20 0x55d954a30930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0443fff640 10 client.15165 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0468ab4640 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d954a31c20 0x55d954a30930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e9f6cd1cec7488b server_cookie=792e9781df011620 in_seq=0 out_seq=0 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0443fff640 1 -- 172.21.15.19:0/2171753293 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55d9548920e0 con 0x55d954a31c20 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0443fff640 10 client.15165 handle_command_reply: tid=0 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 client.15165 shutdown 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 msgr2=0x55d9549c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0454006780 tx=0x7f0454013410 comp rx=0 tx=0).stop 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 shutdown_connections 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55d954a31c20 0x55d954a30930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d9549b2bb0 0x55d9549c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d9549a82f0 0x55d9549c82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 --2- 172.21.15.19:0/2171753293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d9549a79d0 0x55d9549c7d90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 >> 172.21.15.19:0/2171753293 conn(0x55d9549a3f60 msgr2=0x55d9549ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.413+0000 7f0443fff640 0 client.15165 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 shutdown_connections 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046d0f1740 1 -- 172.21.15.19:0/2171753293 wait complete. 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046d0f1740 20 client.15165 trim_cache size 0 max 16384 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-12T03:19:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24463, 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 106.457394967, 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 142.405549052 2024-09-12T03:19:05.425 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 msgr2=0x7f044c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 0x7f044c0a5040 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f045c09bb30 tx=0x7f045c06ec60 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 msgr2=0x7f04641a44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f045c00eaa0 tx=0x7f045c00ef60 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 shutdown_connections 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f044c0a2b80 0x7f044c0a5040 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0464102ef0 0x7f04641a44b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0464102570 0x7f0464199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 --2- 172.21.15.19:0/1429986586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0464100e20 0x7f0464199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.414+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 >> 172.21.15.19:0/1429986586 conn(0x7f04640fc7d0 msgr2=0x7f04640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:05.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.415+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 shutdown_connections 2024-09-12T03:19:05.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:05.415+0000 7f046ad18640 1 -- 172.21.15.19:0/1429986586 wait complete. 2024-09-12T03:19:06.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:19:06.491 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:06.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:06 smithi019 ceph-mon[30953]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:07.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:06 smithi135 ceph-mon[35474]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 -- 172.21.15.19:0/2417831312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 msgr2=0x7f5854103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 --2- 172.21.15.19:0/2417831312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f5854103120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5850009900 tx=0x7f585002a480 comp rx=0 tx=0).stop 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 -- 172.21.15.19:0/2417831312 shutdown_connections 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 --2- 172.21.15.19:0/2417831312 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f585410c170 0x7f585410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 --2- 172.21.15.19:0/2417831312 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5854103660 0x7f585410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 --2- 172.21.15.19:0/2417831312 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f5854103120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 -- 172.21.15.19:0/2417831312 >> 172.21.15.19:0/2417831312 conn(0x7f58540fc7a0 msgr2=0x7f58540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 -- 172.21.15.19:0/2417831312 shutdown_connections 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.027+0000 7f585c116640 1 -- 172.21.15.19:0/2417831312 wait complete. 2024-09-12T03:19:07.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 Processor -- start 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 -- start start 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5854103660 0x7f5854100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f585410c170 0x7f5854101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5854111fb0 con 0x7f5854102d40 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5854111e30 con 0x7f585410c170 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585c116640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112130 con 0x7f5854103660 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42972/0 (socket says 172.21.15.19:42972) 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 -- 172.21.15.19:0/1062099498 learned_addr learned my addr 172.21.15.19:0/1062099498 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f58596b1640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5854103660 0x7f5854100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f585a6b3640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f585410c170 0x7f5854101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 -- 172.21.15.19:0/1062099498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5854103660 msgr2=0x7f5854100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5854103660 0x7f5854100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 -- 172.21.15.19:0/1062099498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f585410c170 msgr2=0x7f5854101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f585410c170 0x7f5854101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.028+0000 7f5859eb2640 1 -- 172.21.15.19:0/1062099498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58541a6940 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f5859eb2640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5850006010 tx=0x7f58500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f584affd640 1 -- 172.21.15.19:0/1062099498 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585003f030 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f584affd640 1 -- 172.21.15.19:0/1062099498 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5850003ce0 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58541a6bd0 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f584affd640 1 -- 172.21.15.19:0/1062099498 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850035070 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.029+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58541a70b0 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.031+0000 7f584affd640 1 -- 172.21.15.19:0/1062099498 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5850042030 con 0x7f5854102d40 2024-09-12T03:19:07.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.031+0000 7f585e4ef740 1 Processor -- start 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 -- start start 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc656cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc6577670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc64d3090 con 0x559fc6577bb0 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fc647aa70 con 0x559fc656d2f0 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585e4ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fc64cf050 con 0x559fc656c9d0 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42974/0 (socket says 172.21.15.19:42974) 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 -- 172.21.15.19:0/344045423 learned_addr learned my addr 172.21.15.19:0/344045423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f5859eb2640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc6577670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585a6b3640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc656cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 -- 172.21.15.19:0/344045423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 msgr2=0x559fc656cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f584affd640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 0x7f58400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc656cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 -- 172.21.15.19:0/344045423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 msgr2=0x559fc6577670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc6577670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f58596b1640 1 -- 172.21.15.19:0/344045423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc6487130 con 0x559fc6577bb0 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f5859eb2640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc6577670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.032+0000 7f585a6b3640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 0x7f58400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f584affd640 1 -- 172.21.15.19:0/1062099498 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58500ef120 con 0x7f5854102d40 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f58596b1640 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f584400bca0 tx=0x7f58440107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81b9743d6644fb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f583ffff640 1 -- 172.21.15.19:0/344045423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844004030 con 0x559fc6577bb0 2024-09-12T03:19:07.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f583ffff640 1 -- 172.21.15.19:0/344045423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58440041d0 con 0x559fc6577bb0 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f583ffff640 1 -- 172.21.15.19:0/344045423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844011730 con 0x559fc6577bb0 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f585e4ef740 1 -- 172.21.15.19:0/344045423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 msgr2=0x559fc6579fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f585a6b3640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 0x7f58400a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f584c00a4c0 tx=0x7f584c004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.033+0000 7f585e4ef740 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f584400bca0 tx=0x7f58440107a0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- 172.21.15.19:0/344045423 shutdown_connections 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc6577bb0 0x559fc6579fa0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc6577670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- 172.21.15.19:0/344045423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc656cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- 172.21.15.19:0/344045423 >> 172.21.15.19:0/344045423 conn(0x559fc6568f60 msgr2=0x559fc6569340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- 172.21.15.19:0/344045423 shutdown_connections 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- 172.21.15.19:0/344045423 wait complete. 2024-09-12T03:19:07.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- start start 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 Processor -- start 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc65746d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc6577bb0 0x559fc6574c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc657a780 con 0x559fc656c9d0 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fc648df20 con 0x559fc656d2f0 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.034+0000 7f585e4ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fc657a900 con 0x559fc6577bb0 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f585e4ef740 10 client.0 _ll_register_callbacks cb 0x559fc6457030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42980/0 (socket says 172.21.15.19:42980) 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 -- 172.21.15.19:0/299763544 learned_addr learned my addr 172.21.15.19:0/299763544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:07.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f58596b1640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc65746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f585a6b3640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc6577bb0 0x559fc6574c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 -- 172.21.15.19:0/299763544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc6577bb0 msgr2=0x559fc6574c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc6577bb0 0x559fc6574c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 -- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 msgr2=0x559fc65746d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc65746d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 -- 172.21.15.19:0/299763544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc62442b0 con 0x559fc656c9d0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f58596b1640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc65746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f5859eb2640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f58500b84d0 tx=0x7f5850003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82acb648a7361c0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f583d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850049070 con 0x559fc656c9d0 2024-09-12T03:19:07.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58500c4230 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58440041d0 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58500bb8d0 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f585e4ef740 10 client.15177 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fc6586010 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.035+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58440041d0 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f58500d4b90 con 0x559fc656c9d0 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 10 client.15177 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 10 client.15177 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:07.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 20 client.15177 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc6588640 0x559fc6589ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 4 client.15177 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fc64e0600 con 0x559fc6588640 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f58596b1640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc6588640 0x559fc6589ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f583d7fa640 10 client.15177 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.036+0000 7f58596b1640 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc6588640 0x559fc6589ab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=889c7d8a117c3f90 server_cookie=1b0762e817bfee16 in_seq=0 out_seq=0 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.037+0000 7f583d7fa640 1 -- 172.21.15.19:0/299763544 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fc64e0600 con 0x559fc6588640 2024-09-12T03:19:07.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.037+0000 7f583d7fa640 10 client.15177 handle_command_reply: tid=0 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.037+0000 7f585e4ef740 1 client.15177 shutdown 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.037+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 msgr2=0x559fc6577380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.037+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f58500b84d0 tx=0x7f5850003f50 comp rx=0 tx=0).stop 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 shutdown_connections 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc6588640 0x559fc6589ab0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc6577bb0 0x559fc6574c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f583d7fa640 0 client.15177 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc656d2f0 0x559fc65746d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 --2- 172.21.15.19:0/299763544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc6577380 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 >> 172.21.15.19:0/299763544 conn(0x559fc6568f60 msgr2=0x559fc656ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 shutdown_connections 2024-09-12T03:19:07.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 1 -- 172.21.15.19:0/299763544 wait complete. 2024-09-12T03:19:07.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.038+0000 7f585e4ef740 20 client.15177 trim_cache size 0 max 16384 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.055+0000 7f585e4ef740 1 Processor -- start 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 -- start start 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc6589ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc6574690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc6587df0 con 0x559fc656d2f0 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fc65884c0 con 0x559fc6577bb0 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585e4ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fc6588b90 con 0x559fc656c9d0 2024-09-12T03:19:07.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42996/0 (socket says 172.21.15.19:42996) 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 -- 172.21.15.19:0/3630088750 learned_addr learned my addr 172.21.15.19:0/3630088750 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585a6b3640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc6574690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f5859eb2640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc6589ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 -- 172.21.15.19:0/3630088750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 msgr2=0x559fc6589ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc6589ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 -- 172.21.15.19:0/3630088750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 msgr2=0x559fc6574690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc6574690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 -- 172.21.15.19:0/3630088750 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fc656d120 con 0x559fc656d2f0 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f585a6b3640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc6574690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:07.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f58596b1640 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f584400bca0 tx=0x7f584400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866721d4a021da55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.056+0000 7f583f7fe640 1 -- 172.21.15.19:0/3630088750 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844003e30 con 0x559fc656d2f0 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f583f7fe640 1 -- 172.21.15.19:0/3630088750 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5844003ad0 con 0x559fc656d2f0 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f583f7fe640 1 -- 172.21.15.19:0/3630088750 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440116a0 con 0x559fc656d2f0 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 -- 172.21.15.19:0/3630088750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 msgr2=0x559fc6573f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f584400bca0 tx=0x7f584400b2e0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 -- 172.21.15.19:0/3630088750 shutdown_connections 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc6574690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656d2f0 0x559fc6573f90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3630088750 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656c9d0 0x559fc6589ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 -- 172.21.15.19:0/3630088750 >> 172.21.15.19:0/3630088750 conn(0x559fc6568f60 msgr2=0x559fc6569340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 -- 172.21.15.19:0/3630088750 shutdown_connections 2024-09-12T03:19:07.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.057+0000 7f585e4ef740 1 -- 172.21.15.19:0/3630088750 wait complete. 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 -- start start 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 Processor -- start 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656d2f0 0x559fc656ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc656f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559fc6589930 con 0x559fc656c9d0 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559fc6589260 con 0x559fc6577bb0 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559fc658a000 con 0x559fc656d2f0 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f585e4ef740 10 client.0 _ll_register_callbacks cb 0x7f5850032ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f5859eb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43010/0 (socket says 172.21.15.19:43010) 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.058+0000 7f5859eb2640 1 -- 172.21.15.19:0/3211503497 learned_addr learned my addr 172.21.15.19:0/3211503497 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:07.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 -- 172.21.15.19:0/3211503497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656d2f0 msgr2=0x559fc656ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656d2f0 0x559fc656ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 -- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 msgr2=0x559fc656f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f585a6b3640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc656f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc656f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 -- 172.21.15.19:0/3211503497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fc656fba0 con 0x559fc656c9d0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f5859eb2640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f58500ba050 tx=0x7f58500da360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8feaace6b3cfbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850049040 con 0x559fc656c9d0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58500c4430 con 0x559fc656c9d0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5844003ad0 con 0x559fc656c9d0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f58500044e0 con 0x559fc656c9d0 2024-09-12T03:19:07.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f585e4ef740 10 client.15183 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559fc65f2d50 con 0x559fc656c9d0 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5844003ad0 con 0x559fc656c9d0 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.059+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f58500b7850 con 0x559fc656c9d0 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 10 client.15183 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 10 client.15183 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 20 client.15183 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc65f7760 0x559fc65f6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 4 client.15183 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "status"}) -- 0x559fc64dab10 con 0x559fc65f7760 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f58596b1640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc65f7760 0x559fc65f6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f58596b1640 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc65f7760 0x559fc65f6470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1f066b3f40ed17 server_cookie=88b2b6ba772c992c in_seq=0 out_seq=0 2024-09-12T03:19:07.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f583d7fa640 10 client.15183 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f583d7fa640 1 -- 172.21.15.19:0/3211503497 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559fc64dab10 con 0x559fc65f7760 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f583d7fa640 10 client.15183 handle_command_reply: tid=0 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.060+0000 7f585e4ef740 1 client.15183 shutdown 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 msgr2=0x559fc65923d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f58500ba050 tx=0x7f58500da360 comp rx=0 tx=0).stop 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 shutdown_connections 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559fc65f7760 0x559fc65f6470 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559fc6577bb0 0x559fc656f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f583d7fa640 0 client.15183 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559fc656d2f0 0x559fc656ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 --2- 172.21.15.19:0/3211503497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559fc656c9d0 0x559fc65923d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 >> 172.21.15.19:0/3211503497 conn(0x559fc6568f60 msgr2=0x559fc6574690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 shutdown_connections 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 1 -- 172.21.15.19:0/3211503497 wait complete. 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585e4ef740 20 client.15183 trim_cache size 0 max 16384 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24467, 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:07.068 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 109.11299045600001, 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 128.462841191 2024-09-12T03:19:07.069 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 msgr2=0x7f58400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 0x7f58400a5040 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f584c00a4c0 tx=0x7f584c004b50 comp rx=0 tx=0).stop 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 msgr2=0x7f58541009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5850006010 tx=0x7f58500042f0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 shutdown_connections 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f585410c170 0x7f5854101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5854103660 0x7f5854100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58400a2b80 0x7f58400a5040 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 --2- 172.21.15.19:0/1062099498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5854102d40 0x7f58541009a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.061+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 >> 172.21.15.19:0/1062099498 conn(0x7f58540fc7a0 msgr2=0x7f58540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.062+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 shutdown_connections 2024-09-12T03:19:07.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:07.062+0000 7f585c116640 1 -- 172.21.15.19:0/1062099498 wait complete. 2024-09-12T03:19:07.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:06 smithi121 ceph-mon[34963]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:07.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:19:08.049 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.543+0000 7f69f4cce640 1 -- 172.21.15.19:0/500919063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102ef0 msgr2=0x7f69f010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.543+0000 7f69f4cce640 1 --2- 172.21.15.19:0/500919063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102ef0 0x7f69f010da40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ca0 tx=0x7f69e802a950 comp rx=0 tx=0).stop 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 -- 172.21.15.19:0/500919063 shutdown_connections 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 --2- 172.21.15.19:0/500919063 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102ef0 0x7f69f010da40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 --2- 172.21.15.19:0/500919063 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0102570 0x7f69f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 --2- 172.21.15.19:0/500919063 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0100e20 0x7f69f0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 -- 172.21.15.19:0/500919063 >> 172.21.15.19:0/500919063 conn(0x7f69f00fc7d0 msgr2=0x7f69f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 -- 172.21.15.19:0/500919063 shutdown_connections 2024-09-12T03:19:08.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 -- 172.21.15.19:0/500919063 wait complete. 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 Processor -- start 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.544+0000 7f69f4cce640 1 -- start start 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0100e20 0x7f69f00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0102ef0 0x7f69f00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0110f80 con 0x7f69f0102570 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0110e00 con 0x7f69f0102ef0 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69f4cce640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69f0111100 con 0x7f69f0100e20 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43020/0 (socket says 172.21.15.19:43020) 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 -- 172.21.15.19:0/3034984175 learned_addr learned my addr 172.21.15.19:0/3034984175 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0100e20 0x7f69f00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0102ef0 0x7f69f00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 -- 172.21.15.19:0/3034984175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0100e20 msgr2=0x7f69f00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0100e20 0x7f69f00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 -- 172.21.15.19:0/3034984175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0102ef0 msgr2=0x7f69f00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0102ef0 0x7f69f00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 -- 172.21.15.19:0/3034984175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69f01a6940 con 0x7f69f0102570 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69edd9b640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d770 tx=0x7f69e400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.545+0000 7f69df7fe640 1 -- 172.21.15.19:0/3034984175 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4017070 con 0x7f69f0102570 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.546+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69f01a6c30 con 0x7f69f0102570 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.546+0000 7f69df7fe640 1 -- 172.21.15.19:0/3034984175 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e4005500 con 0x7f69f0102570 2024-09-12T03:19:08.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.546+0000 7f69df7fe640 1 -- 172.21.15.19:0/3034984175 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e4004e00 con 0x7f69f0102570 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.546+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69f01a7170 con 0x7f69f0102570 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.548+0000 7f69df7fe640 1 -- 172.21.15.19:0/3034984175 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f69e4004fa0 con 0x7f69f0102570 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.548+0000 7f69f70a7740 1 Processor -- start 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.548+0000 7f69f70a7740 1 -- start start 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c138bb0 0x56017c13afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c094090 con 0x56017c12d9d0 2024-09-12T03:19:08.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56017c03ba70 con 0x56017c138bb0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69f70a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56017c090050 con 0x56017c12e2f0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43034/0 (socket says 172.21.15.19:43034) 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 -- 172.21.15.19:0/3436657977 learned_addr learned my addr 172.21.15.19:0/3436657977 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69edd9b640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c138bb0 0x56017c13afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 -- 172.21.15.19:0/3436657977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 msgr2=0x56017c138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 -- 172.21.15.19:0/3436657977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c138bb0 msgr2=0x56017c13afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c138bb0 0x56017c13afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 -- 172.21.15.19:0/3436657977 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c048130 con 0x56017c12d9d0 2024-09-12T03:19:08.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69df7fe640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 0x7f69d80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ca0 tx=0x7f69e8009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8237314de26d3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 0x7f69d80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.549+0000 7f69df7fe640 1 -- 172.21.15.19:0/3034984175 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69e40ce8c0 con 0x7f69f0102570 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69dcff9640 1 -- 172.21.15.19:0/3436657977 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8048070 con 0x56017c12d9d0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69dcff9640 1 -- 172.21.15.19:0/3436657977 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e802f9f0 con 0x56017c12d9d0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69dcff9640 1 -- 172.21.15.19:0/3436657977 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8034040 con 0x56017c12d9d0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 -- 172.21.15.19:0/3436657977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 msgr2=0x56017c12ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 0x7f69d80a50c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56017c12adc0 tx=0x7f69e000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f69e8009ca0 tx=0x7f69e8009820 comp rx=0 tx=0).stop 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 -- 172.21.15.19:0/3436657977 shutdown_connections 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c138bb0 0x56017c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3436657977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c12ddb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.550+0000 7f69f70a7740 1 -- 172.21.15.19:0/3436657977 >> 172.21.15.19:0/3436657977 conn(0x56017c129f60 msgr2=0x56017c12a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- 172.21.15.19:0/3436657977 shutdown_connections 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- 172.21.15.19:0/3436657977 wait complete. 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- start start 2024-09-12T03:19:08.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 Processor -- start 2024-09-12T03:19:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c135770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c138bb0 0x56017c135cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c13b780 con 0x56017c12d9d0 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56017c04ef20 con 0x56017c12e2f0 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.551+0000 7f69f70a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56017c13b900 con 0x56017c138bb0 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69f70a7740 10 client.0 _ll_register_callbacks cb 0x56017c018030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43046/0 (socket says 172.21.15.19:43046) 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 -- 172.21.15.19:0/1017918953 learned_addr learned my addr 172.21.15.19:0/1017918953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:08.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69eed9d640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c138bb0 0x56017c135cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69edd9b640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c135770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 -- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c138bb0 msgr2=0x56017c135cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c138bb0 0x56017c135cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 -- 172.21.15.19:0/1017918953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 msgr2=0x56017c135770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c135770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 -- 172.21.15.19:0/1017918953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017be052b0 con 0x56017c12d9d0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69ee59c640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f69e000bed0 tx=0x7f69e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62d0493325e9bca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e0005220 con 0x56017c12d9d0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69e00053c0 con 0x56017c12d9d0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e802f9f0 con 0x56017c12d9d0 2024-09-12T03:19:08.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69e00055c0 con 0x56017c12d9d0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69f70a7740 10 client.15195 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.552+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56017c147180 con 0x56017c12d9d0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e802f9f0 con 0x56017c12d9d0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f69e0024070 con 0x56017c12d9d0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 10 client.15195 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 20 client.15195 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 4 client.15195 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56017c0a1600 con 0x56017c1497d0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.553+0000 7f69edd9b640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.554+0000 7f69edd9b640 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c14ac40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68527b985d1b076 server_cookie=f3e3c58a0df08432 in_seq=0 out_seq=0 2024-09-12T03:19:08.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.554+0000 7f69d5ffb640 10 client.15195 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.554+0000 7f69d5ffb640 1 -- 172.21.15.19:0/1017918953 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56017c0a1600 con 0x56017c1497d0 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.554+0000 7f69d5ffb640 10 client.15195 handle_command_reply: tid=0 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 client.15195 shutdown 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 msgr2=0x56017c138420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f69e000bed0 tx=0x7f69e00046c0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 shutdown_connections 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c138bb0 0x56017c135cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c14ac40 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c135770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69d5ffb640 0 client.15195 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1017918953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c12d9d0 0x56017c138420 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 >> 172.21.15.19:0/1017918953 conn(0x56017c129f60 msgr2=0x56017c12fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 shutdown_connections 2024-09-12T03:19:08.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 1 -- 172.21.15.19:0/1017918953 wait complete. 2024-09-12T03:19:08.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.555+0000 7f69f70a7740 20 client.15195 trim_cache size 0 max 16384 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 Processor -- start 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 -- start start 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12d9d0 0x56017c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c148f80 con 0x56017c138bb0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56017c149650 con 0x56017c12d9d0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69f70a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56017c149d20 con 0x56017c12e2f0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43048/0 (socket says 172.21.15.19:43048) 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 -- 172.21.15.19:0/1919727259 learned_addr learned my addr 172.21.15.19:0/1919727259 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69ee59c640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12d9d0 0x56017c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69edd9b640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 -- 172.21.15.19:0/1919727259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 msgr2=0x56017c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 -- 172.21.15.19:0/1919727259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12d9d0 msgr2=0x56017c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12d9d0 0x56017c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.573+0000 7f69eed9d640 1 -- 172.21.15.19:0/1919727259 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56017c12d800 con 0x56017c138bb0 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69edd9b640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69eed9d640 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003da0 tx=0x7f69e8003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75473adb59f4a22 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69d7fff640 1 -- 172.21.15.19:0/1919727259 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8053070 con 0x56017c138bb0 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69d7fff640 1 -- 172.21.15.19:0/1919727259 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e802f9f0 con 0x56017c138bb0 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69d7fff640 1 -- 172.21.15.19:0/1919727259 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8048040 con 0x56017c138bb0 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 -- 172.21.15.19:0/1919727259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 msgr2=0x56017c153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003da0 tx=0x7f69e8003dd0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 -- 172.21.15.19:0/1919727259 shutdown_connections 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c153460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12e2f0 0x56017c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 --2- 172.21.15.19:0/1919727259 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12d9d0 0x56017c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.574+0000 7f69f70a7740 1 -- 172.21.15.19:0/1919727259 >> 172.21.15.19:0/1919727259 conn(0x56017c129f60 msgr2=0x56017c134040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- 172.21.15.19:0/1919727259 shutdown_connections 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- 172.21.15.19:0/1919727259 wait complete. 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- start start 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 Processor -- start 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12d9d0 0x56017c14c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c14c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56017c14aac0 con 0x56017c138bb0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56017c14a3f0 con 0x56017c12e2f0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56017c14b190 con 0x56017c12d9d0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69f70a7740 10 client.0 _ll_register_callbacks cb 0x7f69e0005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69edd9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43054/0 (socket says 172.21.15.19:43054) 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69eed9d640 1 -- 172.21.15.19:0/3236751305 learned_addr learned my addr 172.21.15.19:0/3236751305 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.575+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12d9d0 0x56017c14c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 -- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12d9d0 msgr2=0x56017c14c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12d9d0 0x56017c14c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 -- 172.21.15.19:0/3236751305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 msgr2=0x56017c14c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c14c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 -- 172.21.15.19:0/3236751305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56017c1b2410 con 0x56017c138bb0 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69edd9b640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69eed9d640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f69e8046040 tx=0x7f69e8009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ddf73bc9aee96a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e805c060 con 0x56017c138bb0 2024-09-12T03:19:08.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69e802f3a0 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69e802f9f0 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f69e8053050 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69f70a7740 10 client.15201 fetch_fsmap learned FSMap version 27 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56017c1b3650 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.576+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69e802f9f0 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f69e8030440 con 0x56017c138bb0 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 10 client.15201 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 10 client.15201 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 20 client.15201 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:08.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c1b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 4 client.15201 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "status"}) -- 0x56017c09bb10 con 0x56017c1497d0 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c1b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69ee59c640 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c1b67c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcfafa038ebbe6e server_cookie=c6ac2dc60934e983 in_seq=0 out_seq=0 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.577+0000 7f69d5ffb640 10 client.15201 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:08.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69d5ffb640 1 -- 172.21.15.19:0/3236751305 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56017c09bb10 con 0x56017c1497d0 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69d5ffb640 10 client.15201 handle_command_reply: tid=0 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 client.15201 shutdown 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 msgr2=0x56017c1b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f69e8046040 tx=0x7f69e8009910 comp rx=0 tx=0).stop 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 shutdown_connections 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56017c138bb0 0x56017c1b1ed0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56017c1497d0 0x56017c1b67c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56017c12e2f0 0x56017c14c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 --2- 172.21.15.19:0/3236751305 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56017c12d9d0 0x56017c14c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 >> 172.21.15.19:0/3236751305 conn(0x56017c129f60 msgr2=0x56017c12f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69d5ffb640 0 client.15201 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 shutdown_connections 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 1 -- 172.21.15.19:0/3236751305 wait complete. 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f70a7740 20 client.15201 trim_cache size 0 max 16384 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:08.587 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 117.670114478, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 27, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 80, 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 140.11488901600001 2024-09-12T03:19:08.588 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 msgr2=0x7f69d80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 0x7f69d80a50c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x56017c12adc0 tx=0x7f69e000b390 comp rx=0 tx=0).stop 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 msgr2=0x7f69f00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.578+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f69e400d770 tx=0x7f69e400dc30 comp rx=0 tx=0).stop 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 shutdown_connections 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69f0102ef0 0x7f69f00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69d80a2c00 0x7f69d80a50c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69f0102570 0x7f69f00fd1c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 --2- 172.21.15.19:0/3034984175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69f0100e20 0x7f69f00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 >> 172.21.15.19:0/3034984175 conn(0x7f69f00fc7d0 msgr2=0x7f69f006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 shutdown_connections 2024-09-12T03:19:08.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:08.579+0000 7f69f4cce640 1 -- 172.21.15.19:0/3034984175 wait complete. 2024-09-12T03:19:09.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:08 smithi135 ceph-mon[35474]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T03:19:09.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:08 smithi121 ceph-mon[34963]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T03:19:09.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:08 smithi019 ceph-mon[30953]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T03:19:09.262 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T03:19:09.273 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T03:19:09.273 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T03:19:09.273 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T03:19:09.273 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T03:19:09.299 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T03:19:09.299 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T03:19:09.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:09.300 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-12T03:19:09.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:09.318 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-09-12T03:19:09.387 INFO:teuthology.orchestra.run.smithi019.stderr:Cannot find device "ceph-brx" 2024-09-12T03:19:09.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:19:09.389 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-12T03:19:09.389 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T03:19:09.389 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:09.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T03:19:09.688 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.186+0000 7f66f5115640 1 -- 172.21.15.19:0/1931838386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0106f10 msgr2=0x7f66f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.186+0000 7f66f5115640 1 --2- 172.21.15.19:0/1931838386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0106f10 0x7f66f010ea90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f66e8009ca0 tx=0x7f66e802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 -- 172.21.15.19:0/1931838386 shutdown_connections 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 --2- 172.21.15.19:0/1931838386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0106f10 0x7f66f010ea90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 --2- 172.21.15.19:0/1931838386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 0x7f66f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 --2- 172.21.15.19:0/1931838386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0101de0 0x7f66f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 -- 172.21.15.19:0/1931838386 >> 172.21.15.19:0/1931838386 conn(0x7f66f00fd8c0 msgr2=0x7f66f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 -- 172.21.15.19:0/1931838386 shutdown_connections 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 -- 172.21.15.19:0/1931838386 wait complete. 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.187+0000 7f66f5115640 1 Processor -- start 2024-09-12T03:19:10.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 -- start start 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 0x7f66f01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 0x7f66f019cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0111f70 con 0x7f66f0101de0 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0111df0 con 0x7f66f0106f10 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66f5115640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01120f0 con 0x7f66f0102790 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43072/0 (socket says 172.21.15.19:43072) 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 -- 172.21.15.19:0/1239661742 learned_addr learned my addr 172.21.15.19:0/1239661742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66ef59e640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 0x7f66f019cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66ee59c640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 0x7f66f01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 -- 172.21.15.19:0/1239661742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 msgr2=0x7f66f01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 0x7f66f01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 -- 172.21.15.19:0/1239661742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 msgr2=0x7f66f019cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 0x7f66f019cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 -- 172.21.15.19:0/1239661742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f019d230 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66ef59e640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 0x7f66f019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.188+0000 7f66eed9d640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c920 tx=0x7f66e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.189+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0007c70 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.189+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66e0007e10 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.189+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f019d520 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.189+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0018890 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.189+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01a4e00 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.190+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f01021c0 con 0x7f66f0101de0 2024-09-12T03:19:10.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.191+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66e0018a50 con 0x7f66f0101de0 2024-09-12T03:19:10.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.192+0000 7f66d7fff640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 0x7f66d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:10.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.192+0000 7f66ee59c640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 0x7f66d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:10.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.192+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f66e00cf4d0 con 0x7f66f0101de0 2024-09-12T03:19:10.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.192+0000 7f66ee59c640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 0x7f66d80a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f66f00fe0d0 tx=0x7f66e400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:10.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.195+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f66e0014030 con 0x7f66f0101de0 2024-09-12T03:19:10.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.407+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f66f01a50a0 con 0x7f66f0101de0 2024-09-12T03:19:10.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.407+0000 7f66d7fff640 1 -- 172.21.15.19:0/1239661742 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f66e009c4d0 con 0x7f66f0101de0 2024-09-12T03:19:10.408 INFO:teuthology.orchestra.run.smithi019.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 msgr2=0x7f66d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 0x7f66d80a5040 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f66f00fe0d0 tx=0x7f66e400c4a0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 msgr2=0x7f66f0198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f66e000c920 tx=0x7f66e000cde0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 shutdown_connections 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f66f0106f10 0x7f66f019cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f66f0102790 0x7f66f01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f66d80a2b80 0x7f66d80a5040 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 --2- 172.21.15.19:0/1239661742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f66f0101de0 0x7f66f0198280 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:10.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 >> 172.21.15.19:0/1239661742 conn(0x7f66f00fd8c0 msgr2=0x7f66f00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:10.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 shutdown_connections 2024-09-12T03:19:10.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:10.410+0000 7f66f5115640 1 -- 172.21.15.19:0/1239661742 wait complete. 2024-09-12T03:19:11.076 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:10 smithi019 ceph-mon[30953]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T03:19:11.076 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:10 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1239661742' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:19:11.105 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi019.front.sepia.ceph.com 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T03:19:11.106 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:19:11.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:11.107 DEBUG:teuthology.orchestra.run.smithi019:> ip addr 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 ::1/128 scope host 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:11.124 INFO:teuthology.orchestra.run.smithi019.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:12 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f0 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:6b:fd:13 brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp7s0f1 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1e brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f0 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: inet 172.21.15.19/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft 42166sec preferred_lft 42166sec 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: inet6 fe80::ec4:7aff:febd:161e/64 scope link 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout: valid_lft forever preferred_lft forever 2024-09-12T03:19:11.125 INFO:teuthology.orchestra.run.smithi019.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T03:19:11.126 INFO:teuthology.orchestra.run.smithi019.stdout: link/ether 0c:c4:7a:bd:16:1f brd ff:ff:ff:ff:ff:ff 2024-09-12T03:19:11.126 INFO:teuthology.orchestra.run.smithi019.stdout: altname enp1s0f1 2024-09-12T03:19:11.126 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T03:19:11.126 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:11.126 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-12T03:19:11.126 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add name ceph-brx type bridge 2024-09-12T03:19:11.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr flush dev ceph-brx 2024-09-12T03:19:11.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set ceph-brx up 2024-09-12T03:19:11.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T03:19:11.127 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-12T03:19:11.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:10 smithi121 ceph-mon[34963]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T03:19:11.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:10 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1239661742' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:19:11.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:10 smithi135 ceph-mon[35474]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-09-12T03:19:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:10 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1239661742' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T03:19:11.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:11.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:11.350 DEBUG:teuthology.orchestra.run.smithi019:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T03:19:11.377 INFO:teuthology.orchestra.run.smithi019.stdout:1 2024-09-12T03:19:11.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:11.379 DEBUG:teuthology.orchestra.run.smithi019:> route 2024-09-12T03:19:11.458 INFO:teuthology.orchestra.run.smithi019.stdout:Kernel IP routing table 2024-09-12T03:19:11.458 INFO:teuthology.orchestra.run.smithi019.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T03:19:11.458 INFO:teuthology.orchestra.run.smithi019.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T03:19:11.458 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T03:19:11.459 INFO:teuthology.orchestra.run.smithi019.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T03:19:11.459 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-12T03:19:11.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:11.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:11.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:11.772 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-12T03:19:11.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:11.789 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list-id 2024-09-12T03:19:11.845 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:11.845 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-12T03:19:11.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:19:11.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T03:19:11.845 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-12T03:19:11.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:11.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:11.966 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T03:19:11.967 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:11.967 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-12T03:19:11.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T03:19:11.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T03:19:11.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T03:19:11.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T03:19:11.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T03:19:11.968 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-12T03:19:12.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:12.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.128 DEBUG:teuthology.orchestra.run.smithi019:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T03:19:12.128 DEBUG:teuthology.orchestra.run.smithi019:> set -e 2024-09-12T03:19:12.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set brx.0 up 2024-09-12T03:19:12.128 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T03:19:12.128 DEBUG:teuthology.orchestra.run.smithi019:> ') 2024-09-12T03:19:12.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:12.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.207 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:12.207 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:12.223 INFO:teuthology.orchestra.run.smithi019.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T03:19:12.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T03:19:12.224 DEBUG:teuthology.orchestra.run.smithi019:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:12.279 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T03:19:12.279 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T03:19:12.280 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6 2024-09-12T03:19:12.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:12.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:19:12.381 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T03:19:12.382 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:invalid new device string format 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: resolved to: "172.21.15.19:3300,172.21.15.121:3300,172.21.15.135:3300" 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.19:3300,172.21.15.121:3300,172.21.15.135:3300:/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6 2024-09-12T03:19:12.515 INFO:teuthology.orchestra.run.smithi019.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=45fb56a2-70b4-11ef-bcea-c7b262605968" will pass to kernel 2024-09-12T03:19:12.516 INFO:teuthology.orchestra.run.smithi019.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T03:19:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: You just mounted a file system that supports labels which does not 2024-09-12T03:19:12.516 INFO:teuthology.orchestra.run.smithi019.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T03:19:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T03:19:12.517 INFO:teuthology.orchestra.run.smithi019.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T03:19:12.518 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T03:19:12.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:19:12.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:12.546 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:19:12.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:12.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.645 INFO:teuthology.orchestra.run.smithi019.stdout:client15210 2024-09-12T03:19:12.646 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:19:12.646 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:19:12.663 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:19:12.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:12.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.762 INFO:teuthology.orchestra.run.smithi019.stdout:client15210 2024-09-12T03:19:12.762 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T03:19:12.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T03:19:12.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.820 INFO:teuthology.orchestra.run.smithi019.stdout:client15210 2024-09-12T03:19:12.821 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968.client15210/status') 2024-09-12T03:19:12.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T03:19:12.882 INFO:teuthology.orchestra.run.smithi019.stdout:instance: client.15210 (3)192.168.0.1:0/3995404688 2024-09-12T03:19:12.882 INFO:teuthology.orchestra.run.smithi019.stdout:blocklisted: false 2024-09-12T03:19:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records in 2024-09-12T03:19:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:0+1 records out 2024-09-12T03:19:12.882 INFO:teuthology.orchestra.run.smithi019.stderr:70 bytes copied, 2.8437e-05 s, 2.5 MB/s 2024-09-12T03:19:12.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T03:19:12.890 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T03:19:12.900 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T03:19:12.900 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T03:19:12.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-12T03:19:13.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 -- 172.21.15.19:0/2214542860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 msgr2=0x7f8570109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 --2- 172.21.15.19:0/2214542860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f8570109cb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8568009ca0 tx=0x7f856802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 -- 172.21.15.19:0/2214542860 shutdown_connections 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 --2- 172.21.15.19:0/2214542860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f8570109cb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 --2- 172.21.15.19:0/2214542860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8570104f10 0x7f8570107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 --2- 172.21.15.19:0/2214542860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85701025e0 0x7f85701049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 -- 172.21.15.19:0/2214542860 >> 172.21.15.19:0/2214542860 conn(0x7f85700fc1e0 msgr2=0x7f85700fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 -- 172.21.15.19:0/2214542860 shutdown_connections 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.005+0000 7f8576198640 1 -- 172.21.15.19:0/2214542860 wait complete. 2024-09-12T03:19:13.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 Processor -- start 2024-09-12T03:19:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 -- start start 2024-09-12T03:19:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85701025e0 0x7f8570197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8570104f10 0x7f8570198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d820 con 0x7f8570107840 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d6a0 con 0x7f8570104f10 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f8576198640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f857010d9a0 con 0x7f85701025e0 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f856ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f856ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38200/0 (socket says 172.21.15.19:38200) 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f856ffff640 1 -- 172.21.15.19:0/3578858501 learned_addr learned my addr 172.21.15.19:0/3578858501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.006+0000 7f856f7fe640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85701025e0 0x7f8570197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856effd640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8570104f10 0x7f8570198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 -- 172.21.15.19:0/3578858501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85701025e0 msgr2=0x7f8570197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85701025e0 0x7f8570197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 -- 172.21.15.19:0/3578858501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8570104f10 msgr2=0x7f8570198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8570104f10 0x7f8570198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 -- 172.21.15.19:0/3578858501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857019fa40 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856ffff640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8568031a40 tx=0x7f8568031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568003c30 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857019fcd0 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568003dd0 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856802fdb0 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.007+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a01c0 con 0x7f8570107840 2024-09-12T03:19:13.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.008+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85700fdcd0 con 0x7f8570107840 2024-09-12T03:19:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.009+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8568004050 con 0x7f8570107840 2024-09-12T03:19:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.010+0000 7f856cff9640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 0x7f85580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.010+0000 7f856f7fe640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 0x7f85580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.010+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f85680f04e0 con 0x7f8570107840 2024-09-12T03:19:13.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.011+0000 7f856f7fe640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 0x7f85580a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8560005fd0 tx=0x7f8560005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:13.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.014+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f85680bd450 con 0x7f8570107840 2024-09-12T03:19:13.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:12 smithi121 ceph-mon[34963]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:13.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:12 smithi019 ceph-mon[30953]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:13.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.229+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f857002d7c0 con 0x7f8570107840 2024-09-12T03:19:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:12 smithi135 ceph-mon[35474]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.843+0000 7f856cff9640 1 -- 172.21.15.19:0/3578858501 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f85680bcd50 con 0x7f8570107840 2024-09-12T03:19:14.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.846+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 msgr2=0x7f85580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.846+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 0x7f85580a50f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8560005fd0 tx=0x7f8560005850 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.846+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 msgr2=0x7f857019f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.846+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8568031a40 tx=0x7f8568031ee0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.846+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 shutdown_connections 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85580a2c30 0x7f85580a50f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8570107840 0x7f857019f3d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8570104f10 0x7f8570198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 --2- 172.21.15.19:0/3578858501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85701025e0 0x7f8570197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 >> 172.21.15.19:0/3578858501 conn(0x7f85700fc1e0 msgr2=0x7f8570102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 shutdown_connections 2024-09-12T03:19:14.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:13.847+0000 7f8576198640 1 -- 172.21.15.19:0/3578858501 wait complete. 2024-09-12T03:19:14.044 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T03:19:14.058 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T03:19:14.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:19:14.098 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:13 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:19:14.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:13 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.187+0000 7fcac084c640 1 -- 172.21.15.19:0/1521330195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 msgr2=0x7fcabc1049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.187+0000 7fcac084c640 1 --2- 172.21.15.19:0/1521330195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc1049a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009900 tx=0x7fcaac02a410 comp rx=0 tx=0).stop 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.187+0000 7fcac084c640 1 -- 172.21.15.19:0/1521330195 shutdown_connections 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.187+0000 7fcac084c640 1 --2- 172.21.15.19:0/1521330195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc107810 0x7fcabc109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.187+0000 7fcac084c640 1 --2- 172.21.15.19:0/1521330195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc104ee0 0x7fcabc1072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 --2- 172.21.15.19:0/1521330195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc1049a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 -- 172.21.15.19:0/1521330195 >> 172.21.15.19:0/1521330195 conn(0x7fcabc0fc1b0 msgr2=0x7fcabc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 -- 172.21.15.19:0/1521330195 shutdown_connections 2024-09-12T03:19:14.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 -- 172.21.15.19:0/1521330195 wait complete. 2024-09-12T03:19:14.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 Processor -- start 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.188+0000 7fcac084c640 1 -- start start 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc104ee0 0x7fcabc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 0x7fcabc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10d810 con 0x7fcabc1025b0 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10d690 con 0x7fcabc104ee0 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcac084c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcabc10d990 con 0x7fcabc107810 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38216/0 (socket says 172.21.15.19:38216) 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 -- 172.21.15.19:0/3536469812 learned_addr learned my addr 172.21.15.19:0/3536469812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:14.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcab3fff640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc104ee0 0x7fcabc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcabad9d640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 0x7fcabc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 -- 172.21.15.19:0/3536469812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 msgr2=0x7fcabc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 0x7fcabc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 -- 172.21.15.19:0/3536469812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc104ee0 msgr2=0x7fcabc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc104ee0 0x7fcabc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 -- 172.21.15.19:0/3536469812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcabc19fb10 con 0x7fcabc1025b0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcabad9d640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 0x7fcabc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.189+0000 7fcaba59c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009230 tx=0x7fcaac031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.190+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac03f030 con 0x7fcabc1025b0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.190+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcabc19fda0 con 0x7fcabc1025b0 2024-09-12T03:19:14.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.190+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaac003ad0 con 0x7fcabc1025b0 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.190+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaac035070 con 0x7fcabc1025b0 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.190+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcabc1a01d0 con 0x7fcabc1025b0 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.191+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcabc0fdca0 con 0x7fcabc1025b0 2024-09-12T03:19:14.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.192+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcaac042030 con 0x7fcabc1025b0 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.192+0000 7fcab37fe640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 0x7fcaa80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.193+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcaac0efa30 con 0x7fcabc1025b0 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.193+0000 7fcab3fff640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 0x7fcaa80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.193+0000 7fcab3fff640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 0x7fcaa80a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005fd0 tx=0x7fcaa4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.196+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcaac0bc9a0 con 0x7fcabc1025b0 2024-09-12T03:19:14.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:13 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-09-12T03:19:14.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.374+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcabc062e90 con 0x7fcabc1025b0 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.376+0000 7fcab37fe640 1 -- 172.21.15.19:0/3536469812 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7fcaac0bc2a0 con 0x7fcabc1025b0 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:14.376 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 msgr2=0x7fcaa80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 0x7fcaa80a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4005fd0 tx=0x7fcaa4005850 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 msgr2=0x7fcabc197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009230 tx=0x7fcaac031b60 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 shutdown_connections 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcabc107810 0x7fcabc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcabc104ee0 0x7fcabc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcaa80a2b90 0x7fcaa80a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 --2- 172.21.15.19:0/3536469812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcabc1025b0 0x7fcabc197ef0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 >> 172.21.15.19:0/3536469812 conn(0x7fcabc0fc1b0 msgr2=0x7fcabc103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 shutdown_connections 2024-09-12T03:19:14.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.378+0000 7fcac084c640 1 -- 172.21.15.19:0/3536469812 wait complete. 2024-09-12T03:19:14.389 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:12:25.696963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:17:06.299019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:17:08.893970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:19:14.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 -- 172.21.15.19:0/2377331498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101390 msgr2=0x7f57d4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2377331498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101390 0x7f57d4101810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009900 tx=0x7f57c002a410 comp rx=0 tx=0).stop 2024-09-12T03:19:14.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 -- 172.21.15.19:0/2377331498 shutdown_connections 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2377331498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d4101d50 0x7f57d41085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2377331498 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101390 0x7f57d4101810 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2377331498 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d41001a0 0x7f57d41005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 -- 172.21.15.19:0/2377331498 >> 172.21.15.19:0/2377331498 conn(0x7f57d40fb900 msgr2=0x7f57d40fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 -- 172.21.15.19:0/2377331498 shutdown_connections 2024-09-12T03:19:14.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.537+0000 7f57dae3c640 1 -- 172.21.15.19:0/2377331498 wait complete. 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 Processor -- start 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 -- start start 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d41001a0 0x7f57d4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d4101390 0x7f57d41983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f57d410c6e0 con 0x7f57d4101d50 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f57d410c560 con 0x7f57d41001a0 2024-09-12T03:19:14.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57dae3c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f57d410c860 con 0x7f57d4101390 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38224/0 (socket says 172.21.15.19:38224) 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 -- 172.21.15.19:0/2098218349 learned_addr learned my addr 172.21.15.19:0/2098218349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d3fff640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d4101390 0x7f57d41983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d8bd8640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d41001a0 0x7f57d4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 -- 172.21.15.19:0/2098218349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d4101390 msgr2=0x7f57d41983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d4101390 0x7f57d41983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 -- 172.21.15.19:0/2098218349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d41001a0 msgr2=0x7f57d4197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d41001a0 0x7f57d4197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.538+0000 7f57d93d9640 1 -- 172.21.15.19:0/2098218349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d419fa80 con 0x7f57d4101d50 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57d93d9640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009fd0 tx=0x7f57cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc017070 con 0x7f57d4101d50 2024-09-12T03:19:14.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d419fd70 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57cc003e50 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc0106c0 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.539+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d41a01e0 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.540+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d41a0410 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.541+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57cc003970 con 0x7f57d4101d50 2024-09-12T03:19:14.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.542+0000 7f57d1ffb640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 0x7f57c40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.542+0000 7f57d8bd8640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 0x7f57c40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.542+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f57cc0ced80 con 0x7f57d4101d50 2024-09-12T03:19:14.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.542+0000 7f57d8bd8640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 0x7f57c40a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57d41011f0 tx=0x7f57c800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.546+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57cc09b640 con 0x7f57d4101d50 2024-09-12T03:19:14.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.722+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57d41a05e0 con 0x7f57d4101d50 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.722+0000 7f57d1ffb640 1 -- 172.21.15.19:0/2098218349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f57cc097020 con 0x7f57d4101d50 2024-09-12T03:19:14.722 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-12T03:19:14.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 msgr2=0x7f57c40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 0x7f57c40a5050 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57d41011f0 tx=0x7f57c800b390 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 msgr2=0x7f57d419f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009fd0 tx=0x7f57cc00b260 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 shutdown_connections 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57c40a2b90 0x7f57c40a5050 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f57d4101d50 0x7f57d419f410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f57d4101390 0x7f57d41983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 --2- 172.21.15.19:0/2098218349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f57d41001a0 0x7f57d4197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 >> 172.21.15.19:0/2098218349 conn(0x7f57d40fb900 msgr2=0x7f57d4109050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.724+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 shutdown_connections 2024-09-12T03:19:14.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.725+0000 7f57dae3c640 1 -- 172.21.15.19:0/2098218349 wait complete. 2024-09-12T03:19:14.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1415061821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 msgr2=0x7fe174104a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1415061821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174104a40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe164009900 tx=0x7fe16402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1415061821 shutdown_connections 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1415061821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 0x7fe174109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1415061821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 0x7fe174107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1415061821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174104a40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1415061821 >> 172.21.15.19:0/1415061821 conn(0x7fe1740fc1b0 msgr2=0x7fe1740fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1415061821 shutdown_connections 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1415061821 wait complete. 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 Processor -- start 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.843+0000 7fe17aa1e640 1 -- start start 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 0x7fe174100600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 0x7fe174100b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe17410d7d0 con 0x7fe174102650 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe17410d650 con 0x7fe174104f80 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe17aa1e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe17410d950 con 0x7fe1741078b0 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38234/0 (socket says 172.21.15.19:38234) 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 -- 172.21.15.19:0/1966621437 learned_addr learned my addr 172.21.15.19:0/1966621437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe16bfff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 0x7fe174100600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe178fbb640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 0x7fe174100b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 -- 172.21.15.19:0/1966621437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 msgr2=0x7fe174100b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 0x7fe174100b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 -- 172.21.15.19:0/1966621437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 msgr2=0x7fe174100600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 0x7fe174100600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:14.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 -- 172.21.15.19:0/1966621437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1741013d0 con 0x7fe174102650 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.844+0000 7fe173fff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe164009230 tx=0x7fe164004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.847+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe16403f030 con 0x7fe174102650 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.847+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe164003ce0 con 0x7fe174102650 2024-09-12T03:19:14.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.847+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1741a1f10 con 0x7fe174102650 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.847+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164035070 con 0x7fe174102650 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.847+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1741a2370 con 0x7fe174102650 2024-09-12T03:19:14.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.848+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1740fdca0 con 0x7fe174102650 2024-09-12T03:19:14.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.849+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe164042030 con 0x7fe174102650 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.850+0000 7fe171ffb640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 0x7fe1600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.850+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe1640ef900 con 0x7fe174102650 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.850+0000 7fe16bfff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 0x7fe1600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:14.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.850+0000 7fe16bfff640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 0x7fe1600a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe174101d30 tx=0x7fe15c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:14.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:14.854+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1640bc1c0 con 0x7fe174102650 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.033+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe174062f30 con 0x7fe174102650 2024-09-12T03:19:15.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.033+0000 7fe171ffb640 1 -- 172.21.15.19:0/1966621437 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fe1640c04c0 con 0x7fe174102650 2024-09-12T03:19:15.034 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 msgr2=0x7fe1600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 0x7fe1600a50d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe174101d30 tx=0x7fe15c00b390 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 msgr2=0x7fe174101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe164009230 tx=0x7fe164004360 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 shutdown_connections 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe1741078b0 0x7fe174100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe174104f80 0x7fe174100600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe1600a2c10 0x7fe1600a50d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 --2- 172.21.15.19:0/1966621437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe174102650 0x7fe174101fc0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 >> 172.21.15.19:0/1966621437 conn(0x7fe1740fc1b0 msgr2=0x7fe174103030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 shutdown_connections 2024-09-12T03:19:15.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.036+0000 7fe17aa1e640 1 -- 172.21.15.19:0/1966621437 wait complete. 2024-09-12T03:19:15.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:14 smithi019 ceph-mon[30953]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:15.045 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:19:15.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:14 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:19:15.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3536469812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:15.046 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2098218349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:19:15.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:14 smithi121 ceph-mon[34963]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:15.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:19:15.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:14 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:19:15.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3536469812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:15.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2098218349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 -- 172.21.15.19:0/2514577012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c101b10 msgr2=0x7f4a2c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/2514577012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c101b10 0x7f4a2c101f10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009900 tx=0x7f4a2802a410 comp rx=0 tx=0).stop 2024-09-12T03:19:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 -- 172.21.15.19:0/2514577012 shutdown_connections 2024-09-12T03:19:15.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/2514577012 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/2514577012 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c102d00 0x7f4a2c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/2514577012 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c101b10 0x7f4a2c101f10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 -- 172.21.15.19:0/2514577012 >> 172.21.15.19:0/2514577012 conn(0x7f4a2c0fd2b0 msgr2=0x7f4a2c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 -- 172.21.15.19:0/2514577012 shutdown_connections 2024-09-12T03:19:15.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.153+0000 7f4a33a1e640 1 -- 172.21.15.19:0/2514577012 wait complete. 2024-09-12T03:19:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 Processor -- start 2024-09-12T03:19:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 -- start start 2024-09-12T03:19:15.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 0x7f4a2c075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c102d00 0x7f4a2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c10d790 con 0x7f4a2c1036c0 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c10d610 con 0x7f4a2c101b10 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a33a1e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2c10d910 con 0x7f4a2c102d00 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38250/0 (socket says 172.21.15.19:38250) 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 -- 172.21.15.19:0/1745226999 learned_addr learned my addr 172.21.15.19:0/1745226999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a317ba640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 0x7f4a2c075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a30fb9640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c102d00 0x7f4a2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 -- 172.21.15.19:0/1745226999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c102d00 msgr2=0x7f4a2c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c102d00 0x7f4a2c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 -- 172.21.15.19:0/1745226999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 msgr2=0x7f4a2c075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 0x7f4a2c075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a31fbb640 1 -- 172.21.15.19:0/1745226999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a2c0753b0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.154+0000 7f4a317ba640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 0x7f4a2c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a31fbb640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400eaa0 tx=0x7f4a2400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2400cdb0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a24005020 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a2c1a1f90 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a240116b0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.155+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a2c1a23f0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.156+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a2c108f40 con 0x7f4a2c1036c0 2024-09-12T03:19:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.157+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a24005560 con 0x7f4a2c1036c0 2024-09-12T03:19:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.158+0000 7f4a227fc640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 0x7f4a180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.158+0000 7f4a317ba640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 0x7f4a180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.158+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4a240cf6b0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.158+0000 7f4a317ba640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 0x7f4a180a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006010 tx=0x7f4a28032cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:15.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.161+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a24016030 con 0x7f4a2c1036c0 2024-09-12T03:19:15.195 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:14 smithi135 ceph-mon[35474]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s 2024-09-12T03:19:15.195 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3578858501' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-12T03:19:15.196 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:14 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:19:15.196 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3536469812' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:15.196 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2098218349' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:19:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.336+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a2c062e90 con 0x7f4a2c1036c0 2024-09-12T03:19:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.337+0000 7f4a227fc640 1 -- 172.21.15.19:0/1745226999 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f4a2409c5c0 con 0x7f4a2c1036c0 2024-09-12T03:19:15.337 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 msgr2=0x7f4a180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 0x7f4a180a5050 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006010 tx=0x7f4a28032cf0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 msgr2=0x7f4a2c074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4a2400eaa0 tx=0x7f4a2400ef60 comp rx=0 tx=0).stop 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 shutdown_connections 2024-09-12T03:19:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a180a2b90 0x7f4a180a5050 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a2c1036c0 0x7f4a2c074b20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a2c102d00 0x7f4a2c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 --2- 172.21.15.19:0/1745226999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a2c101b10 0x7f4a2c075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 >> 172.21.15.19:0/1745226999 conn(0x7f4a2c0fd2b0 msgr2=0x7f4a2c106110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 shutdown_connections 2024-09-12T03:19:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:15.339+0000 7f4a33a1e640 1 -- 172.21.15.19:0/1745226999 wait complete. 2024-09-12T03:19:15.350 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T03:19:15.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:15.608 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:16.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1966621437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.057 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1745226999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1966621437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1745226999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 -- 172.21.15.19:0/2481845949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc100a80 msgr2=0x7f5ecc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 --2- 172.21.15.19:0/2481845949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc100a80 0x7f5ecc100ee0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009930 tx=0x7f5ec002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 -- 172.21.15.19:0/2481845949 shutdown_connections 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 --2- 172.21.15.19:0/2481845949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc101420 0x7f5ecc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 --2- 172.21.15.19:0/2481845949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc100a80 0x7f5ecc100ee0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 --2- 172.21.15.19:0/2481845949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 0x7f5ecc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 -- 172.21.15.19:0/2481845949 >> 172.21.15.19:0/2481845949 conn(0x7f5ecc0fc7a0 msgr2=0x7f5ecc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:16.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 -- 172.21.15.19:0/2481845949 shutdown_connections 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.141+0000 7f5ed2072640 1 -- 172.21.15.19:0/2481845949 wait complete. 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 Processor -- start 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 -- start start 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc100a80 0x7f5ecc0fcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 0x7f5ecc0fd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc110ea0 con 0x7f5ecc101420 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc110d20 con 0x7f5ecc106a80 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ed2072640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111020 con 0x7f5ecc100a80 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38268/0 (socket says 172.21.15.19:38268) 2024-09-12T03:19:16.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.142+0000 7f5ecaffd640 1 -- 172.21.15.19:0/760442209 learned_addr learned my addr 172.21.15.19:0/760442209 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecb7fe640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc100a80 0x7f5ecc0fcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecbfff640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 0x7f5ecc0fd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 -- 172.21.15.19:0/760442209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc100a80 msgr2=0x7f5ecc0fcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc100a80 0x7f5ecc0fcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 -- 172.21.15.19:0/760442209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 msgr2=0x7f5ecc0fd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 0x7f5ecc0fd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 -- 172.21.15.19:0/760442209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc1a6990 con 0x7f5ecc101420 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ecaffd640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009230 tx=0x7f5ec0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec003f030 con 0x7f5ecc101420 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc1a6c20 con 0x7f5ecc101420 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec0003ad0 con 0x7f5ecc101420 2024-09-12T03:19:16.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0035070 con 0x7f5ecc101420 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.143+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a7100 con 0x7f5ecc101420 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.144+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc100ee0 con 0x7f5ecc101420 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.145+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ec0042030 con 0x7f5ecc101420 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.146+0000 7f5ec8ff9640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 0x7f5eb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.146+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5ec00ef4e0 con 0x7f5ecc101420 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.146+0000 7f5ecb7fe640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 0x7f5eb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.147+0000 7f5ecb7fe640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 0x7f5eb40a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00bbe0 tx=0x7f5ebc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:16.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.150+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ec00bbda0 con 0x7f5ecc101420 2024-09-12T03:19:16.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1966621437' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1745226999' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:19:16.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.366+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ecc10a020 con 0x7f5ecc101420 2024-09-12T03:19:16.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.367+0000 7f5ec8ff9640 1 -- 172.21.15.19:0/760442209 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f5ec00c00a0 con 0x7f5ecc101420 2024-09-12T03:19:16.367 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:16.368 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:19:16.368 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:19:16.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 msgr2=0x7f5eb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 0x7f5eb40a5040 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc00bbe0 tx=0x7f5ebc00b390 comp rx=0 tx=0).stop 2024-09-12T03:19:16.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 msgr2=0x7f5ecc0fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:16.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009230 tx=0x7f5ec0004270 comp rx=0 tx=0).stop 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 shutdown_connections 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5ecc106a80 0x7f5ecc0fd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5eb40a2b80 0x7f5eb40a5040 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5ecc101420 0x7f5ecc0fd2f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 --2- 172.21.15.19:0/760442209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5ecc100a80 0x7f5ecc0fcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.369+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 >> 172.21.15.19:0/760442209 conn(0x7f5ecc0fc7a0 msgr2=0x7f5ecc06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.370+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 shutdown_connections 2024-09-12T03:19:16.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:16.370+0000 7f5ed2072640 1 -- 172.21.15.19:0/760442209 wait complete. 2024-09-12T03:19:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:16 smithi019 ceph-mon[30953]: pgmap v344: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:19:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:16 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/760442209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.071 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:16 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.102 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T03:19:17.102 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T03:19:17.103 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:17.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:16 smithi121 ceph-mon[34963]: pgmap v344: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:19:17.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:16 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/760442209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:16 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 -- 172.21.15.19:0/2989930401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858100f50 msgr2=0x7fe858101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 --2- 172.21.15.19:0/2989930401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858100f50 0x7fe858101350 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe848009900 tx=0x7fe84802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 -- 172.21.15.19:0/2989930401 shutdown_connections 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 --2- 172.21.15.19:0/2989930401 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858102620 0x7fe858108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 --2- 172.21.15.19:0/2989930401 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 0x7fe8581020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 --2- 172.21.15.19:0/2989930401 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858100f50 0x7fe858101350 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 -- 172.21.15.19:0/2989930401 >> 172.21.15.19:0/2989930401 conn(0x7fe8580fb950 msgr2=0x7fe8580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 -- 172.21.15.19:0/2989930401 shutdown_connections 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.216+0000 7fe85e952640 1 -- 172.21.15.19:0/2989930401 wait complete. 2024-09-12T03:19:17.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 Processor -- start 2024-09-12T03:19:17.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 -- start start 2024-09-12T03:19:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858100f50 0x7fe858100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 0x7fe85819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe85810c6b0 con 0x7fe858102620 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe85810c530 con 0x7fe858100f50 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85e952640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe85810c830 con 0x7fe858101c60 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85ceef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85ceef640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38284/0 (socket says 172.21.15.19:38284) 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.217+0000 7fe85ceef640 1 -- 172.21.15.19:0/928307098 learned_addr learned my addr 172.21.15.19:0/928307098 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe8577fe640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 0x7fe85819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe857fff640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858100f50 0x7fe858100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 -- 172.21.15.19:0/928307098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 msgr2=0x7fe85819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 0x7fe85819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 -- 172.21.15.19:0/928307098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858100f50 msgr2=0x7fe858100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858100f50 0x7fe858100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 -- 172.21.15.19:0/928307098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe85819b800 con 0x7fe858102620 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85ceef640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe850009fd0 tx=0x7fe85000def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850017070 con 0x7fe858102620 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe85819baf0 con 0x7fe858102620 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe850003e50 con 0x7fe858102620 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe850010460 con 0x7fe858102620 2024-09-12T03:19:17.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.218+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe85819c040 con 0x7fe858102620 2024-09-12T03:19:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.220+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe85819ca60 con 0x7fe858102620 2024-09-12T03:19:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.221+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe850004000 con 0x7fe858102620 2024-09-12T03:19:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.222+0000 7fe8557fa640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 0x7fe8400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.222+0000 7fe857fff640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 0x7fe8400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.222+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe8500cec70 con 0x7fe858102620 2024-09-12T03:19:17.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.222+0000 7fe857fff640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 0x7fe8400a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe848005ab0 tx=0x7fe848000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.225+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe85009bbe0 con 0x7fe858102620 2024-09-12T03:19:17.238 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:16 smithi135 ceph-mon[35474]: pgmap v344: 97 pgs: 97 active+clean; 648 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:19:17.238 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:16 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/760442209' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.238 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:16 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:19:17.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.401+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe858107c90 con 0x7fe8400a2b90 2024-09-12T03:19:17.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.407+0000 7fe8557fa640 1 -- 172.21.15.19:0/928307098 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259115 (secure 0 0 0) 0x7fe858107c90 con 0x7fe8400a2b90 2024-09-12T03:19:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:17.412 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 msgr2=0x7fe8400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 0x7fe8400a5050 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe848005ab0 tx=0x7fe848000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 msgr2=0x7fe85819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe850009fd0 tx=0x7fe85000def0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 shutdown_connections 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8400a2b90 0x7fe8400a5050 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe858102620 0x7fe85819af40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe858101c60 0x7fe85819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 --2- 172.21.15.19:0/928307098 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe858100f50 0x7fe858100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 >> 172.21.15.19:0/928307098 conn(0x7fe8580fb950 msgr2=0x7fe858105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.410+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 shutdown_connections 2024-09-12T03:19:17.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.411+0000 7fe85e952640 1 -- 172.21.15.19:0/928307098 wait complete. 2024-09-12T03:19:17.422 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-12T03:19:17.309043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1269,"num_read_kb":783,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":58,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279454+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279454+0000","last_peered":"2024-09-12T03:18:41.279454+0000","last_clean":"2024-09-12T03:18:41.279454+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279454+0000","last_undegraded":"2024-09-12T03:18:41.279454+0000","last_fullsized":"2024-09-12T03:18:41.279454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279635+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279635+0000","last_peered":"2024-09-12T03:18:41.279635+0000","last_clean":"2024-09-12T03:18:41.279635+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279635+0000","last_undegraded":"2024-09-12T03:18:41.279635+0000","last_fullsized":"2024-09-12T03:18:41.279635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281377+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.281377+0000","last_peered":"2024-09-12T03:18:41.281377+0000","last_clean":"2024-09-12T03:18:41.281377+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.281377+0000","last_undegraded":"2024-09-12T03:18:41.281377+0000","last_fullsized":"2024-09-12T03:18:41.281377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286421+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286421+0000","last_peered":"2024-09-12T03:18:41.286421+0000","last_clean":"2024-09-12T03:18:41.286421+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286421+0000","last_undegraded":"2024-09-12T03:18:41.286421+0000","last_fullsized":"2024-09-12T03:18:41.286421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281777+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.281777+0000","last_peered":"2024-09-12T03:18:41.281777+0000","last_clean":"2024-09-12T03:18:41.281777+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.281777+0000","last_undegraded":"2024-09-12T03:18:41.281777+0000","last_fullsized":"2024-09-12T03:18:41.281777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279401+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279401+0000","last_peered":"2024-09-12T03:18:41.279401+0000","last_clean":"2024-09-12T03:18:41.279401+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279401+0000","last_undegraded":"2024-09-12T03:18:41.279401+0000","last_fullsized":"2024-09-12T03:18:41.279401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272383+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272383+0000","last_peered":"2024-09-12T03:18:41.272383+0000","last_clean":"2024-09-12T03:18:41.272383+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272383+0000","last_undegraded":"2024-09-12T03:18:41.272383+0000","last_fullsized":"2024-09-12T03:18:41.272383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276667+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276667+0000","last_peered":"2024-09-12T03:18:41.276667+0000","last_clean":"2024-09-12T03:18:41.276667+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276667+0000","last_undegraded":"2024-09-12T03:18:41.276667+0000","last_fullsized":"2024-09-12T03:18:41.276667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294031+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294031+0000","last_peered":"2024-09-12T03:18:41.294031+0000","last_clean":"2024-09-12T03:18:41.294031+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294031+0000","last_undegraded":"2024-09-12T03:18:41.294031+0000","last_fullsized":"2024-09-12T03:18:41.294031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.971545+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:11.971545+0000","last_peered":"2024-09-12T03:19:11.971545+0000","last_clean":"2024-09-12T03:19:11.971545+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:11.971545+0000","last_undegraded":"2024-09-12T03:19:11.971545+0000","last_fullsized":"2024-09-12T03:19:11.971545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":256,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.356818+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:14.356818+0000","last_peered":"2024-09-12T03:19:14.356818+0000","last_clean":"2024-09-12T03:19:14.356818+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:14.356818+0000","last_undegraded":"2024-09-12T03:19:14.356818+0000","last_fullsized":"2024-09-12T03:19:14.356818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":301,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.976450+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:14.976450+0000","last_peered":"2024-09-12T03:19:14.976450+0000","last_clean":"2024-09-12T03:19:14.976450+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:14.976450+0000","last_undegraded":"2024-09-12T03:19:14.976450+0000","last_fullsized":"2024-09-12T03:19:14.976450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.977118+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:15.977118+0000","last_peered":"2024-09-12T03:19:15.977118+0000","last_clean":"2024-09-12T03:19:15.977118+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:15.977118+0000","last_undegraded":"2024-09-12T03:19:15.977118+0000","last_fullsized":"2024-09-12T03:19:15.977118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276489+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276489+0000","last_peered":"2024-09-12T03:18:41.276489+0000","last_clean":"2024-09-12T03:18:41.276489+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276489+0000","last_undegraded":"2024-09-12T03:18:41.276489+0000","last_fullsized":"2024-09-12T03:18:41.276489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278619+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.278619+0000","last_peered":"2024-09-12T03:18:41.278619+0000","last_clean":"2024-09-12T03:18:41.278619+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.278619+0000","last_undegraded":"2024-09-12T03:18:41.278619+0000","last_fullsized":"2024-09-12T03:18:41.278619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271760+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271760+0000","last_peered":"2024-09-12T03:18:41.271760+0000","last_clean":"2024-09-12T03:18:41.271760+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271760+0000","last_undegraded":"2024-09-12T03:18:41.271760+0000","last_fullsized":"2024-09-12T03:18:41.271760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.358409+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:15.358409+0000","last_peered":"2024-09-12T03:19:15.358409+0000","last_clean":"2024-09-12T03:19:15.358409+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:15.358409+0000","last_undegraded":"2024-09-12T03:19:15.358409+0000","last_fullsized":"2024-09-12T03:19:15.358409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280684+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280684+0000","last_peered":"2024-09-12T03:18:41.280684+0000","last_clean":"2024-09-12T03:18:41.280684+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280684+0000","last_undegraded":"2024-09-12T03:18:41.280684+0000","last_fullsized":"2024-09-12T03:18:41.280684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283283+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283283+0000","last_peered":"2024-09-12T03:18:41.283283+0000","last_clean":"2024-09-12T03:18:41.283283+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283283+0000","last_undegraded":"2024-09-12T03:18:41.283283+0000","last_fullsized":"2024-09-12T03:18:41.283283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.970221+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:11.970221+0000","last_peered":"2024-09-12T03:19:11.970221+0000","last_clean":"2024-09-12T03:19:11.970221+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:11.970221+0000","last_undegraded":"2024-09-12T03:19:11.970221+0000","last_fullsized":"2024-09-12T03:19:11.970221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.998969+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:15.998969+0000","last_peered":"2024-09-12T03:19:15.998969+0000","last_clean":"2024-09-12T03:19:15.998969+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:15.998969+0000","last_undegraded":"2024-09-12T03:19:15.998969+0000","last_fullsized":"2024-09-12T03:19:15.998969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282287+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282287+0000","last_peered":"2024-09-12T03:18:41.282287+0000","last_clean":"2024-09-12T03:18:41.282287+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282287+0000","last_undegraded":"2024-09-12T03:18:41.282287+0000","last_fullsized":"2024-09-12T03:18:41.282287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.927419+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:41.927419+0000","last_peered":"2024-09-12T03:18:41.927419+0000","last_clean":"2024-09-12T03:18:41.927419+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:41.927419+0000","last_undegraded":"2024-09-12T03:18:41.927419+0000","last_fullsized":"2024-09-12T03:18:41.927419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284898+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.284898+0000","last_peered":"2024-09-12T03:18:41.284898+0000","last_clean":"2024-09-12T03:18:41.284898+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.284898+0000","last_undegraded":"2024-09-12T03:18:41.284898+0000","last_fullsized":"2024-09-12T03:18:41.284898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289041+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289041+0000","last_peered":"2024-09-12T03:18:41.289041+0000","last_clean":"2024-09-12T03:18:41.289041+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289041+0000","last_undegraded":"2024-09-12T03:18:41.289041+0000","last_fullsized":"2024-09-12T03:18:41.289041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.244249+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.244249+0000","last_peered":"2024-09-12T03:18:41.244249+0000","last_clean":"2024-09-12T03:18:41.244249+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.244249+0000","last_undegraded":"2024-09-12T03:18:41.244249+0000","last_fullsized":"2024-09-12T03:18:41.244249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285191+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285191+0000","last_peered":"2024-09-12T03:18:41.285191+0000","last_clean":"2024-09-12T03:18:41.285191+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285191+0000","last_undegraded":"2024-09-12T03:18:41.285191+0000","last_fullsized":"2024-09-12T03:18:41.285191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278354+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.278354+0000","last_peered":"2024-09-12T03:18:41.278354+0000","last_clean":"2024-09-12T03:18:41.278354+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.278354+0000","last_undegraded":"2024-09-12T03:18:41.278354+0000","last_fullsized":"2024-09-12T03:18:41.278354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282822+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282822+0000","last_peered":"2024-09-12T03:18:41.282822+0000","last_clean":"2024-09-12T03:18:41.282822+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282822+0000","last_undegraded":"2024-09-12T03:18:41.282822+0000","last_fullsized":"2024-09-12T03:18:41.282822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.997184+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:14.997184+0000","last_peered":"2024-09-12T03:19:14.997184+0000","last_clean":"2024-09-12T03:19:14.997184+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:14.997184+0000","last_undegraded":"2024-09-12T03:19:14.997184+0000","last_fullsized":"2024-09-12T03:19:14.997184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1143,"num_read_kb":674,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579944,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:17.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 -- 172.21.15.19:0/3139175811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7870104f80 msgr2=0x7f7870107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 --2- 172.21.15.19:0/3139175811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7870104f80 0x7f7870107370 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7860009ca0 tx=0x7f786002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 -- 172.21.15.19:0/3139175811 shutdown_connections 2024-09-12T03:19:17.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 --2- 172.21.15.19:0/3139175811 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f78701078b0 0x7f7870109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 --2- 172.21.15.19:0/3139175811 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7870104f80 0x7f7870107370 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 --2- 172.21.15.19:0/3139175811 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870102650 0x7f7870104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 -- 172.21.15.19:0/3139175811 >> 172.21.15.19:0/3139175811 conn(0x7f78700fc170 msgr2=0x7f78700fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 -- 172.21.15.19:0/3139175811 shutdown_connections 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.571+0000 7f7877250640 1 -- 172.21.15.19:0/3139175811 wait complete. 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 Processor -- start 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 -- start start 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 0x7f7870197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104f80 0x7f78701983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d7e0 con 0x7f78701078b0 2024-09-12T03:19:17.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d660 con 0x7f7870104f80 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7877250640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f787010d960 con 0x7f7870102650 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38302/0 (socket says 172.21.15.19:38302) 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 -- 172.21.15.19:0/114461003 learned_addr learned my addr 172.21.15.19:0/114461003 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7874fec640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 0x7f7870197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f786ffff640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104f80 0x7f78701983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 -- 172.21.15.19:0/114461003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 msgr2=0x7f7870197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 0x7f7870197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 -- 172.21.15.19:0/114461003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104f80 msgr2=0x7f78701983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104f80 0x7f78701983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f78757ed640 1 -- 172.21.15.19:0/114461003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f787019fa90 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.572+0000 7f7874fec640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 0x7f7870197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f78757ed640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f786800e970 tx=0x7f786800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786800ccc0 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f786800ce60 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f787019fd80 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7868010490 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.573+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78701a02a0 con 0x7f78701078b0 2024-09-12T03:19:17.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.574+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78700fdc60 con 0x7f78701078b0 2024-09-12T03:19:17.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.575+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7868005020 con 0x7f78701078b0 2024-09-12T03:19:17.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.576+0000 7f786dffb640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 0x7f785c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.576+0000 7f7874fec640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 0x7f785c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.576+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f78680cf4f0 con 0x7f78701078b0 2024-09-12T03:19:17.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.576+0000 7f7874fec640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 0x7f785c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7858009fc0 tx=0x7f7858005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.580+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f786809c400 con 0x7f78701078b0 2024-09-12T03:19:17.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.752+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7870062630 con 0x7f785c0a2c10 2024-09-12T03:19:17.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.758+0000 7f786dffb640 1 -- 172.21.15.19:0/114461003 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259115 (secure 0 0 0) 0x7f7870062630 con 0x7f785c0a2c10 2024-09-12T03:19:17.761 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:17.762 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 msgr2=0x7f785c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 0x7f785c0a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7858009fc0 tx=0x7f7858005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 msgr2=0x7f787019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f786800e970 tx=0x7f786800ed70 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 shutdown_connections 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f785c0a2c10 0x7f785c0a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f78701078b0 0x7f787019f420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7870104f80 0x7f78701983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 --2- 172.21.15.19:0/114461003 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7870102650 0x7f7870197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 >> 172.21.15.19:0/114461003 conn(0x7f78700fc170 msgr2=0x7f78701030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 shutdown_connections 2024-09-12T03:19:17.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.761+0000 7f7877250640 1 -- 172.21.15.19:0/114461003 wait complete. 2024-09-12T03:19:17.772 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-12T03:19:17.309043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1269,"num_read_kb":783,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":58,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279454+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279454+0000","last_peered":"2024-09-12T03:18:41.279454+0000","last_clean":"2024-09-12T03:18:41.279454+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279454+0000","last_undegraded":"2024-09-12T03:18:41.279454+0000","last_fullsized":"2024-09-12T03:18:41.279454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279635+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279635+0000","last_peered":"2024-09-12T03:18:41.279635+0000","last_clean":"2024-09-12T03:18:41.279635+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279635+0000","last_undegraded":"2024-09-12T03:18:41.279635+0000","last_fullsized":"2024-09-12T03:18:41.279635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281377+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.281377+0000","last_peered":"2024-09-12T03:18:41.281377+0000","last_clean":"2024-09-12T03:18:41.281377+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.281377+0000","last_undegraded":"2024-09-12T03:18:41.281377+0000","last_fullsized":"2024-09-12T03:18:41.281377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286421+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286421+0000","last_peered":"2024-09-12T03:18:41.286421+0000","last_clean":"2024-09-12T03:18:41.286421+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286421+0000","last_undegraded":"2024-09-12T03:18:41.286421+0000","last_fullsized":"2024-09-12T03:18:41.286421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281777+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.281777+0000","last_peered":"2024-09-12T03:18:41.281777+0000","last_clean":"2024-09-12T03:18:41.281777+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.281777+0000","last_undegraded":"2024-09-12T03:18:41.281777+0000","last_fullsized":"2024-09-12T03:18:41.281777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279401+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279401+0000","last_peered":"2024-09-12T03:18:41.279401+0000","last_clean":"2024-09-12T03:18:41.279401+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279401+0000","last_undegraded":"2024-09-12T03:18:41.279401+0000","last_fullsized":"2024-09-12T03:18:41.279401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272383+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272383+0000","last_peered":"2024-09-12T03:18:41.272383+0000","last_clean":"2024-09-12T03:18:41.272383+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272383+0000","last_undegraded":"2024-09-12T03:18:41.272383+0000","last_fullsized":"2024-09-12T03:18:41.272383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276667+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276667+0000","last_peered":"2024-09-12T03:18:41.276667+0000","last_clean":"2024-09-12T03:18:41.276667+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276667+0000","last_undegraded":"2024-09-12T03:18:41.276667+0000","last_fullsized":"2024-09-12T03:18:41.276667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294031+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294031+0000","last_peered":"2024-09-12T03:18:41.294031+0000","last_clean":"2024-09-12T03:18:41.294031+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294031+0000","last_undegraded":"2024-09-12T03:18:41.294031+0000","last_fullsized":"2024-09-12T03:18:41.294031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.971545+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:11.971545+0000","last_peered":"2024-09-12T03:19:11.971545+0000","last_clean":"2024-09-12T03:19:11.971545+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:11.971545+0000","last_undegraded":"2024-09-12T03:19:11.971545+0000","last_fullsized":"2024-09-12T03:19:11.971545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":256,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.356818+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:14.356818+0000","last_peered":"2024-09-12T03:19:14.356818+0000","last_clean":"2024-09-12T03:19:14.356818+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:14.356818+0000","last_undegraded":"2024-09-12T03:19:14.356818+0000","last_fullsized":"2024-09-12T03:19:14.356818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":301,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.976450+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:14.976450+0000","last_peered":"2024-09-12T03:19:14.976450+0000","last_clean":"2024-09-12T03:19:14.976450+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:14.976450+0000","last_undegraded":"2024-09-12T03:19:14.976450+0000","last_fullsized":"2024-09-12T03:19:14.976450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.977118+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:15.977118+0000","last_peered":"2024-09-12T03:19:15.977118+0000","last_clean":"2024-09-12T03:19:15.977118+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:15.977118+0000","last_undegraded":"2024-09-12T03:19:15.977118+0000","last_fullsized":"2024-09-12T03:19:15.977118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276489+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276489+0000","last_peered":"2024-09-12T03:18:41.276489+0000","last_clean":"2024-09-12T03:18:41.276489+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276489+0000","last_undegraded":"2024-09-12T03:18:41.276489+0000","last_fullsized":"2024-09-12T03:18:41.276489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278619+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.278619+0000","last_peered":"2024-09-12T03:18:41.278619+0000","last_clean":"2024-09-12T03:18:41.278619+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.278619+0000","last_undegraded":"2024-09-12T03:18:41.278619+0000","last_fullsized":"2024-09-12T03:18:41.278619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271760+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271760+0000","last_peered":"2024-09-12T03:18:41.271760+0000","last_clean":"2024-09-12T03:18:41.271760+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271760+0000","last_undegraded":"2024-09-12T03:18:41.271760+0000","last_fullsized":"2024-09-12T03:18:41.271760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.358409+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:15.358409+0000","last_peered":"2024-09-12T03:19:15.358409+0000","last_clean":"2024-09-12T03:19:15.358409+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:15.358409+0000","last_undegraded":"2024-09-12T03:19:15.358409+0000","last_fullsized":"2024-09-12T03:19:15.358409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280684+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280684+0000","last_peered":"2024-09-12T03:18:41.280684+0000","last_clean":"2024-09-12T03:18:41.280684+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280684+0000","last_undegraded":"2024-09-12T03:18:41.280684+0000","last_fullsized":"2024-09-12T03:18:41.280684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283283+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283283+0000","last_peered":"2024-09-12T03:18:41.283283+0000","last_clean":"2024-09-12T03:18:41.283283+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283283+0000","last_undegraded":"2024-09-12T03:18:41.283283+0000","last_fullsized":"2024-09-12T03:18:41.283283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.970221+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:11.970221+0000","last_peered":"2024-09-12T03:19:11.970221+0000","last_clean":"2024-09-12T03:19:11.970221+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:11.970221+0000","last_undegraded":"2024-09-12T03:19:11.970221+0000","last_fullsized":"2024-09-12T03:19:11.970221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.998969+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:15.998969+0000","last_peered":"2024-09-12T03:19:15.998969+0000","last_clean":"2024-09-12T03:19:15.998969+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:15.998969+0000","last_undegraded":"2024-09-12T03:19:15.998969+0000","last_fullsized":"2024-09-12T03:19:15.998969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282287+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282287+0000","last_peered":"2024-09-12T03:18:41.282287+0000","last_clean":"2024-09-12T03:18:41.282287+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282287+0000","last_undegraded":"2024-09-12T03:18:41.282287+0000","last_fullsized":"2024-09-12T03:18:41.282287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.927419+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:41.927419+0000","last_peered":"2024-09-12T03:18:41.927419+0000","last_clean":"2024-09-12T03:18:41.927419+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:41.927419+0000","last_undegraded":"2024-09-12T03:18:41.927419+0000","last_fullsized":"2024-09-12T03:18:41.927419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284898+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.284898+0000","last_peered":"2024-09-12T03:18:41.284898+0000","last_clean":"2024-09-12T03:18:41.284898+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.284898+0000","last_undegraded":"2024-09-12T03:18:41.284898+0000","last_fullsized":"2024-09-12T03:18:41.284898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289041+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289041+0000","last_peered":"2024-09-12T03:18:41.289041+0000","last_clean":"2024-09-12T03:18:41.289041+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289041+0000","last_undegraded":"2024-09-12T03:18:41.289041+0000","last_fullsized":"2024-09-12T03:18:41.289041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.244249+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.244249+0000","last_peered":"2024-09-12T03:18:41.244249+0000","last_clean":"2024-09-12T03:18:41.244249+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.244249+0000","last_undegraded":"2024-09-12T03:18:41.244249+0000","last_fullsized":"2024-09-12T03:18:41.244249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285191+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285191+0000","last_peered":"2024-09-12T03:18:41.285191+0000","last_clean":"2024-09-12T03:18:41.285191+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285191+0000","last_undegraded":"2024-09-12T03:18:41.285191+0000","last_fullsized":"2024-09-12T03:18:41.285191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278354+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.278354+0000","last_peered":"2024-09-12T03:18:41.278354+0000","last_clean":"2024-09-12T03:18:41.278354+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.278354+0000","last_undegraded":"2024-09-12T03:18:41.278354+0000","last_fullsized":"2024-09-12T03:18:41.278354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282822+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282822+0000","last_peered":"2024-09-12T03:18:41.282822+0000","last_clean":"2024-09-12T03:18:41.282822+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282822+0000","last_undegraded":"2024-09-12T03:18:41.282822+0000","last_fullsized":"2024-09-12T03:18:41.282822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.997184+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:14.997184+0000","last_peered":"2024-09-12T03:19:14.997184+0000","last_clean":"2024-09-12T03:19:14.997184+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:14.997184+0000","last_undegraded":"2024-09-12T03:19:14.997184+0000","last_fullsized":"2024-09-12T03:19:14.997184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1143,"num_read_kb":674,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579944,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:17.776 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T03:19:17.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T03:19:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 -- 172.21.15.19:0/2949057805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 msgr2=0x7fbaa4109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/2949057805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa4109f70 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009900 tx=0x7fba9c02a400 comp rx=0 tx=0).stop 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 -- 172.21.15.19:0/2949057805 shutdown_connections 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/2949057805 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa4109f70 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/2949057805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4102d20 0x7fbaa41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/2949057805 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4101b30 0x7fbaa4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 -- 172.21.15.19:0/2949057805 >> 172.21.15.19:0/2949057805 conn(0x7fbaa40fd2b0 msgr2=0x7fbaa40ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 -- 172.21.15.19:0/2949057805 shutdown_connections 2024-09-12T03:19:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.923+0000 7fbaa9cba640 1 -- 172.21.15.19:0/2949057805 wait complete. 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 Processor -- start 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 -- start start 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 0x7fbaa4197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 0x7fbaa4198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa419f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d7e0 con 0x7fbaa41036e0 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d660 con 0x7fbaa4101b30 2024-09-12T03:19:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa9cba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa410d960 con 0x7fbaa4102d20 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.924+0000 7fbaa37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 0x7fbaa4197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 0x7fbaa4198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 0x7fbaa4198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:37276/0 (socket says 172.21.15.19:37276) 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa2ffd640 1 -- 172.21.15.19:0/3547614014 learned_addr learned my addr 172.21.15.19:0/3547614014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa419f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 -- 172.21.15.19:0/3547614014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 msgr2=0x7fbaa4198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 0x7fbaa4198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 -- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 msgr2=0x7fbaa419f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa419f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 -- 172.21.15.19:0/3547614014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa419fa00 con 0x7fbaa4101b30 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.925+0000 7fbaa37fe640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 0x7fbaa4197de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba94005fd0 tx=0x7fba9400ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.926+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9400cde0 con 0x7fbaa4101b30 2024-09-12T03:19:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.926+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba94011950 con 0x7fbaa4101b30 2024-09-12T03:19:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.926+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9401aa80 con 0x7fbaa4101b30 2024-09-12T03:19:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.926+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa419fcf0 con 0x7fbaa4101b30 2024-09-12T03:19:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.926+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a01d0 con 0x7fbaa4101b30 2024-09-12T03:19:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.927+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4108e50 con 0x7fbaa4101b30 2024-09-12T03:19:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.928+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba94011b00 con 0x7fbaa4101b30 2024-09-12T03:19:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.929+0000 7fbaa0ff9640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 0x7fba900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.930+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fba940cf540 con 0x7fbaa4101b30 2024-09-12T03:19:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.930+0000 7fbaa2ffd640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 0x7fba900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.930+0000 7fbaa2ffd640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 0x7fba900a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbaa41992f0 tx=0x7fba8c00b4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:17.935+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba94016030 con 0x7fbaa4101b30 2024-09-12T03:19:18.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.123+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbaa4062630 con 0x7fba900a2c10 2024-09-12T03:19:18.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.134+0000 7fbaa0ff9640 1 -- 172.21.15.19:0/3547614014 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259115 (secure 0 0 0) 0x7fbaa4062630 con 0x7fba900a2c10 2024-09-12T03:19:18.137 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:18.138 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 msgr2=0x7fba900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 0x7fba900a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbaa41992f0 tx=0x7fba8c00b4a0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 msgr2=0x7fbaa4197de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 0x7fbaa4197de0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fba94005fd0 tx=0x7fba9400ef30 comp rx=0 tx=0).stop 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 shutdown_connections 2024-09-12T03:19:18.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba900a2c10 0x7fba900a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbaa41036e0 0x7fbaa419f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbaa4102d20 0x7fbaa4198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 --2- 172.21.15.19:0/3547614014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbaa4101b30 0x7fbaa4197de0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 >> 172.21.15.19:0/3547614014 conn(0x7fbaa40fd2b0 msgr2=0x7fbaa4106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 shutdown_connections 2024-09-12T03:19:18.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.137+0000 7fbaa9cba640 1 -- 172.21.15.19:0/3547614014 wait complete. 2024-09-12T03:19:18.150 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-12T03:19:17.309043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1269,"num_read_kb":783,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":58,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002949"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279454+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279454+0000","last_peered":"2024-09-12T03:18:41.279454+0000","last_clean":"2024-09-12T03:18:41.279454+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279454+0000","last_undegraded":"2024-09-12T03:18:41.279454+0000","last_fullsized":"2024-09-12T03:18:41.279454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279635+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279635+0000","last_peered":"2024-09-12T03:18:41.279635+0000","last_clean":"2024-09-12T03:18:41.279635+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279635+0000","last_undegraded":"2024-09-12T03:18:41.279635+0000","last_fullsized":"2024-09-12T03:18:41.279635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281377+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.281377+0000","last_peered":"2024-09-12T03:18:41.281377+0000","last_clean":"2024-09-12T03:18:41.281377+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.281377+0000","last_undegraded":"2024-09-12T03:18:41.281377+0000","last_fullsized":"2024-09-12T03:18:41.281377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286421+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286421+0000","last_peered":"2024-09-12T03:18:41.286421+0000","last_clean":"2024-09-12T03:18:41.286421+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286421+0000","last_undegraded":"2024-09-12T03:18:41.286421+0000","last_fullsized":"2024-09-12T03:18:41.286421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281777+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.281777+0000","last_peered":"2024-09-12T03:18:41.281777+0000","last_clean":"2024-09-12T03:18:41.281777+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.281777+0000","last_undegraded":"2024-09-12T03:18:41.281777+0000","last_fullsized":"2024-09-12T03:18:41.281777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279401+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279401+0000","last_peered":"2024-09-12T03:18:41.279401+0000","last_clean":"2024-09-12T03:18:41.279401+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279401+0000","last_undegraded":"2024-09-12T03:18:41.279401+0000","last_fullsized":"2024-09-12T03:18:41.279401+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272383+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272383+0000","last_peered":"2024-09-12T03:18:41.272383+0000","last_clean":"2024-09-12T03:18:41.272383+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272383+0000","last_undegraded":"2024-09-12T03:18:41.272383+0000","last_fullsized":"2024-09-12T03:18:41.272383+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276667+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276667+0000","last_peered":"2024-09-12T03:18:41.276667+0000","last_clean":"2024-09-12T03:18:41.276667+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276667+0000","last_undegraded":"2024-09-12T03:18:41.276667+0000","last_fullsized":"2024-09-12T03:18:41.276667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294031+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294031+0000","last_peered":"2024-09-12T03:18:41.294031+0000","last_clean":"2024-09-12T03:18:41.294031+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294031+0000","last_undegraded":"2024-09-12T03:18:41.294031+0000","last_fullsized":"2024-09-12T03:18:41.294031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.971545+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:11.971545+0000","last_peered":"2024-09-12T03:19:11.971545+0000","last_clean":"2024-09-12T03:19:11.971545+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:11.971545+0000","last_undegraded":"2024-09-12T03:19:11.971545+0000","last_fullsized":"2024-09-12T03:19:11.971545+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":256,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.356818+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:14.356818+0000","last_peered":"2024-09-12T03:19:14.356818+0000","last_clean":"2024-09-12T03:19:14.356818+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:14.356818+0000","last_undegraded":"2024-09-12T03:19:14.356818+0000","last_fullsized":"2024-09-12T03:19:14.356818+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":301,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.976450+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:14.976450+0000","last_peered":"2024-09-12T03:19:14.976450+0000","last_clean":"2024-09-12T03:19:14.976450+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:14.976450+0000","last_undegraded":"2024-09-12T03:19:14.976450+0000","last_fullsized":"2024-09-12T03:19:14.976450+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":243,"num_read_kb":186,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.977118+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:15.977118+0000","last_peered":"2024-09-12T03:19:15.977118+0000","last_clean":"2024-09-12T03:19:15.977118+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:15.977118+0000","last_undegraded":"2024-09-12T03:19:15.977118+0000","last_fullsized":"2024-09-12T03:19:15.977118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276489+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276489+0000","last_peered":"2024-09-12T03:18:41.276489+0000","last_clean":"2024-09-12T03:18:41.276489+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276489+0000","last_undegraded":"2024-09-12T03:18:41.276489+0000","last_fullsized":"2024-09-12T03:18:41.276489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278619+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.278619+0000","last_peered":"2024-09-12T03:18:41.278619+0000","last_clean":"2024-09-12T03:18:41.278619+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.278619+0000","last_undegraded":"2024-09-12T03:18:41.278619+0000","last_fullsized":"2024-09-12T03:18:41.278619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271760+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271760+0000","last_peered":"2024-09-12T03:18:41.271760+0000","last_clean":"2024-09-12T03:18:41.271760+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271760+0000","last_undegraded":"2024-09-12T03:18:41.271760+0000","last_fullsized":"2024-09-12T03:18:41.271760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.358409+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:15.358409+0000","last_peered":"2024-09-12T03:19:15.358409+0000","last_clean":"2024-09-12T03:19:15.358409+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:15.358409+0000","last_undegraded":"2024-09-12T03:19:15.358409+0000","last_fullsized":"2024-09-12T03:19:15.358409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280684+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280684+0000","last_peered":"2024-09-12T03:18:41.280684+0000","last_clean":"2024-09-12T03:18:41.280684+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280684+0000","last_undegraded":"2024-09-12T03:18:41.280684+0000","last_fullsized":"2024-09-12T03:18:41.280684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283283+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283283+0000","last_peered":"2024-09-12T03:18:41.283283+0000","last_clean":"2024-09-12T03:18:41.283283+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283283+0000","last_undegraded":"2024-09-12T03:18:41.283283+0000","last_fullsized":"2024-09-12T03:18:41.283283+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:11.970221+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:11.970221+0000","last_peered":"2024-09-12T03:19:11.970221+0000","last_clean":"2024-09-12T03:19:11.970221+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:11.970221+0000","last_undegraded":"2024-09-12T03:19:11.970221+0000","last_fullsized":"2024-09-12T03:19:11.970221+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.998969+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:15.998969+0000","last_peered":"2024-09-12T03:19:15.998969+0000","last_clean":"2024-09-12T03:19:15.998969+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:15.998969+0000","last_undegraded":"2024-09-12T03:19:15.998969+0000","last_fullsized":"2024-09-12T03:19:15.998969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282287+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282287+0000","last_peered":"2024-09-12T03:18:41.282287+0000","last_clean":"2024-09-12T03:18:41.282287+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282287+0000","last_undegraded":"2024-09-12T03:18:41.282287+0000","last_fullsized":"2024-09-12T03:18:41.282287+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.927419+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:18:41.927419+0000","last_peered":"2024-09-12T03:18:41.927419+0000","last_clean":"2024-09-12T03:18:41.927419+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:18:41.927419+0000","last_undegraded":"2024-09-12T03:18:41.927419+0000","last_fullsized":"2024-09-12T03:18:41.927419+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284898+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.284898+0000","last_peered":"2024-09-12T03:18:41.284898+0000","last_clean":"2024-09-12T03:18:41.284898+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.284898+0000","last_undegraded":"2024-09-12T03:18:41.284898+0000","last_fullsized":"2024-09-12T03:18:41.284898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289041+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289041+0000","last_peered":"2024-09-12T03:18:41.289041+0000","last_clean":"2024-09-12T03:18:41.289041+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289041+0000","last_undegraded":"2024-09-12T03:18:41.289041+0000","last_fullsized":"2024-09-12T03:18:41.289041+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.244249+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.244249+0000","last_peered":"2024-09-12T03:18:41.244249+0000","last_clean":"2024-09-12T03:18:41.244249+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.244249+0000","last_undegraded":"2024-09-12T03:18:41.244249+0000","last_fullsized":"2024-09-12T03:18:41.244249+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227672+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227672+0000","last_peered":"2024-09-12T03:18:13.227672+0000","last_clean":"2024-09-12T03:18:13.227672+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227672+0000","last_undegraded":"2024-09-12T03:18:13.227672+0000","last_fullsized":"2024-09-12T03:18:13.227672+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285191+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285191+0000","last_peered":"2024-09-12T03:18:41.285191+0000","last_clean":"2024-09-12T03:18:41.285191+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285191+0000","last_undegraded":"2024-09-12T03:18:41.285191+0000","last_fullsized":"2024-09-12T03:18:41.285191+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221383+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221383+0000","last_peered":"2024-09-12T03:18:13.221383+0000","last_clean":"2024-09-12T03:18:13.221383+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221383+0000","last_undegraded":"2024-09-12T03:18:13.221383+0000","last_fullsized":"2024-09-12T03:18:13.221383+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278354+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.278354+0000","last_peered":"2024-09-12T03:18:41.278354+0000","last_clean":"2024-09-12T03:18:41.278354+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.278354+0000","last_undegraded":"2024-09-12T03:18:41.278354+0000","last_fullsized":"2024-09-12T03:18:41.278354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282822+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282822+0000","last_peered":"2024-09-12T03:18:41.282822+0000","last_clean":"2024-09-12T03:18:41.282822+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282822+0000","last_undegraded":"2024-09-12T03:18:41.282822+0000","last_fullsized":"2024-09-12T03:18:41.282822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:14.997184+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:14.997184+0000","last_peered":"2024-09-12T03:19:14.997184+0000","last_clean":"2024-09-12T03:19:14.997184+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:14.997184+0000","last_undegraded":"2024-09-12T03:19:14.997184+0000","last_fullsized":"2024-09-12T03:19:14.997184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1143,"num_read_kb":674,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579944,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972401,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234039,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626497,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855430,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444118,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:18.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:18.464 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.963+0000 7fad58960640 1 -- 172.21.15.19:0/3592760255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 msgr2=0x7fad54103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.963+0000 7fad58960640 1 --2- 172.21.15.19:0/3592760255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad54103170 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fad44009930 tx=0x7fad4402a420 comp rx=0 tx=0).stop 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.963+0000 7fad58960640 1 -- 172.21.15.19:0/3592760255 shutdown_connections 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.963+0000 7fad58960640 1 --2- 172.21.15.19:0/3592760255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54108290 0x7fad5410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 --2- 172.21.15.19:0/3592760255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54103740 0x7fad54107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 --2- 172.21.15.19:0/3592760255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad54103170 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 -- 172.21.15.19:0/3592760255 >> 172.21.15.19:0/3592760255 conn(0x7fad540fc7d0 msgr2=0x7fad540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 -- 172.21.15.19:0/3592760255 shutdown_connections 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 -- 172.21.15.19:0/3592760255 wait complete. 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 Processor -- start 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.964+0000 7fad58960640 1 -- start start 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54103740 0x7fad54100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54108290 0x7fad54101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad54112090 con 0x7fad54102d90 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad54111f10 con 0x7fad54108290 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad54112210 con 0x7fad54103740 2024-09-12T03:19:18.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38350/0 (socket says 172.21.15.19:38350) 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 -- 172.21.15.19:0/4255283673 learned_addr learned my addr 172.21.15.19:0/4255283673 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad51d9b640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54103740 0x7fad54100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad52d9d640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54108290 0x7fad54101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 -- 172.21.15.19:0/4255283673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54103740 msgr2=0x7fad54100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54103740 0x7fad54100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 -- 172.21.15.19:0/4255283673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54108290 msgr2=0x7fad54101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54108290 0x7fad54101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 -- 172.21.15.19:0/4255283673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad541a6a60 con 0x7fad54102d90 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad5259c640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad44004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad4403f030 con 0x7fad54102d90 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad541a6cf0 con 0x7fad54102d90 2024-09-12T03:19:18.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.965+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad44004440 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.966+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad44035070 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.966+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad541a71c0 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.967+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad5406a7f0 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.968+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad44042030 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.969+0000 7fad437fe640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 0x7fad3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.969+0000 7fad51d9b640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 0x7fad3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.969+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fad440ef3c0 con 0x7fad54102d90 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.969+0000 7fad51d9b640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 0x7fad3c0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad54102480 tx=0x7fad48005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:18.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:18.973+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad440bc330 con 0x7fad54102d90 2024-09-12T03:19:19.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T03:19:19.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:19.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.186+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad5410a2a0 con 0x7fad54102d90 2024-09-12T03:19:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.187+0000 7fad437fe640 1 -- 172.21.15.19:0/4255283673 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7fad440bbc30 con 0x7fad54102d90 2024-09-12T03:19:19.188 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:19.188 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:19:19.188 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:19:19.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 msgr2=0x7fad3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 0x7fad3c0a5040 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fad54102480 tx=0x7fad48005f90 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 msgr2=0x7fad541009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fad44009230 tx=0x7fad44004040 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 shutdown_connections 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad54108290 0x7fad54101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad54103740 0x7fad54100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad3c0a2b80 0x7fad3c0a5040 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 --2- 172.21.15.19:0/4255283673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad54102d90 0x7fad541009f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 >> 172.21.15.19:0/4255283673 conn(0x7fad540fc7d0 msgr2=0x7fad54105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 shutdown_connections 2024-09-12T03:19:19.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:19.189+0000 7fad58960640 1 -- 172.21.15.19:0/4255283673 wait complete. 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:19.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:19.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: pgmap v345: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='client.15252 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='client.24890 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:19:19.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:19:19.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:20.106 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:20.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:20 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4255283673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:20.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:20 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4255283673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:20.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:20 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4255283673' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:20.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 -- 172.21.15.19:0/3558031852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102570 msgr2=0x7f47001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 --2- 172.21.15.19:0/3558031852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102570 0x7f47001029b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009900 tx=0x7f46e802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 -- 172.21.15.19:0/3558031852 shutdown_connections 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 --2- 172.21.15.19:0/3558031852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700102ef0 0x7f470010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 --2- 172.21.15.19:0/3558031852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102570 0x7f47001029b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 --2- 172.21.15.19:0/3558031852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700100e20 0x7f4700101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 -- 172.21.15.19:0/3558031852 >> 172.21.15.19:0/3558031852 conn(0x7f47000fc7d0 msgr2=0x7f47000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 -- 172.21.15.19:0/3558031852 shutdown_connections 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.623+0000 7f470636b640 1 -- 172.21.15.19:0/3558031852 wait complete. 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 Processor -- start 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 -- start start 2024-09-12T03:19:20.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700100e20 0x7f47001993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700102570 0x7f4700199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4700110fa0 con 0x7f4700102ef0 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4700110e20 con 0x7f4700100e20 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f470636b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4700111120 con 0x7f4700102570 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34298/0 (socket says 172.21.15.19:34298) 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 -- 172.21.15.19:0/1238176688 learned_addr learned my addr 172.21.15.19:0/1238176688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f46fffff640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700100e20 0x7f47001993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f46f7fff640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700102570 0x7f4700199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 -- 172.21.15.19:0/1238176688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700102570 msgr2=0x7f4700199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700102570 0x7f4700199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 -- 172.21.15.19:0/1238176688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700100e20 msgr2=0x7f47001993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700100e20 0x7f47001993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.624+0000 7f4704908640 1 -- 172.21.15.19:0/1238176688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47001a49b0 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f4704908640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f46f800eaa0 tx=0x7f46f800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8004d20 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47001a4b80 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f8005020 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80116d0 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.625+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47001a50c0 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.626+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4700101200 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.627+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46f80051d0 con 0x7f4700102ef0 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.628+0000 7f46fdffb640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 0x7f46ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:20.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.628+0000 7f46fffff640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 0x7f46ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.628+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f46f80cf630 con 0x7f4700102ef0 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.628+0000 7f46fffff640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 0x7f46ec0a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009990 tx=0x7f46f0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:20.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.631+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f46f8016030 con 0x7f4700102ef0 2024-09-12T03:19:20.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.809+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4700109090 con 0x7f4700102ef0 2024-09-12T03:19:20.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.811+0000 7f46fdffb640 1 -- 172.21.15.19:0/1238176688 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7f46f809c5a0 con 0x7f4700102ef0 2024-09-12T03:19:20.811 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:20.812 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 msgr2=0x7f46ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 0x7f46ec0a5040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009990 tx=0x7f46f0009110 comp rx=0 tx=0).stop 2024-09-12T03:19:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 msgr2=0x7f47001a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f46f800eaa0 tx=0x7f46f800eea0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 shutdown_connections 2024-09-12T03:19:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f46ec0a2b80 0x7f46ec0a5040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4700102ef0 0x7f47001a4470 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4700102570 0x7f4700199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 --2- 172.21.15.19:0/1238176688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4700100e20 0x7f47001993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 >> 172.21.15.19:0/1238176688 conn(0x7f47000fc7d0 msgr2=0x7f47000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 shutdown_connections 2024-09-12T03:19:20.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:20.813+0000 7f470636b640 1 -- 172.21.15.19:0/1238176688 wait complete. 2024-09-12T03:19:21.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:21 smithi019 ceph-mon[30953]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-12T03:19:21.391 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1238176688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:21 smithi121 ceph-mon[34963]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-12T03:19:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1238176688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:21.494 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T03:19:21.494 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T03:19:21.508 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T03:19:21.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:19:21.509 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T03:19:21.509 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:19:21.509 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:19:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:21 smithi135 ceph-mon[35474]: pgmap v346: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-12T03:19:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1238176688' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:21.778 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3409357273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac101de0 msgr2=0x7fb4ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3409357273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1021c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009900 tx=0x7fb4a802a410 comp rx=0 tx=0).stop 2024-09-12T03:19:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3409357273 shutdown_connections 2024-09-12T03:19:22.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3409357273 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 0x7fb4ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3409357273 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac102790 0x7fb4ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3409357273 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac101de0 0x7fb4ac1021c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.493+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3409357273 >> 172.21.15.19:0/3409357273 conn(0x7fb4ac0fd8c0 msgr2=0x7fb4ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.494+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3409357273 shutdown_connections 2024-09-12T03:19:22.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.494+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3409357273 wait complete. 2024-09-12T03:19:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 Processor -- start 2024-09-12T03:19:22.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 -- start start 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac101de0 0x7fb4ac198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111f80 con 0x7fb4ac102790 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac111e00 con 0x7fb4ac101de0 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b3f77640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac112100 con 0x7fb4ac106f10 2024-09-12T03:19:22.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b1512640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b1512640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34330/0 (socket says 172.21.15.19:34330) 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b1512640 1 -- 172.21.15.19:0/3647076378 learned_addr learned my addr 172.21.15.19:0/3647076378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac101de0 0x7fb4ac198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.495+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 -- 172.21.15.19:0/3647076378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 msgr2=0x7fb4ac19cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 -- 172.21.15.19:0/3647076378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac101de0 msgr2=0x7fb4ac198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac101de0 0x7fb4ac198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 -- 172.21.15.19:0/3647076378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac19d410 con 0x7fb4ac102790 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b1512640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c890 tx=0x7fb49c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4a2ffd640 1 -- 172.21.15.19:0/3647076378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c004220 con 0x7fb4ac102790 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4a2ffd640 1 -- 172.21.15.19:0/3647076378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb49c0043c0 con 0x7fb4ac102790 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac19d700 con 0x7fb4ac102790 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4a2ffd640 1 -- 172.21.15.19:0/3647076378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0049e0 con 0x7fb4ac102790 2024-09-12T03:19:22.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.496+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a4fe0 con 0x7fb4ac102790 2024-09-12T03:19:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.499+0000 7fb4a2ffd640 1 -- 172.21.15.19:0/3647076378 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb49c0079b0 con 0x7fb4ac102790 2024-09-12T03:19:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.499+0000 7fb4b6350740 1 Processor -- start 2024-09-12T03:19:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.499+0000 7fb4b6350740 1 -- start start 2024-09-12T03:19:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 0x56190158df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x56190159b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5619014f3280 con 0x56190158e480 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56190149ab10 con 0x56190158db60 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4a2ffd640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b6350740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5619014ef240 con 0x561901598d40 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34332/0 (socket says 172.21.15.19:34332) 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 -- 172.21.15.19:0/4063609765 learned_addr learned my addr 172.21.15.19:0/4063609765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4a2ffd640 1 -- 172.21.15.19:0/3647076378 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb49c0cefe0 con 0x7fb4ac102790 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1512640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x56190159b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b2514640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 0x56190158df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 -- 172.21.15.19:0/4063609765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 msgr2=0x56190159b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x56190159b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 -- 172.21.15.19:0/4063609765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 msgr2=0x56190158df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 0x56190158df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b1d13640 1 -- 172.21.15.19:0/4063609765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619014a71f0 con 0x56190158e480 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.500+0000 7fb4b2514640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 0x56190158df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8031550 tx=0x7fb4a80041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e934b6cb62f02b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098b0 tx=0x7fb4a4009840 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb497fff640 1 -- 172.21.15.19:0/4063609765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8035040 con 0x56190158e480 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb497fff640 1 -- 172.21.15.19:0/4063609765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8005520 con 0x56190158e480 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb497fff640 1 -- 172.21.15.19:0/4063609765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8003850 con 0x56190158e480 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb4b6350740 1 -- 172.21.15.19:0/4063609765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 msgr2=0x561901598800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.501+0000 7fb4b6350740 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8031550 tx=0x7fb4a80041a0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 -- 172.21.15.19:0/4063609765 shutdown_connections 2024-09-12T03:19:22.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x56190159b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158e480 0x561901598800 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 --2- 172.21.15.19:0/4063609765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158db60 0x56190158df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 -- 172.21.15.19:0/4063609765 >> 172.21.15.19:0/4063609765 conn(0x56190158a0f0 msgr2=0x56190158a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 -- 172.21.15.19:0/4063609765 shutdown_connections 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 -- 172.21.15.19:0/4063609765 wait complete. 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 -- start start 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 Processor -- start 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.502+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158e480 0x561901596120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561901598d40 0x561901596660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56190159b910 con 0x56190158db60 2024-09-12T03:19:22.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5619014adf10 con 0x561901598d40 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56190159ba90 con 0x56190158e480 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 10 client.0 _ll_register_callbacks cb 0x561901476a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34348/0 (socket says 172.21.15.19:34348) 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1320272456 learned_addr learned my addr 172.21.15.19:0/1320272456 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1512640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158e480 0x561901596120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b2514640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561901598d40 0x561901596660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158e480 msgr2=0x561901596120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158e480 0x561901596120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1320272456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561901598d40 msgr2=0x561901596660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561901598d40 0x561901596660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1320272456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619012646a0 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4a802a930 tx=0x7fb4a8005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61504afe37c32d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8035070 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4a80048d0 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.503+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a8005520 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4a8049070 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5619015a74e0 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8005520 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a80035f0 con 0x56190158db60 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:22.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 10 client.15288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 20 client.15288 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619015a8830 0x5619015aac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 4 client.15288 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.504+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619015007f0 con 0x5619015a8830 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.505+0000 7fb4b1512640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619015a8830 0x5619015aac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.505+0000 7fb4957fa640 10 client.15288 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.505+0000 7fb4b1512640 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619015a8830 0x5619015aac20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf21d69a109be968 server_cookie=47aee6eeb8bc1c91 in_seq=0 out_seq=0 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4957fa640 1 -- 172.21.15.19:0/1320272456 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619015007f0 con 0x5619015a8830 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4957fa640 10 client.15288 handle_command_reply: tid=0 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 client.15288 shutdown 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 msgr2=0x561901595be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb4a802a930 tx=0x7fb4a8005230 comp rx=0 tx=0).stop 2024-09-12T03:19:22.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 shutdown_connections 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619015a8830 0x5619015aac20 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561901598d40 0x561901596660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158e480 0x561901596120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4957fa640 0 client.15288 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1320272456 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x561901595be0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 >> 172.21.15.19:0/1320272456 conn(0x56190158a0f0 msgr2=0x56190158fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 shutdown_connections 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 1 -- 172.21.15.19:0/1320272456 wait complete. 2024-09-12T03:19:22.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.506+0000 7fb4b6350740 20 client.15288 trim_cache size 0 max 16384 2024-09-12T03:19:22.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.523+0000 7fb4b6350740 1 Processor -- start 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 -- start start 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015adc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x5619015ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ad580 con 0x56190158db60 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1af0c0 con 0x56190158e480 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b6350740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1adc50 con 0x561901598d40 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34356/0 (socket says 172.21.15.19:34356) 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1981339976 learned_addr learned my addr 172.21.15.19:0/1981339976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1512640 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1981339976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 msgr2=0x5619015ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x5619015ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1981339976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 msgr2=0x5619015adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 -- 172.21.15.19:0/1981339976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190158e2b0 con 0x56190158db60 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1512640 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.524+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009900 tx=0x7fb4a8005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea4ed8ea13ef48d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4977fe640 1 -- 172.21.15.19:0/1981339976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a805d030 con 0x56190158db60 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4977fe640 1 -- 172.21.15.19:0/1981339976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a8004150 con 0x56190158db60 2024-09-12T03:19:22.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4977fe640 1 -- 172.21.15.19:0/1981339976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8035070 con 0x56190158db60 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 -- 172.21.15.19:0/1981339976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 msgr2=0x5619015905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009900 tx=0x7fb4a8005900 comp rx=0 tx=0).stop 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 -- 172.21.15.19:0/1981339976 shutdown_connections 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561901598d40 0x5619015ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 --2- 172.21.15.19:0/1981339976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56190158db60 0x5619015905c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 -- 172.21.15.19:0/1981339976 >> 172.21.15.19:0/1981339976 conn(0x56190158a0f0 msgr2=0x5619015a4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 -- 172.21.15.19:0/1981339976 shutdown_connections 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.525+0000 7fb4b6350740 1 -- 172.21.15.19:0/1981339976 wait complete. 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 -- start start 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 Processor -- start 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158db60 0x5619015abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015ac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ae9f0 con 0x561901598d40 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ae320 con 0x56190158e480 2024-09-12T03:19:22.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ab560 con 0x56190158db60 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b6350740 10 client.0 _ll_register_callbacks cb 0x7fb49c004590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b2514640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b2514640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34370/0 (socket says 172.21.15.19:34370) 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.526+0000 7fb4b2514640 1 -- 172.21.15.19:0/3504093874 learned_addr learned my addr 172.21.15.19:0/3504093874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158db60 0x5619015abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b1512640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 -- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158db60 msgr2=0x5619015abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158db60 0x5619015abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 -- 172.21.15.19:0/3504093874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 msgr2=0x5619015ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:22.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 -- 172.21.15.19:0/3504093874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619015acb60 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b1512640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b2514640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4012810 tx=0x7fb4a4009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2d207d9aac400e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4004030 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4a40041d0 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619014fad00 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb4a40043d0 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b6350740 10 client.15294 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.527+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561901617b80 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5619014fad00 con 0x561901598d40 2024-09-12T03:19:22.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fb4a400f040 con 0x561901598d40 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 20 client.15294 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619016182a0 0x56190161a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 4 client.15294 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561901477110 con 0x5619016182a0 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.528+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619016182a0 0x56190161a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.529+0000 7fb4b1d13640 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619016182a0 0x56190161a690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bcfa58eabc021a5 server_cookie=aa20740548c2d424 in_seq=0 out_seq=0 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.529+0000 7fb4957fa640 10 client.15294 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4957fa640 1 -- 172.21.15.19:0/3504093874 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561901477110 con 0x5619016182a0 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4957fa640 10 client.15294 handle_command_reply: tid=0 2024-09-12T03:19:22.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 client.15294 shutdown 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 msgr2=0x561901616580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4012810 tx=0x7fb4a4009d90 comp rx=0 tx=0).stop 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 shutdown_connections 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561901598d40 0x561901616580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619016182a0 0x56190161a690 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56190158e480 0x5619015ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 --2- 172.21.15.19:0/3504093874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56190158db60 0x5619015abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4957fa640 0 client.15294 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 >> 172.21.15.19:0/3504093874 conn(0x56190158a0f0 msgr2=0x5619015a3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 shutdown_connections 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 1 -- 172.21.15.19:0/3504093874 wait complete. 2024-09-12T03:19:22.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.530+0000 7fb4b6350740 20 client.15294 trim_cache size 0 max 16384 2024-09-12T03:19:22.538 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:22.538 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:19:22.538 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ba58f53f-af27-42c8-94b4-401f249e066d", 2024-09-12T03:19:22.538 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:19:22.538 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 msgr2=0x7fb4980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 0x7fb4980a5040 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb4a40098b0 tx=0x7fb4a4009840 comp rx=0 tx=0).stop 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 msgr2=0x7fb4ac1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c890 tx=0x7fb49c00cd50 comp rx=0 tx=0).stop 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 shutdown_connections 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4ac106f10 0x7fb4ac19cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4980a2b80 0x7fb4980a5040 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4ac102790 0x7fb4ac1989a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 --2- 172.21.15.19:0/3647076378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4ac101de0 0x7fb4ac198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 >> 172.21.15.19:0/3647076378 conn(0x7fb4ac0fd8c0 msgr2=0x7fb4ac0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 shutdown_connections 2024-09-12T03:19:22.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.531+0000 7fb4b3f77640 1 -- 172.21.15.19:0/3647076378 wait complete. 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 -- 172.21.15.19:0/190197949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01082a0 msgr2=0x7effd010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 --2- 172.21.15.19:0/190197949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01082a0 0x7effd010ec40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effc8009930 tx=0x7effc802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 -- 172.21.15.19:0/190197949 shutdown_connections 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 --2- 172.21.15.19:0/190197949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01082a0 0x7effd010ec40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 --2- 172.21.15.19:0/190197949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01036e0 0x7effd0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 --2- 172.21.15.19:0/190197949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 -- 172.21.15.19:0/190197949 >> 172.21.15.19:0/190197949 conn(0x7effd00fc7d0 msgr2=0x7effd00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 -- 172.21.15.19:0/190197949 shutdown_connections 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 -- 172.21.15.19:0/190197949 wait complete. 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.673+0000 7effd5572640 1 Processor -- start 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 -- start start 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01082a0 0x7effd019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effd01120a0 con 0x7effd01036e0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7effd0111f20 con 0x7effd01082a0 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effd5572640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effd0112220 con 0x7effd0102d30 2024-09-12T03:19:23.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34380/0 (socket says 172.21.15.19:34380) 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 -- 172.21.15.19:0/1625543960 learned_addr learned my addr 172.21.15.19:0/1625543960 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effcf7fe640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01082a0 0x7effd019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effceffd640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 -- 172.21.15.19:0/1625543960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 msgr2=0x7effd019a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd019a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 -- 172.21.15.19:0/1625543960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01082a0 msgr2=0x7effd019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01082a0 0x7effd019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 -- 172.21.15.19:0/1625543960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd019f6f0 con 0x7effd01036e0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effceffd640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.674+0000 7effce7fc640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effb800e9f0 tx=0x7effb800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.675+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb800ccc0 con 0x7effd01036e0 2024-09-12T03:19:23.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.675+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8010910 con 0x7effd01036e0 2024-09-12T03:19:23.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.675+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd019f9e0 con 0x7effd01036e0 2024-09-12T03:19:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.675+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8018920 con 0x7effd01036e0 2024-09-12T03:19:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.675+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd01a72c0 con 0x7effd01036e0 2024-09-12T03:19:23.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.676+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd0103110 con 0x7effd01036e0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.677+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effb8010ac0 con 0x7effd01036e0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.678+0000 7effc7fff640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.678+0000 7effceffd640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.678+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7effb80cf720 con 0x7effd01036e0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.678+0000 7effceffd640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 0x7effbc0a50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc00094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.681+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effb809c690 con 0x7effd01036e0 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.858+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7effd010c2f0 con 0x7effbc0a2c00 2024-09-12T03:19:23.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:22.865+0000 7effc7fff640 1 -- 172.21.15.19:0/1625543960 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259115 (secure 0 0 0) 0x7effd010c2f0 con 0x7effbc0a2c00 2024-09-12T03:19:23.034 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 msgr2=0x7effbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 0x7effbc0a50c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7effc0005fd0 tx=0x7effc00094d0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 msgr2=0x7effd019ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effb800e9f0 tx=0x7effb800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 shutdown_connections 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effd01082a0 0x7effd019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effbc0a2c00 0x7effbc0a50c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effd01036e0 0x7effd019ac80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 --2- 172.21.15.19:0/1625543960 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effd0102d30 0x7effd019a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 >> 172.21.15.19:0/1625543960 conn(0x7effd00fc7d0 msgr2=0x7effd00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 shutdown_connections 2024-09-12T03:19:23.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:23.029+0000 7effd5572640 1 -- 172.21.15.19:0/1625543960 wait complete. 2024-09-12T03:19:23.388 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:23 smithi019 ceph-mon[30953]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:23.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:23 smithi121 ceph-mon[34963]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:23.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:19:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:23 smithi135 ceph-mon[35474]: pgmap v347: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:24.132 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-12T03:19:21.310052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1304,"num_read_kb":808,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":128,"num_read_kb":78,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002935"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279454+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279454+0000","last_peered":"2024-09-12T03:18:41.279454+0000","last_clean":"2024-09-12T03:18:41.279454+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279454+0000","last_undegraded":"2024-09-12T03:18:41.279454+0000","last_fullsized":"2024-09-12T03:18:41.279454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279635+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279635+0000","last_peered":"2024-09-12T03:18:41.279635+0000","last_clean":"2024-09-12T03:18:41.279635+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279635+0000","last_undegraded":"2024-09-12T03:18:41.279635+0000","last_fullsized":"2024-09-12T03:18:41.279635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281377+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.281377+0000","last_peered":"2024-09-12T03:18:41.281377+0000","last_clean":"2024-09-12T03:18:41.281377+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.281377+0000","last_undegraded":"2024-09-12T03:18:41.281377+0000","last_fullsized":"2024-09-12T03:18:41.281377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286421+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286421+0000","last_peered":"2024-09-12T03:18:41.286421+0000","last_clean":"2024-09-12T03:18:41.286421+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286421+0000","last_undegraded":"2024-09-12T03:18:41.286421+0000","last_fullsized":"2024-09-12T03:18:41.286421+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281777+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.281777+0000","last_peered":"2024-09-12T03:18:41.281777+0000","last_clean":"2024-09-12T03:18:41.281777+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.281777+0000","last_undegraded":"2024-09-12T03:18:41.281777+0000","last_fullsized":"2024-09-12T03:18:41.281777+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279505+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279505+0000","last_peered":"2024-09-12T03:18:41.279505+0000","last_clean":"2024-09-12T03:18:41.279505+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279505+0000","last_undegraded":"2024-09-12T03:18:41.279505+0000","last_fullsized":"2024-09-12T03:18:41.279505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272567+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272567+0000","last_peered":"2024-09-12T03:18:41.272567+0000","last_clean":"2024-09-12T03:18:41.272567+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272567+0000","last_undegraded":"2024-09-12T03:18:41.272567+0000","last_fullsized":"2024-09-12T03:18:41.272567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276770+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276770+0000","last_peered":"2024-09-12T03:18:41.276770+0000","last_clean":"2024-09-12T03:18:41.276770+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276770+0000","last_undegraded":"2024-09-12T03:18:41.276770+0000","last_fullsized":"2024-09-12T03:18:41.276770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294031+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294031+0000","last_peered":"2024-09-12T03:18:41.294031+0000","last_clean":"2024-09-12T03:18:41.294031+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294031+0000","last_undegraded":"2024-09-12T03:18:41.294031+0000","last_fullsized":"2024-09-12T03:18:41.294031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:16.978757+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:16.978757+0000","last_peered":"2024-09-12T03:19:16.978757+0000","last_clean":"2024-09-12T03:19:16.978757+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:16.978757+0000","last_undegraded":"2024-09-12T03:19:16.978757+0000","last_fullsized":"2024-09-12T03:19:16.978757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":266,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:19.362999+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:19.362999+0000","last_peered":"2024-09-12T03:19:19.362999+0000","last_clean":"2024-09-12T03:19:19.362999+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:19.362999+0000","last_undegraded":"2024-09-12T03:19:19.362999+0000","last_fullsized":"2024-09-12T03:19:19.362999+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":239,"num_read_kb":239,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":311,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:19.981882+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:19.981882+0000","last_peered":"2024-09-12T03:19:19.981882+0000","last_clean":"2024-09-12T03:19:19.981882+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:19.981882+0000","last_undegraded":"2024-09-12T03:19:19.981882+0000","last_fullsized":"2024-09-12T03:19:19.981882+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":253,"num_read_kb":191,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":151,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.977118+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:15.977118+0000","last_peered":"2024-09-12T03:19:15.977118+0000","last_clean":"2024-09-12T03:19:15.977118+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:15.977118+0000","last_undegraded":"2024-09-12T03:19:15.977118+0000","last_fullsized":"2024-09-12T03:19:15.977118+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276489+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276489+0000","last_peered":"2024-09-12T03:18:41.276489+0000","last_clean":"2024-09-12T03:18:41.276489+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276489+0000","last_undegraded":"2024-09-12T03:18:41.276489+0000","last_fullsized":"2024-09-12T03:18:41.276489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220590+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220590+0000","last_peered":"2024-09-12T03:18:13.220590+0000","last_clean":"2024-09-12T03:18:13.220590+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220590+0000","last_undegraded":"2024-09-12T03:18:13.220590+0000","last_fullsized":"2024-09-12T03:18:13.220590+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278619+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.278619+0000","last_peered":"2024-09-12T03:18:41.278619+0000","last_clean":"2024-09-12T03:18:41.278619+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.278619+0000","last_undegraded":"2024-09-12T03:18:41.278619+0000","last_fullsized":"2024-09-12T03:18:41.278619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227197+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227197+0000","last_peered":"2024-09-12T03:18:13.227197+0000","last_clean":"2024-09-12T03:18:13.227197+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227197+0000","last_undegraded":"2024-09-12T03:18:13.227197+0000","last_fullsized":"2024-09-12T03:18:13.227197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271760+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271760+0000","last_peered":"2024-09-12T03:18:41.271760+0000","last_clean":"2024-09-12T03:18:41.271760+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271760+0000","last_undegraded":"2024-09-12T03:18:41.271760+0000","last_fullsized":"2024-09-12T03:18:41.271760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":83,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.358409+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:15.358409+0000","last_peered":"2024-09-12T03:19:15.358409+0000","last_clean":"2024-09-12T03:19:15.358409+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:15.358409+0000","last_undegraded":"2024-09-12T03:19:15.358409+0000","last_fullsized":"2024-09-12T03:19:15.358409+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":65,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280684+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280684+0000","last_peered":"2024-09-12T03:18:41.280684+0000","last_clean":"2024-09-12T03:18:41.280684+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280684+0000","last_undegraded":"2024-09-12T03:18:41.280684+0000","last_fullsized":"2024-09-12T03:18:41.280684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283494+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283494+0000","last_peered":"2024-09-12T03:18:41.283494+0000","last_clean":"2024-09-12T03:18:41.283494+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283494+0000","last_undegraded":"2024-09-12T03:18:41.283494+0000","last_fullsized":"2024-09-12T03:18:41.283494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":148,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.976407+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:15.976407+0000","last_peered":"2024-09-12T03:19:15.976407+0000","last_clean":"2024-09-12T03:19:15.976407+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:15.976407+0000","last_undegraded":"2024-09-12T03:19:15.976407+0000","last_fullsized":"2024-09-12T03:19:15.976407+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":119,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":147,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:15.998969+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:15.998969+0000","last_peered":"2024-09-12T03:19:15.998969+0000","last_clean":"2024-09-12T03:19:15.998969+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:15.998969+0000","last_undegraded":"2024-09-12T03:19:15.998969+0000","last_fullsized":"2024-09-12T03:19:15.998969+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282537+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282537+0000","last_peered":"2024-09-12T03:18:41.282537+0000","last_clean":"2024-09-12T03:18:41.282537+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282537+0000","last_undegraded":"2024-09-12T03:18:41.282537+0000","last_fullsized":"2024-09-12T03:18:41.282537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:12.931837+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:19:12.931837+0000","last_peered":"2024-09-12T03:19:12.931837+0000","last_clean":"2024-09-12T03:19:12.931837+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:19:12.931837+0000","last_undegraded":"2024-09-12T03:19:12.931837+0000","last_fullsized":"2024-09-12T03:19:12.931837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285295+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.285295+0000","last_peered":"2024-09-12T03:18:41.285295+0000","last_clean":"2024-09-12T03:18:41.285295+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.285295+0000","last_undegraded":"2024-09-12T03:18:41.285295+0000","last_fullsized":"2024-09-12T03:18:41.285295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289155+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289155+0000","last_peered":"2024-09-12T03:18:41.289155+0000","last_clean":"2024-09-12T03:18:41.289155+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289155+0000","last_undegraded":"2024-09-12T03:18:41.289155+0000","last_fullsized":"2024-09-12T03:18:41.289155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.275190+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.275190+0000","last_peered":"2024-09-12T03:18:41.275190+0000","last_clean":"2024-09-12T03:18:41.275190+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.275190+0000","last_undegraded":"2024-09-12T03:18:41.275190+0000","last_fullsized":"2024-09-12T03:18:41.275190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227726+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227726+0000","last_peered":"2024-09-12T03:18:13.227726+0000","last_clean":"2024-09-12T03:18:13.227726+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227726+0000","last_undegraded":"2024-09-12T03:18:13.227726+0000","last_fullsized":"2024-09-12T03:18:13.227726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285539+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285539+0000","last_peered":"2024-09-12T03:18:41.285539+0000","last_clean":"2024-09-12T03:18:41.285539+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285539+0000","last_undegraded":"2024-09-12T03:18:41.285539+0000","last_fullsized":"2024-09-12T03:18:41.285539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221452+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221452+0000","last_peered":"2024-09-12T03:18:13.221452+0000","last_clean":"2024-09-12T03:18:13.221452+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221452+0000","last_undegraded":"2024-09-12T03:18:13.221452+0000","last_fullsized":"2024-09-12T03:18:13.221452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278354+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.278354+0000","last_peered":"2024-09-12T03:18:41.278354+0000","last_clean":"2024-09-12T03:18:41.278354+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.278354+0000","last_undegraded":"2024-09-12T03:18:41.278354+0000","last_fullsized":"2024-09-12T03:18:41.278354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":39,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236108+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236108+0000","last_peered":"2024-09-12T03:18:13.236108+0000","last_clean":"2024-09-12T03:18:13.236108+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236108+0000","last_undegraded":"2024-09-12T03:18:13.236108+0000","last_fullsized":"2024-09-12T03:18:13.236108+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282822+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282822+0000","last_peered":"2024-09-12T03:18:41.282822+0000","last_clean":"2024-09-12T03:18:41.282822+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282822+0000","last_undegraded":"2024-09-12T03:18:41.282822+0000","last_fullsized":"2024-09-12T03:18:41.282822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:19.001897+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:19.001897+0000","last_peered":"2024-09-12T03:19:19.001897+0000","last_clean":"2024-09-12T03:19:19.001897+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:19.001897+0000","last_undegraded":"2024-09-12T03:19:19.001897+0000","last_fullsized":"2024-09-12T03:19:19.001897+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1178,"num_read_kb":699,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972402,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462972,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626498,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855431,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:24.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:19:24.209 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:24.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:24 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:19:24.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:24 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:24 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:19:24.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:24 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:24 smithi019 ceph-mon[30953]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:24.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:24 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:19:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:24 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:24 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:19:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:24 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:24 smithi121 ceph-mon[34963]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:24 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:19:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:24 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:24 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:19:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:24 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:19:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:24 smithi135 ceph-mon[35474]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 -- 172.21.15.19:0/426121899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750101de0 msgr2=0x7f07501021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 --2- 172.21.15.19:0/426121899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750101de0 0x7f07501021c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0744009900 tx=0x7f074402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 -- 172.21.15.19:0/426121899 shutdown_connections 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 --2- 172.21.15.19:0/426121899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750106f10 0x7f075010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 --2- 172.21.15.19:0/426121899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 0x7f07501067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 --2- 172.21.15.19:0/426121899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750101de0 0x7f07501021c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 -- 172.21.15.19:0/426121899 >> 172.21.15.19:0/426121899 conn(0x7f07500fd8c0 msgr2=0x7f07500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 -- 172.21.15.19:0/426121899 shutdown_connections 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.715+0000 7f0756db2640 1 -- 172.21.15.19:0/426121899 wait complete. 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 Processor -- start 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 -- start start 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750101de0 0x7f07500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 0x7f0750074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111f70 con 0x7f0750106f10 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0750111df0 con 0x7f0750102790 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0756db2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07501120f0 con 0x7f0750101de0 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f075534f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f075534f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34408/0 (socket says 172.21.15.19:34408) 2024-09-12T03:19:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f075534f640 1 -- 172.21.15.19:0/450498911 learned_addr learned my addr 172.21.15.19:0/450498911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f074ffff640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 0x7f0750074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.716+0000 7f0754b4e640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750101de0 0x7f07500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 -- 172.21.15.19:0/450498911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750101de0 msgr2=0x7f07500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750101de0 0x7f07500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 -- 172.21.15.19:0/450498911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 msgr2=0x7f0750074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 0x7f0750074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 -- 172.21.15.19:0/450498911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07501a6890 con 0x7f0750106f10 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f075534f640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f074800cce0 tx=0x7f074800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f074dffb640 1 -- 172.21.15.19:0/450498911 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074800f450 con 0x7f0750106f10 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f074dffb640 1 -- 172.21.15.19:0/450498911 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f074800fad0 con 0x7f0750106f10 2024-09-12T03:19:24.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07501a6b80 con 0x7f0750106f10 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f074dffb640 1 -- 172.21.15.19:0/450498911 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748018c10 con 0x7f0750106f10 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.717+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07501a7070 con 0x7f0750106f10 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.719+0000 7f074dffb640 1 -- 172.21.15.19:0/450498911 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f074800f5f0 con 0x7f0750106f10 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 Processor -- start 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 -- start start 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f074dffb640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 0x7f07400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b99c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b99ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56071b8f72e0 con 0x56071b9914a0 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f074dffb640 1 -- 172.21.15.19:0/450498911 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f07480cf780 con 0x7f0750106f10 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56071b89ece0 con 0x56071b991dc0 2024-09-12T03:19:24.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075918b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56071b8f32a0 con 0x56071b99c680 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f074ffff640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 0x7f07400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f0754b4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f0754b4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34424/0 (socket says 172.21.15.19:34424) 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f0754b4e640 1 -- 172.21.15.19:0/2184287453 learned_addr learned my addr 172.21.15.19:0/2184287453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f075534f640 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b99ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.720+0000 7f074ffff640 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b99c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f074ffff640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 0x7f07400a5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f073c005d90 tx=0x7f073c00a5a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 -- 172.21.15.19:0/2184287453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 msgr2=0x56071b99ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b99ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 -- 172.21.15.19:0/2184287453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 msgr2=0x56071b99c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b99c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 -- 172.21.15.19:0/2184287453 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071b883440 con 0x56071b9914a0 2024-09-12T03:19:24.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f0754b4e640 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0744005ab0 tx=0x7f0744031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353971c7c7d4f5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f073affd640 1 -- 172.21.15.19:0/2184287453 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074403f070 con 0x56071b9914a0 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f073affd640 1 -- 172.21.15.19:0/2184287453 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744031a20 con 0x56071b9914a0 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.721+0000 7f073affd640 1 -- 172.21.15.19:0/2184287453 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744035070 con 0x56071b9914a0 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- 172.21.15.19:0/2184287453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 msgr2=0x56071b991880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0744005ab0 tx=0x7f0744031630 comp rx=0 tx=0).stop 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- 172.21.15.19:0/2184287453 shutdown_connections 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b99ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b99c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 --2- 172.21.15.19:0/2184287453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b991880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- 172.21.15.19:0/2184287453 >> 172.21.15.19:0/2184287453 conn(0x56071b98da30 msgr2=0x56071b98de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- 172.21.15.19:0/2184287453 shutdown_connections 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- 172.21.15.19:0/2184287453 wait complete. 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 -- start start 2024-09-12T03:19:24.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.722+0000 7f075918b740 1 Processor -- start 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b99bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9996e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56071b99f250 con 0x56071b991dc0 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56071b8b2050 con 0x56071b9914a0 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56071b99f3d0 con 0x56071b99c680 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075918b740 10 client.0 _ll_register_callbacks cb 0x56071b8f2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:24.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34432/0 (socket says 172.21.15.19:34432) 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 -- 172.21.15.19:0/765693165 learned_addr learned my addr 172.21.15.19:0/765693165 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f0754b4e640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b99bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f075534f640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 -- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 msgr2=0x56071b9996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 -- 172.21.15.19:0/765693165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 msgr2=0x56071b99bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b99bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 -- 172.21.15.19:0/765693165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071b6683c0 con 0x56071b991dc0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f074ffff640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f073c012a60 tx=0x7f073c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b12117bf22ddc0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.723+0000 7f072ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c011cf0 con 0x56071b991dc0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f073c005020 con 0x56071b991dc0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0744031a20 con 0x56071b991dc0 2024-09-12T03:19:24.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f073c005220 con 0x56071b991dc0 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 10 client.15312 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56071b9aabd0 con 0x56071b991dc0 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0744031a20 con 0x56071b991dc0 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f073c01e5f0 con 0x56071b991dc0 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 10 client.15312 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 10 client.15312 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 20 client.15312 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071b9abfe0 0x56071b9ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 4 client.15312 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.724+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56071b904850 con 0x56071b9abfe0 2024-09-12T03:19:24.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.725+0000 7f0754b4e640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071b9abfe0 0x56071b9ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.725+0000 7f0754b4e640 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071b9abfe0 0x56071b9ae4a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b223799125791ad4 server_cookie=d673dec3c8801d1f in_seq=0 out_seq=0 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.725+0000 7f072ffff640 10 client.15312 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f072ffff640 1 -- 172.21.15.19:0/765693165 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56071b904850 con 0x56071b9abfe0 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f072ffff640 10 client.15312 handle_command_reply: tid=0 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 client.15312 shutdown 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 msgr2=0x56071b9991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f073c012a60 tx=0x7f073c004730 comp rx=0 tx=0).stop 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 shutdown_connections 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b9991a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071b9abfe0 0x56071b9ae4a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 --2- 172.21.15.19:0/765693165 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b99bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f072ffff640 0 client.15312 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 >> 172.21.15.19:0/765693165 conn(0x56071b98da30 msgr2=0x56071b98cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 shutdown_connections 2024-09-12T03:19:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 1 -- 172.21.15.19:0/765693165 wait complete. 2024-09-12T03:19:24.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.726+0000 7f075918b740 20 client.15312 trim_cache size 0 max 16384 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.744+0000 7f075918b740 1 Processor -- start 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.744+0000 7f075918b740 1 -- start start 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b98de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07501af5f0 con 0x56071b991dc0 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b1130 con 0x56071b9914a0 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075918b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07501afcc0 con 0x56071b99c680 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34446/0 (socket says 172.21.15.19:34446) 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 -- 172.21.15.19:0/281640549 learned_addr learned my addr 172.21.15.19:0/281640549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075534f640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f0754b4e640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b98de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 -- 172.21.15.19:0/281640549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 msgr2=0x56071b9b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 -- 172.21.15.19:0/281640549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 msgr2=0x56071b98de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b98de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 -- 172.21.15.19:0/281640549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56071b998150 con 0x56071b991dc0 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f075534f640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.745+0000 7f074ffff640 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f073c005f40 tx=0x7f073c0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274a4fce6252aa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f073a7fc640 1 -- 172.21.15.19:0/281640549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c010fc0 con 0x56071b991dc0 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f073a7fc640 1 -- 172.21.15.19:0/281640549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f073c02c030 con 0x56071b991dc0 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f073a7fc640 1 -- 172.21.15.19:0/281640549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073c01e690 con 0x56071b991dc0 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 -- 172.21.15.19:0/281640549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 msgr2=0x56071b99b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f073c005f40 tx=0x7f073c0051b0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 -- 172.21.15.19:0/281640549 shutdown_connections 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b991dc0 0x56071b99b110 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 --2- 172.21.15.19:0/281640549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b9914a0 0x56071b98de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.746+0000 7f075918b740 1 -- 172.21.15.19:0/281640549 >> 172.21.15.19:0/281640549 conn(0x56071b98da30 msgr2=0x56071b99aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.747+0000 7f075918b740 1 -- 172.21.15.19:0/281640549 shutdown_connections 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.747+0000 7f075918b740 1 -- 172.21.15.19:0/281640549 wait complete. 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.747+0000 7f075918b740 1 -- start start 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.747+0000 7f075918b740 1 Processor -- start 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b9a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b0a60 con 0x56071b9914a0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f07501b0390 con 0x56071b991dc0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f07501ad5d0 con 0x56071b99c680 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075918b740 10 client.0 _ll_register_callbacks cb 0x7f073c012730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34452/0 (socket says 172.21.15.19:34452) 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 -- 172.21.15.19:0/4182142502 learned_addr learned my addr 172.21.15.19:0/4182142502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f074ffff640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b9a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f075534f640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 -- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 msgr2=0x56071b9a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 -- 172.21.15.19:0/4182142502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 msgr2=0x56071b9a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:24.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b9a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 -- 172.21.15.19:0/4182142502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071b99c260 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f074ffff640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b9a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f0754b4e640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f07440096c0 tx=0x7f0744005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbf2b240c5696a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.748+0000 7f072ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0744035040 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0744004240 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073c02c030 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f074403f040 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 10 client.15318 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56071ba17170 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f073c02c030 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0744003680 con 0x56071b9914a0 2024-09-12T03:19:24.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 20 client.15318 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071ba174b0 0x56071ba198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 4 client.15318 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.749+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56071b8fed60 con 0x56071ba174b0 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f074ffff640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071ba174b0 0x56071ba198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f074ffff640 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071ba174b0 0x56071ba198a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df845bb95735e5e2 server_cookie=25a086b785578170 in_seq=0 out_seq=0 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f072ffff640 10 client.15318 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f072ffff640 1 -- 172.21.15.19:0/4182142502 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56071b8fed60 con 0x56071ba174b0 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f072ffff640 10 client.15318 handle_command_reply: tid=0 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f075918b740 1 client.15318 shutdown 2024-09-12T03:19:24.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 msgr2=0x56071b9a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.750+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f07440096c0 tx=0x7f0744005590 comp rx=0 tx=0).stop 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 shutdown_connections 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56071b99c680 0x56071b9a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56071ba174b0 0x56071ba198a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56071b991dc0 0x56071b9a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 --2- 172.21.15.19:0/4182142502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56071b9914a0 0x56071b9a06c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f072ffff640 0 client.15318 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 >> 172.21.15.19:0/4182142502 conn(0x56071b98da30 msgr2=0x56071b99b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 shutdown_connections 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 1 -- 172.21.15.19:0/4182142502 wait complete. 2024-09-12T03:19:24.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.751+0000 7f075918b740 20 client.15318 trim_cache size 0 max 16384 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "ba58f53f-af27-42c8-94b4-401f249e066d": { 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ba58f53f-af27-42c8-94b4-401f249e066d", 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:19:24.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 msgr2=0x7f07400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 0x7f07400a5040 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f073c005d90 tx=0x7f073c00a5a0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 msgr2=0x7f0750074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f074800cce0 tx=0x7f074800a620 comp rx=0 tx=0).stop 2024-09-12T03:19:24.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 shutdown_connections 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f07400a2b80 0x7f07400a5040 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0750106f10 0x7f0750074f70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0750102790 0x7f0750074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 --2- 172.21.15.19:0/450498911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0750101de0 0x7f07500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 >> 172.21.15.19:0/450498911 conn(0x7f07500fd8c0 msgr2=0x7f07500febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 shutdown_connections 2024-09-12T03:19:24.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:24.752+0000 7f0756db2640 1 -- 172.21.15.19:0/450498911 wait complete. 2024-09-12T03:19:24.957 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:25.363 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:25 smithi019 ceph-mon[30953]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:25.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:25 smithi121 ceph-mon[34963]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:25.434 INFO:tasks.cephfs.filesystem:scrub status for tag:ba58f53f-af27-42c8-94b4-401f249e066d - {'path': '/', 'tag': 'ba58f53f-af27-42c8-94b4-401f249e066d', 'options': 'recursive,force'} 2024-09-12T03:19:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:25 smithi135 ceph-mon[35474]: pgmap v348: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 -- 172.21.15.19:0/994487651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 msgr2=0x7f51d4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 --2- 172.21.15.19:0/994487651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d4106e60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009930 tx=0x7f51d002a420 comp rx=0 tx=0).stop 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 -- 172.21.15.19:0/994487651 shutdown_connections 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 --2- 172.21.15.19:0/994487651 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 --2- 172.21.15.19:0/994487651 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 --2- 172.21.15.19:0/994487651 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d4106e60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 -- 172.21.15.19:0/994487651 >> 172.21.15.19:0/994487651 conn(0x7f51d40fc7a0 msgr2=0x7f51d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:25.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 -- 172.21.15.19:0/994487651 shutdown_connections 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.835+0000 7f51dc60b640 1 -- 172.21.15.19:0/994487651 wait complete. 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 Processor -- start 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 -- start start 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111fa0 con 0x7f51d4106a80 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111e20 con 0x7f51d4101420 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51dc60b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4112120 con 0x7f51d4100a80 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34458/0 (socket says 172.21.15.19:34458) 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 -- 172.21.15.19:0/1473218437 learned_addr learned my addr 172.21.15.19:0/1473218437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51d9ba6640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51da3a7640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 -- 172.21.15.19:0/1473218437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 msgr2=0x7f51d410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 -- 172.21.15.19:0/1473218437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 msgr2=0x7f51d410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:25.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51daba8640 1 -- 172.21.15.19:0/1473218437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d41a6960 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51d9ba6640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.836+0000 7f51da3a7640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51daba8640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00eb10 tx=0x7f51cc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc004b20 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41a6c50 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc004cc0 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc010760 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.837+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a7140 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.838+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d4100ee0 con 0x7f51d4106a80 2024-09-12T03:19:25.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.839+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51cc005020 con 0x7f51d4106a80 2024-09-12T03:19:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.840+0000 7f51cb7fe640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.840+0000 7f51da3a7640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.840+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51cc0cf6b0 con 0x7f51d4106a80 2024-09-12T03:19:25.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.841+0000 7f51da3a7640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 0x7f51c00a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:25.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:25.844+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51cc09c620 con 0x7f51d4106a80 2024-09-12T03:19:26.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.021+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f51d4062630 con 0x7f51c00a2b80 2024-09-12T03:19:26.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.027+0000 7f51cb7fe640 1 -- 172.21.15.19:0/1473218437 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259115 (secure 0 0 0) 0x7f51d4062630 con 0x7f51c00a2b80 2024-09-12T03:19:26.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:26.031 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 msgr2=0x7f51c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 0x7f51c00a5040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f51d0000c00 tx=0x7f51d0032c70 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 msgr2=0x7f51d410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00eb10 tx=0x7f51cc00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 shutdown_connections 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51c00a2b80 0x7f51c00a5040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51d4106a80 0x7f51d410ee60 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51d4101420 0x7f51d410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 --2- 172.21.15.19:0/1473218437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51d4100a80 0x7f51d410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.030+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 >> 172.21.15.19:0/1473218437 conn(0x7f51d40fc7a0 msgr2=0x7f51d40fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:26.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.031+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 shutdown_connections 2024-09-12T03:19:26.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:26.031+0000 7f51dc60b640 1 -- 172.21.15.19:0/1473218437 wait complete. 2024-09-12T03:19:26.713 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-12T03:19:25.311254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1356,"num_read_kb":835,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5989,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":80,"num_write":7,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003225"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279454+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279454+0000","last_peered":"2024-09-12T03:18:41.279454+0000","last_clean":"2024-09-12T03:18:41.279454+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279454+0000","last_undegraded":"2024-09-12T03:18:41.279454+0000","last_fullsized":"2024-09-12T03:18:41.279454+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279635+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279635+0000","last_peered":"2024-09-12T03:18:41.279635+0000","last_clean":"2024-09-12T03:18:41.279635+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279635+0000","last_undegraded":"2024-09-12T03:18:41.279635+0000","last_fullsized":"2024-09-12T03:18:41.279635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281377+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.281377+0000","last_peered":"2024-09-12T03:18:41.281377+0000","last_clean":"2024-09-12T03:18:41.281377+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.281377+0000","last_undegraded":"2024-09-12T03:18:41.281377+0000","last_fullsized":"2024-09-12T03:18:41.281377+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286543+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286543+0000","last_peered":"2024-09-12T03:18:41.286543+0000","last_clean":"2024-09-12T03:18:41.286543+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286543+0000","last_undegraded":"2024-09-12T03:18:41.286543+0000","last_fullsized":"2024-09-12T03:18:41.286543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285760+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.285760+0000","last_peered":"2024-09-12T03:18:41.285760+0000","last_clean":"2024-09-12T03:18:41.285760+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.285760+0000","last_undegraded":"2024-09-12T03:18:41.285760+0000","last_fullsized":"2024-09-12T03:18:41.285760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279505+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279505+0000","last_peered":"2024-09-12T03:18:41.279505+0000","last_clean":"2024-09-12T03:18:41.279505+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279505+0000","last_undegraded":"2024-09-12T03:18:41.279505+0000","last_fullsized":"2024-09-12T03:18:41.279505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272567+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272567+0000","last_peered":"2024-09-12T03:18:41.272567+0000","last_clean":"2024-09-12T03:18:41.272567+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272567+0000","last_undegraded":"2024-09-12T03:18:41.272567+0000","last_fullsized":"2024-09-12T03:18:41.272567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276770+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276770+0000","last_peered":"2024-09-12T03:18:41.276770+0000","last_clean":"2024-09-12T03:18:41.276770+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276770+0000","last_undegraded":"2024-09-12T03:18:41.276770+0000","last_fullsized":"2024-09-12T03:18:41.276770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294123+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294123+0000","last_peered":"2024-09-12T03:18:41.294123+0000","last_clean":"2024-09-12T03:18:41.294123+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294123+0000","last_undegraded":"2024-09-12T03:18:41.294123+0000","last_fullsized":"2024-09-12T03:18:41.294123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:21.985523+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:21.985523+0000","last_peered":"2024-09-12T03:19:21.985523+0000","last_clean":"2024-09-12T03:19:21.985523+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:21.985523+0000","last_undegraded":"2024-09-12T03:19:21.985523+0000","last_fullsized":"2024-09-12T03:19:21.985523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":276,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.370761+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:24.370761+0000","last_peered":"2024-09-12T03:19:24.370761+0000","last_clean":"2024-09-12T03:19:24.370761+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:24.370761+0000","last_undegraded":"2024-09-12T03:19:24.370761+0000","last_fullsized":"2024-09-12T03:19:24.370761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":249,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.989166+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:24.989166+0000","last_peered":"2024-09-12T03:19:24.989166+0000","last_clean":"2024-09-12T03:19:24.989166+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:24.989166+0000","last_undegraded":"2024-09-12T03:19:24.989166+0000","last_fullsized":"2024-09-12T03:19:24.989166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":197,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":156,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.993788+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:20.993788+0000","last_peered":"2024-09-12T03:19:20.993788+0000","last_clean":"2024-09-12T03:19:20.993788+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:20.993788+0000","last_undegraded":"2024-09-12T03:19:20.993788+0000","last_fullsized":"2024-09-12T03:19:20.993788+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276489+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276489+0000","last_peered":"2024-09-12T03:18:41.276489+0000","last_clean":"2024-09-12T03:18:41.276489+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276489+0000","last_undegraded":"2024-09-12T03:18:41.276489+0000","last_fullsized":"2024-09-12T03:18:41.276489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220643+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220643+0000","last_peered":"2024-09-12T03:18:13.220643+0000","last_clean":"2024-09-12T03:18:13.220643+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220643+0000","last_undegraded":"2024-09-12T03:18:13.220643+0000","last_fullsized":"2024-09-12T03:18:13.220643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278619+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.278619+0000","last_peered":"2024-09-12T03:18:41.278619+0000","last_clean":"2024-09-12T03:18:41.278619+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.278619+0000","last_undegraded":"2024-09-12T03:18:41.278619+0000","last_fullsized":"2024-09-12T03:18:41.278619+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227246+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227246+0000","last_peered":"2024-09-12T03:18:13.227246+0000","last_clean":"2024-09-12T03:18:13.227246+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227246+0000","last_undegraded":"2024-09-12T03:18:13.227246+0000","last_fullsized":"2024-09-12T03:18:13.227246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":37,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271760+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271760+0000","last_peered":"2024-09-12T03:18:41.271760+0000","last_clean":"2024-09-12T03:18:41.271760+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271760+0000","last_undegraded":"2024-09-12T03:18:41.271760+0000","last_fullsized":"2024-09-12T03:18:41.271760+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":88,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.364806+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:20.364806+0000","last_peered":"2024-09-12T03:19:20.364806+0000","last_clean":"2024-09-12T03:19:20.364806+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:20.364806+0000","last_undegraded":"2024-09-12T03:19:20.364806+0000","last_fullsized":"2024-09-12T03:19:20.364806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280684+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280684+0000","last_peered":"2024-09-12T03:18:41.280684+0000","last_clean":"2024-09-12T03:18:41.280684+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280684+0000","last_undegraded":"2024-09-12T03:18:41.280684+0000","last_fullsized":"2024-09-12T03:18:41.280684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283494+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283494+0000","last_peered":"2024-09-12T03:18:41.283494+0000","last_clean":"2024-09-12T03:18:41.283494+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283494+0000","last_undegraded":"2024-09-12T03:18:41.283494+0000","last_fullsized":"2024-09-12T03:18:41.283494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.993281+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:20.993281+0000","last_peered":"2024-09-12T03:19:20.993281+0000","last_clean":"2024-09-12T03:19:20.993281+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:20.993281+0000","last_undegraded":"2024-09-12T03:19:20.993281+0000","last_fullsized":"2024-09-12T03:19:20.993281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":152,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:21.004844+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:21.004844+0000","last_peered":"2024-09-12T03:19:21.004844+0000","last_clean":"2024-09-12T03:19:21.004844+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:21.004844+0000","last_undegraded":"2024-09-12T03:19:21.004844+0000","last_fullsized":"2024-09-12T03:19:21.004844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":123,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282537+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282537+0000","last_peered":"2024-09-12T03:18:41.282537+0000","last_clean":"2024-09-12T03:18:41.282537+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282537+0000","last_undegraded":"2024-09-12T03:18:41.282537+0000","last_fullsized":"2024-09-12T03:18:41.282537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:12.931837+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:19:12.931837+0000","last_peered":"2024-09-12T03:19:12.931837+0000","last_clean":"2024-09-12T03:19:12.931837+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:19:12.931837+0000","last_undegraded":"2024-09-12T03:19:12.931837+0000","last_fullsized":"2024-09-12T03:19:12.931837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285295+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.285295+0000","last_peered":"2024-09-12T03:18:41.285295+0000","last_clean":"2024-09-12T03:18:41.285295+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.285295+0000","last_undegraded":"2024-09-12T03:18:41.285295+0000","last_fullsized":"2024-09-12T03:18:41.285295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289155+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289155+0000","last_peered":"2024-09-12T03:18:41.289155+0000","last_clean":"2024-09-12T03:18:41.289155+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289155+0000","last_undegraded":"2024-09-12T03:18:41.289155+0000","last_fullsized":"2024-09-12T03:18:41.289155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.275190+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.275190+0000","last_peered":"2024-09-12T03:18:41.275190+0000","last_clean":"2024-09-12T03:18:41.275190+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.275190+0000","last_undegraded":"2024-09-12T03:18:41.275190+0000","last_fullsized":"2024-09-12T03:18:41.275190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227726+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227726+0000","last_peered":"2024-09-12T03:18:13.227726+0000","last_clean":"2024-09-12T03:18:13.227726+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227726+0000","last_undegraded":"2024-09-12T03:18:13.227726+0000","last_fullsized":"2024-09-12T03:18:13.227726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285539+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285539+0000","last_peered":"2024-09-12T03:18:41.285539+0000","last_clean":"2024-09-12T03:18:41.285539+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285539+0000","last_undegraded":"2024-09-12T03:18:41.285539+0000","last_fullsized":"2024-09-12T03:18:41.285539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221452+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221452+0000","last_peered":"2024-09-12T03:18:13.221452+0000","last_clean":"2024-09-12T03:18:13.221452+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221452+0000","last_undegraded":"2024-09-12T03:18:13.221452+0000","last_fullsized":"2024-09-12T03:18:13.221452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285008+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.285008+0000","last_peered":"2024-09-12T03:18:41.285008+0000","last_clean":"2024-09-12T03:18:41.285008+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.285008+0000","last_undegraded":"2024-09-12T03:18:41.285008+0000","last_fullsized":"2024-09-12T03:18:41.285008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236604+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236604+0000","last_peered":"2024-09-12T03:18:13.236604+0000","last_clean":"2024-09-12T03:18:13.236604+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236604+0000","last_undegraded":"2024-09-12T03:18:13.236604+0000","last_fullsized":"2024-09-12T03:18:13.236604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282965+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282965+0000","last_peered":"2024-09-12T03:18:41.282965+0000","last_clean":"2024-09-12T03:18:41.282965+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282965+0000","last_undegraded":"2024-09-12T03:18:41.282965+0000","last_fullsized":"2024-09-12T03:18:41.282965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":163,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.008069+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:24.008069+0000","last_peered":"2024-09-12T03:19:24.008069+0000","last_clean":"2024-09-12T03:19:24.008069+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:24.008069+0000","last_undegraded":"2024-09-12T03:19:24.008069+0000","last_fullsized":"2024-09-12T03:19:24.008069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1230,"num_read_kb":726,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444119,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:26.719 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T03:19:26.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T03:19:26.976 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:27.398 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:27 smithi019 ceph-mon[30953]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:27.399 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:27 smithi019 ceph-mon[30953]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:27 smithi121 ceph-mon[34963]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:27 smithi121 ceph-mon[34963]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:27.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 -- 172.21.15.19:0/3066318843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 msgr2=0x7f72fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 --2- 172.21.15.19:0/3066318843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc103110 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009930 tx=0x7f72ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 -- 172.21.15.19:0/3066318843 shutdown_connections 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 --2- 172.21.15.19:0/3066318843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 0x7f72fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 --2- 172.21.15.19:0/3066318843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 0x7f72fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 --2- 172.21.15.19:0/3066318843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc103110 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 -- 172.21.15.19:0/3066318843 >> 172.21.15.19:0/3066318843 conn(0x7f72fc0fc7d0 msgr2=0x7f72fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 -- 172.21.15.19:0/3066318843 shutdown_connections 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.481+0000 7f7302903640 1 -- 172.21.15.19:0/3066318843 wait complete. 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 Processor -- start 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 -- start start 2024-09-12T03:19:27.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 0x7f72fc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 0x7f72fc19ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc112100 con 0x7f72fc102d30 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111f80 con 0x7f72fc1036e0 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f7302903640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc112280 con 0x7f72fc1082a0 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f72fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f72fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34476/0 (socket says 172.21.15.19:34476) 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f72fbfff640 1 -- 172.21.15.19:0/2160493031 learned_addr learned my addr 172.21.15.19:0/2160493031 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.482+0000 7f72fb7fe640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 0x7f72fc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f7300ea0640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 0x7f72fc19ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 -- 172.21.15.19:0/2160493031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 msgr2=0x7f72fc19ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 0x7f72fc19ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 -- 172.21.15.19:0/2160493031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 msgr2=0x7f72fc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 0x7f72fc198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 -- 172.21.15.19:0/2160493031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc19d520 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72fbfff640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009a50 tx=0x7f72ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec03f030 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc19d7b0 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72ec003ad0 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec035070 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.483+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1a50f0 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.484+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc103110 con 0x7f72fc102d30 2024-09-12T03:19:27.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.485+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72ec042030 con 0x7f72fc102d30 2024-09-12T03:19:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.486+0000 7f72f97fa640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 0x7f72e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.486+0000 7f72fb7fe640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 0x7f72e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.486+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f72ec0efcb0 con 0x7f72fc102d30 2024-09-12T03:19:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.487+0000 7f72fb7fe640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 0x7f72e40a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0feda0 tx=0x7f72f000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:27.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.489+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72ec0bcbc0 con 0x7f72fc102d30 2024-09-12T03:19:27.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:27 smithi135 ceph-mon[35474]: pgmap v349: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:27 smithi135 ceph-mon[35474]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:27.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.660+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 --> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f72fc062630 con 0x7f72e40a2b80 2024-09-12T03:19:27.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.666+0000 7f72f97fa640 1 -- 172.21.15.19:0/2160493031 <== mgr.14150 v2:172.21.15.19:6800/2264764467 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259113 (secure 0 0 0) 0x7f72fc062630 con 0x7f72e40a2b80 2024-09-12T03:19:27.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:27.670 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-12T03:19:27.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.669+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 msgr2=0x7f72e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.669+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 0x7f72e40a5040 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0feda0 tx=0x7f72f000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 msgr2=0x7f72fc198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009a50 tx=0x7f72ec004270 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 shutdown_connections 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc1082a0 0x7f72fc19ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc1036e0 0x7f72fc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e40a2b80 0x7f72e40a5040 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 --2- 172.21.15.19:0/2160493031 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc102d30 0x7f72fc198570 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 >> 172.21.15.19:0/2160493031 conn(0x7f72fc0fc7d0 msgr2=0x7f72fc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 shutdown_connections 2024-09-12T03:19:27.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:27.670+0000 7f7302903640 1 -- 172.21.15.19:0/2160493031 wait complete. 2024-09-12T03:19:28.362 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-12T03:19:27.311766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664094,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1371,"num_read_kb":835,"num_write":442,"num_write_kb":3091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1182627,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360384,"kb_used_data":11136,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581440,"statfs":{"total":1151940427776,"available":1151571394560,"internally_reserved":0,"allocated":11403264,"data_stored":7233126,"data_compressed":113049,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":53,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003150"},"pg_stats":[{"pgid":"3.3f","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286305+0000","last_change":"2024-09-12T03:17:09.932494+0000","last_active":"2024-09-12T03:18:41.286305+0000","last_peered":"2024-09-12T03:18:41.286305+0000","last_clean":"2024-09-12T03:18:41.286305+0000","last_became_active":"2024-09-12T03:17:09.931951+0000","last_became_peered":"2024-09-12T03:17:09.931951+0000","last_unstale":"2024-09-12T03:18:41.286305+0000","last_undegraded":"2024-09-12T03:18:41.286305+0000","last_fullsized":"2024-09-12T03:18:41.286305+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:23:05.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288714+0000","last_change":"2024-09-12T03:17:09.929148+0000","last_active":"2024-09-12T03:18:41.288714+0000","last_peered":"2024-09-12T03:18:41.288714+0000","last_clean":"2024-09-12T03:18:41.288714+0000","last_became_active":"2024-09-12T03:17:09.928048+0000","last_became_peered":"2024-09-12T03:17:09.928048+0000","last_unstale":"2024-09-12T03:18:41.288714+0000","last_undegraded":"2024-09-12T03:18:41.288714+0000","last_fullsized":"2024-09-12T03:18:41.288714+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:20:20.528857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293703+0000","last_change":"2024-09-12T03:17:09.923181+0000","last_active":"2024-09-12T03:18:41.293703+0000","last_peered":"2024-09-12T03:18:41.293703+0000","last_clean":"2024-09-12T03:18:41.293703+0000","last_became_active":"2024-09-12T03:17:09.922778+0000","last_became_peered":"2024-09-12T03:17:09.922778+0000","last_unstale":"2024-09-12T03:18:41.293703+0000","last_undegraded":"2024-09-12T03:18:41.293703+0000","last_fullsized":"2024-09-12T03:18:41.293703+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:19:37.809088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278085+0000","last_change":"2024-09-12T03:18:13.227085+0000","last_active":"2024-09-12T03:18:41.278085+0000","last_peered":"2024-09-12T03:18:41.278085+0000","last_clean":"2024-09-12T03:18:41.278085+0000","last_became_active":"2024-09-12T03:18:13.226686+0000","last_became_peered":"2024-09-12T03:18:13.226686+0000","last_unstale":"2024-09-12T03:18:41.278085+0000","last_undegraded":"2024-09-12T03:18:41.278085+0000","last_fullsized":"2024-09-12T03:18:41.278085+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:34:32.155374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"82'4","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269846+0000","last_change":"2024-09-12T03:17:09.922111+0000","last_active":"2024-09-12T03:18:41.269846+0000","last_peered":"2024-09-12T03:18:41.269846+0000","last_clean":"2024-09-12T03:18:41.269846+0000","last_became_active":"2024-09-12T03:17:09.921297+0000","last_became_peered":"2024-09-12T03:17:09.921297+0000","last_unstale":"2024-09-12T03:18:41.269846+0000","last_undegraded":"2024-09-12T03:18:41.269846+0000","last_fullsized":"2024-09-12T03:18:41.269846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:01:06.374624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.268784+0000","last_change":"2024-09-12T03:17:09.932160+0000","last_active":"2024-09-12T03:18:41.268784+0000","last_peered":"2024-09-12T03:18:41.268784+0000","last_clean":"2024-09-12T03:18:41.268784+0000","last_became_active":"2024-09-12T03:17:09.931787+0000","last_became_peered":"2024-09-12T03:17:09.931787+0000","last_unstale":"2024-09-12T03:18:41.268784+0000","last_undegraded":"2024-09-12T03:18:41.268784+0000","last_fullsized":"2024-09-12T03:18:41.268784+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:02.831240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"82'5","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.269482+0000","last_change":"2024-09-12T03:17:09.931039+0000","last_active":"2024-09-12T03:18:41.269482+0000","last_peered":"2024-09-12T03:18:41.269482+0000","last_clean":"2024-09-12T03:18:41.269482+0000","last_became_active":"2024-09-12T03:17:09.930683+0000","last_became_peered":"2024-09-12T03:17:09.930683+0000","last_unstale":"2024-09-12T03:18:41.269482+0000","last_undegraded":"2024-09-12T03:18:41.269482+0000","last_fullsized":"2024-09-12T03:18:41.269482+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:05:42.699375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285506+0000","last_change":"2024-09-12T03:17:09.930298+0000","last_active":"2024-09-12T03:18:41.285506+0000","last_peered":"2024-09-12T03:18:41.285506+0000","last_clean":"2024-09-12T03:18:41.285506+0000","last_became_active":"2024-09-12T03:17:09.929827+0000","last_became_peered":"2024-09-12T03:17:09.929827+0000","last_unstale":"2024-09-12T03:18:41.285506+0000","last_undegraded":"2024-09-12T03:18:41.285506+0000","last_fullsized":"2024-09-12T03:18:41.285506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:14.230715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286188+0000","last_change":"2024-09-12T03:17:09.928862+0000","last_active":"2024-09-12T03:18:41.286188+0000","last_peered":"2024-09-12T03:18:41.286188+0000","last_clean":"2024-09-12T03:18:41.286188+0000","last_became_active":"2024-09-12T03:17:09.928542+0000","last_became_peered":"2024-09-12T03:17:09.928542+0000","last_unstale":"2024-09-12T03:18:41.286188+0000","last_undegraded":"2024-09-12T03:18:41.286188+0000","last_fullsized":"2024-09-12T03:18:41.286188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:50:56.147226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"82'16","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.262384+0000","last_change":"2024-09-12T03:17:09.932985+0000","last_active":"2024-09-12T03:18:41.262384+0000","last_peered":"2024-09-12T03:18:41.262384+0000","last_clean":"2024-09-12T03:18:41.262384+0000","last_became_active":"2024-09-12T03:17:09.932586+0000","last_became_peered":"2024-09-12T03:17:09.932586+0000","last_unstale":"2024-09-12T03:18:41.262384+0000","last_undegraded":"2024-09-12T03:18:41.262384+0000","last_fullsized":"2024-09-12T03:18:41.262384+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:49:45.334598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280568+0000","last_change":"2024-09-12T03:17:09.932430+0000","last_active":"2024-09-12T03:18:41.280568+0000","last_peered":"2024-09-12T03:18:41.280568+0000","last_clean":"2024-09-12T03:18:41.280568+0000","last_became_active":"2024-09-12T03:17:09.928755+0000","last_became_peered":"2024-09-12T03:17:09.928755+0000","last_unstale":"2024-09-12T03:18:41.280568+0000","last_undegraded":"2024-09-12T03:18:41.280568+0000","last_fullsized":"2024-09-12T03:18:41.280568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:01:02.817625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.263100+0000","last_change":"2024-09-12T03:17:09.928143+0000","last_active":"2024-09-12T03:18:41.263100+0000","last_peered":"2024-09-12T03:18:41.263100+0000","last_clean":"2024-09-12T03:18:41.263100+0000","last_became_active":"2024-09-12T03:17:09.927832+0000","last_became_peered":"2024-09-12T03:17:09.927832+0000","last_unstale":"2024-09-12T03:18:41.263100+0000","last_undegraded":"2024-09-12T03:18:41.263100+0000","last_fullsized":"2024-09-12T03:18:41.263100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:50.279340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280344+0000","last_change":"2024-09-12T03:17:09.928805+0000","last_active":"2024-09-12T03:18:41.280344+0000","last_peered":"2024-09-12T03:18:41.280344+0000","last_clean":"2024-09-12T03:18:41.280344+0000","last_became_active":"2024-09-12T03:17:09.928423+0000","last_became_peered":"2024-09-12T03:17:09.928423+0000","last_unstale":"2024-09-12T03:18:41.280344+0000","last_undegraded":"2024-09-12T03:18:41.280344+0000","last_fullsized":"2024-09-12T03:18:41.280344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:40:47.056732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.288243+0000","last_change":"2024-09-12T03:17:09.928370+0000","last_active":"2024-09-12T03:18:41.288243+0000","last_peered":"2024-09-12T03:18:41.288243+0000","last_clean":"2024-09-12T03:18:41.288243+0000","last_became_active":"2024-09-12T03:17:09.928028+0000","last_became_peered":"2024-09-12T03:17:09.928028+0000","last_unstale":"2024-09-12T03:18:41.288243+0000","last_undegraded":"2024-09-12T03:18:41.288243+0000","last_fullsized":"2024-09-12T03:18:41.288243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:43:22.996810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282846+0000","last_change":"2024-09-12T03:17:09.921059+0000","last_active":"2024-09-12T03:18:41.282846+0000","last_peered":"2024-09-12T03:18:41.282846+0000","last_clean":"2024-09-12T03:18:41.282846+0000","last_became_active":"2024-09-12T03:17:09.920748+0000","last_became_peered":"2024-09-12T03:17:09.920748+0000","last_unstale":"2024-09-12T03:18:41.282846+0000","last_undegraded":"2024-09-12T03:18:41.282846+0000","last_fullsized":"2024-09-12T03:18:41.282846+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:41:39.125582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293481+0000","last_change":"2024-09-12T03:17:09.920607+0000","last_active":"2024-09-12T03:18:41.293481+0000","last_peered":"2024-09-12T03:18:41.293481+0000","last_clean":"2024-09-12T03:18:41.293481+0000","last_became_active":"2024-09-12T03:17:09.920292+0000","last_became_peered":"2024-09-12T03:17:09.920292+0000","last_unstale":"2024-09-12T03:18:41.293481+0000","last_undegraded":"2024-09-12T03:18:41.293481+0000","last_fullsized":"2024-09-12T03:18:41.293481+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:17:42.418582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286023+0000","last_change":"2024-09-12T03:17:09.931193+0000","last_active":"2024-09-12T03:18:41.286023+0000","last_peered":"2024-09-12T03:18:41.286023+0000","last_clean":"2024-09-12T03:18:41.286023+0000","last_became_active":"2024-09-12T03:17:09.930878+0000","last_became_peered":"2024-09-12T03:17:09.930878+0000","last_unstale":"2024-09-12T03:18:41.286023+0000","last_undegraded":"2024-09-12T03:18:41.286023+0000","last_fullsized":"2024-09-12T03:18:41.286023+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:46:54.452459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285090+0000","last_change":"2024-09-12T03:17:09.930766+0000","last_active":"2024-09-12T03:18:41.285090+0000","last_peered":"2024-09-12T03:18:41.285090+0000","last_clean":"2024-09-12T03:18:41.285090+0000","last_became_active":"2024-09-12T03:17:09.930425+0000","last_became_peered":"2024-09-12T03:17:09.930425+0000","last_unstale":"2024-09-12T03:18:41.285090+0000","last_undegraded":"2024-09-12T03:18:41.285090+0000","last_fullsized":"2024-09-12T03:18:41.285090+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:43:53.916028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279605+0000","last_change":"2024-09-12T03:17:09.924317+0000","last_active":"2024-09-12T03:18:41.279605+0000","last_peered":"2024-09-12T03:18:41.279605+0000","last_clean":"2024-09-12T03:18:41.279605+0000","last_became_active":"2024-09-12T03:17:09.923229+0000","last_became_peered":"2024-09-12T03:17:09.923229+0000","last_unstale":"2024-09-12T03:18:41.279605+0000","last_undegraded":"2024-09-12T03:18:41.279605+0000","last_fullsized":"2024-09-12T03:18:41.279605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:00:16.713432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279741+0000","last_change":"2024-09-12T03:17:09.923400+0000","last_active":"2024-09-12T03:18:41.279741+0000","last_peered":"2024-09-12T03:18:41.279741+0000","last_clean":"2024-09-12T03:18:41.279741+0000","last_became_active":"2024-09-12T03:17:09.923024+0000","last_became_peered":"2024-09-12T03:17:09.923024+0000","last_unstale":"2024-09-12T03:18:41.279741+0000","last_undegraded":"2024-09-12T03:18:41.279741+0000","last_fullsized":"2024-09-12T03:18:41.279741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:36:43.342096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293843+0000","last_change":"2024-09-12T03:17:09.919144+0000","last_active":"2024-09-12T03:18:41.293843+0000","last_peered":"2024-09-12T03:18:41.293843+0000","last_clean":"2024-09-12T03:18:41.293843+0000","last_became_active":"2024-09-12T03:17:09.918792+0000","last_became_peered":"2024-09-12T03:17:09.918792+0000","last_unstale":"2024-09-12T03:18:41.293843+0000","last_undegraded":"2024-09-12T03:18:41.293843+0000","last_fullsized":"2024-09-12T03:18:41.293843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:02:30.353271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"82'10","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283930+0000","last_change":"2024-09-12T03:17:09.926336+0000","last_active":"2024-09-12T03:18:41.283930+0000","last_peered":"2024-09-12T03:18:41.283930+0000","last_clean":"2024-09-12T03:18:41.283930+0000","last_became_active":"2024-09-12T03:17:09.926017+0000","last_became_peered":"2024-09-12T03:17:09.926017+0000","last_unstale":"2024-09-12T03:18:41.283930+0000","last_undegraded":"2024-09-12T03:18:41.283930+0000","last_fullsized":"2024-09-12T03:18:41.283930+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:57:51.763114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286543+0000","last_change":"2024-09-12T03:17:09.923626+0000","last_active":"2024-09-12T03:18:41.286543+0000","last_peered":"2024-09-12T03:18:41.286543+0000","last_clean":"2024-09-12T03:18:41.286543+0000","last_became_active":"2024-09-12T03:17:09.923293+0000","last_became_peered":"2024-09-12T03:17:09.923293+0000","last_unstale":"2024-09-12T03:18:41.286543+0000","last_undegraded":"2024-09-12T03:18:41.286543+0000","last_fullsized":"2024-09-12T03:18:41.286543+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:17:44.129994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285760+0000","last_change":"2024-09-12T03:17:09.921097+0000","last_active":"2024-09-12T03:18:41.285760+0000","last_peered":"2024-09-12T03:18:41.285760+0000","last_clean":"2024-09-12T03:18:41.285760+0000","last_became_active":"2024-09-12T03:17:09.920104+0000","last_became_peered":"2024-09-12T03:17:09.920104+0000","last_unstale":"2024-09-12T03:18:41.285760+0000","last_undegraded":"2024-09-12T03:18:41.285760+0000","last_fullsized":"2024-09-12T03:18:41.285760+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:50.529468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279505+0000","last_change":"2024-09-12T03:17:09.932428+0000","last_active":"2024-09-12T03:18:41.279505+0000","last_peered":"2024-09-12T03:18:41.279505+0000","last_clean":"2024-09-12T03:18:41.279505+0000","last_became_active":"2024-09-12T03:17:09.932079+0000","last_became_peered":"2024-09-12T03:17:09.932079+0000","last_unstale":"2024-09-12T03:18:41.279505+0000","last_undegraded":"2024-09-12T03:18:41.279505+0000","last_fullsized":"2024-09-12T03:18:41.279505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T15:02:12.039825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"82'15","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.272567+0000","last_change":"2024-09-12T03:17:09.925356+0000","last_active":"2024-09-12T03:18:41.272567+0000","last_peered":"2024-09-12T03:18:41.272567+0000","last_clean":"2024-09-12T03:18:41.272567+0000","last_became_active":"2024-09-12T03:17:09.925016+0000","last_became_peered":"2024-09-12T03:17:09.925016+0000","last_unstale":"2024-09-12T03:18:41.272567+0000","last_undegraded":"2024-09-12T03:18:41.272567+0000","last_fullsized":"2024-09-12T03:18:41.272567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:51:00.853413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276770+0000","last_change":"2024-09-12T03:17:09.925170+0000","last_active":"2024-09-12T03:18:41.276770+0000","last_peered":"2024-09-12T03:18:41.276770+0000","last_clean":"2024-09-12T03:18:41.276770+0000","last_became_active":"2024-09-12T03:17:09.924879+0000","last_became_peered":"2024-09-12T03:17:09.924879+0000","last_unstale":"2024-09-12T03:18:41.276770+0000","last_undegraded":"2024-09-12T03:18:41.276770+0000","last_fullsized":"2024-09-12T03:18:41.276770+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:26:59.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"82'4","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.294123+0000","last_change":"2024-09-12T03:17:09.920915+0000","last_active":"2024-09-12T03:18:41.294123+0000","last_peered":"2024-09-12T03:18:41.294123+0000","last_clean":"2024-09-12T03:18:41.294123+0000","last_became_active":"2024-09-12T03:17:09.920593+0000","last_became_peered":"2024-09-12T03:17:09.920593+0000","last_unstale":"2024-09-12T03:18:41.294123+0000","last_undegraded":"2024-09-12T03:18:41.294123+0000","last_fullsized":"2024-09-12T03:18:41.294123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:50:05.380617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.291209+0000","last_change":"2024-09-12T03:17:09.925241+0000","last_active":"2024-09-12T03:18:41.291209+0000","last_peered":"2024-09-12T03:18:41.291209+0000","last_clean":"2024-09-12T03:18:41.291209+0000","last_became_active":"2024-09-12T03:17:09.924872+0000","last_became_peered":"2024-09-12T03:17:09.924872+0000","last_unstale":"2024-09-12T03:18:41.291209+0000","last_undegraded":"2024-09-12T03:18:41.291209+0000","last_fullsized":"2024-09-12T03:18:41.291209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:48:05.043478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277243+0000","last_change":"2024-09-12T03:17:09.922412+0000","last_active":"2024-09-12T03:18:41.277243+0000","last_peered":"2024-09-12T03:18:41.277243+0000","last_clean":"2024-09-12T03:18:41.277243+0000","last_became_active":"2024-09-12T03:17:09.922062+0000","last_became_peered":"2024-09-12T03:17:09.922062+0000","last_unstale":"2024-09-12T03:18:41.277243+0000","last_undegraded":"2024-09-12T03:18:41.277243+0000","last_fullsized":"2024-09-12T03:18:41.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:07:10.070964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"82'12","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283464+0000","last_change":"2024-09-12T03:17:09.926608+0000","last_active":"2024-09-12T03:18:41.283464+0000","last_peered":"2024-09-12T03:18:41.283464+0000","last_clean":"2024-09-12T03:18:41.283464+0000","last_became_active":"2024-09-12T03:17:09.926299+0000","last_became_peered":"2024-09-12T03:17:09.926299+0000","last_unstale":"2024-09-12T03:18:41.283464+0000","last_undegraded":"2024-09-12T03:18:41.283464+0000","last_fullsized":"2024-09-12T03:18:41.283464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:51:18.372146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.286008+0000","last_change":"2024-09-12T03:17:09.927307+0000","last_active":"2024-09-12T03:18:41.286008+0000","last_peered":"2024-09-12T03:18:41.286008+0000","last_clean":"2024-09-12T03:18:41.286008+0000","last_became_active":"2024-09-12T03:17:09.926994+0000","last_became_peered":"2024-09-12T03:17:09.926994+0000","last_unstale":"2024-09-12T03:18:41.286008+0000","last_undegraded":"2024-09-12T03:18:41.286008+0000","last_fullsized":"2024-09-12T03:18:41.286008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:37.579207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"82'7","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280772+0000","last_change":"2024-09-12T03:17:09.927475+0000","last_active":"2024-09-12T03:18:41.280772+0000","last_peered":"2024-09-12T03:18:41.280772+0000","last_clean":"2024-09-12T03:18:41.280772+0000","last_became_active":"2024-09-12T03:17:09.927180+0000","last_became_peered":"2024-09-12T03:17:09.927180+0000","last_unstale":"2024-09-12T03:18:41.280772+0000","last_undegraded":"2024-09-12T03:18:41.280772+0000","last_fullsized":"2024-09-12T03:18:41.280772+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:37:13.745956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"82'7","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:21.985523+0000","last_change":"2024-09-12T03:17:07.832267+0000","last_active":"2024-09-12T03:19:21.985523+0000","last_peered":"2024-09-12T03:19:21.985523+0000","last_clean":"2024-09-12T03:19:21.985523+0000","last_became_active":"2024-09-12T03:17:07.831949+0000","last_became_peered":"2024-09-12T03:17:07.831949+0000","last_unstale":"2024-09-12T03:19:21.985523+0000","last_undegraded":"2024-09-12T03:19:21.985523+0000","last_fullsized":"2024-09-12T03:19:21.985523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:41.350842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1156,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281945+0000","last_change":"2024-09-12T03:17:09.931586+0000","last_active":"2024-09-12T03:18:41.281945+0000","last_peered":"2024-09-12T03:18:41.281945+0000","last_clean":"2024-09-12T03:18:41.281945+0000","last_became_active":"2024-09-12T03:17:09.931223+0000","last_became_peered":"2024-09-12T03:17:09.931223+0000","last_unstale":"2024-09-12T03:18:41.281945+0000","last_undegraded":"2024-09-12T03:18:41.281945+0000","last_fullsized":"2024-09-12T03:18:41.281945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:00:49.028410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"80'7","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.222032+0000","last_change":"2024-09-12T03:17:07.836523+0000","last_active":"2024-09-12T03:18:13.222032+0000","last_peered":"2024-09-12T03:18:13.222032+0000","last_clean":"2024-09-12T03:18:13.222032+0000","last_became_active":"2024-09-12T03:17:07.835001+0000","last_became_peered":"2024-09-12T03:17:07.835001+0000","last_unstale":"2024-09-12T03:18:13.222032+0000","last_undegraded":"2024-09-12T03:18:13.222032+0000","last_fullsized":"2024-09-12T03:18:13.222032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:22.964161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"80'3","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220165+0000","last_change":"2024-09-12T03:17:07.825563+0000","last_active":"2024-09-12T03:18:13.220165+0000","last_peered":"2024-09-12T03:18:13.220165+0000","last_clean":"2024-09-12T03:18:13.220165+0000","last_became_active":"2024-09-12T03:17:07.825263+0000","last_became_peered":"2024-09-12T03:17:07.825263+0000","last_unstale":"2024-09-12T03:18:13.220165+0000","last_undegraded":"2024-09-12T03:18:13.220165+0000","last_fullsized":"2024-09-12T03:18:13.220165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:52.030659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"82'8","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284475+0000","last_change":"2024-09-12T03:17:09.924522+0000","last_active":"2024-09-12T03:18:41.284475+0000","last_peered":"2024-09-12T03:18:41.284475+0000","last_clean":"2024-09-12T03:18:41.284475+0000","last_became_active":"2024-09-12T03:17:09.923460+0000","last_became_peered":"2024-09-12T03:17:09.923460+0000","last_unstale":"2024-09-12T03:18:41.284475+0000","last_undegraded":"2024-09-12T03:18:41.284475+0000","last_fullsized":"2024-09-12T03:18:41.284475+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:46.153632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225803+0000","last_change":"2024-09-12T03:17:07.829898+0000","last_active":"2024-09-12T03:18:13.225803+0000","last_peered":"2024-09-12T03:18:13.225803+0000","last_clean":"2024-09-12T03:18:13.225803+0000","last_became_active":"2024-09-12T03:17:07.829574+0000","last_became_peered":"2024-09-12T03:17:07.829574+0000","last_unstale":"2024-09-12T03:18:13.225803+0000","last_undegraded":"2024-09-12T03:18:13.225803+0000","last_fullsized":"2024-09-12T03:18:13.225803+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:55:18.037173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277135+0000","last_change":"2024-09-12T03:17:09.928277+0000","last_active":"2024-09-12T03:18:41.277135+0000","last_peered":"2024-09-12T03:18:41.277135+0000","last_clean":"2024-09-12T03:18:41.277135+0000","last_became_active":"2024-09-12T03:17:09.927918+0000","last_became_peered":"2024-09-12T03:17:09.927918+0000","last_unstale":"2024-09-12T03:18:41.277135+0000","last_undegraded":"2024-09-12T03:18:41.277135+0000","last_fullsized":"2024-09-12T03:18:41.277135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:24:25.587115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"82'5","reported_seq":276,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.370761+0000","last_change":"2024-09-12T03:17:07.835859+0000","last_active":"2024-09-12T03:19:24.370761+0000","last_peered":"2024-09-12T03:19:24.370761+0000","last_clean":"2024-09-12T03:19:24.370761+0000","last_became_active":"2024-09-12T03:17:07.835487+0000","last_became_peered":"2024-09-12T03:17:07.835487+0000","last_unstale":"2024-09-12T03:19:24.370761+0000","last_undegraded":"2024-09-12T03:19:24.370761+0000","last_fullsized":"2024-09-12T03:19:24.370761+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:19:36.821929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":249,"num_read_kb":249,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"82'10","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284207+0000","last_change":"2024-09-12T03:17:09.932105+0000","last_active":"2024-09-12T03:18:41.284207+0000","last_peered":"2024-09-12T03:18:41.284207+0000","last_clean":"2024-09-12T03:18:41.284207+0000","last_became_active":"2024-09-12T03:17:09.931784+0000","last_became_peered":"2024-09-12T03:17:09.931784+0000","last_unstale":"2024-09-12T03:18:41.284207+0000","last_undegraded":"2024-09-12T03:18:41.284207+0000","last_fullsized":"2024-09-12T03:18:41.284207+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:35:24.002245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'4","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224474+0000","last_change":"2024-09-12T03:17:07.834790+0000","last_active":"2024-09-12T03:18:13.224474+0000","last_peered":"2024-09-12T03:18:13.224474+0000","last_clean":"2024-09-12T03:18:13.224474+0000","last_became_active":"2024-09-12T03:17:07.834319+0000","last_became_peered":"2024-09-12T03:17:07.834319+0000","last_unstale":"2024-09-12T03:18:13.224474+0000","last_undegraded":"2024-09-12T03:18:13.224474+0000","last_fullsized":"2024-09-12T03:18:13.224474+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:43:19.816407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293169+0000","last_change":"2024-09-12T03:17:09.932236+0000","last_active":"2024-09-12T03:18:41.293169+0000","last_peered":"2024-09-12T03:18:41.293169+0000","last_clean":"2024-09-12T03:18:41.293169+0000","last_became_active":"2024-09-12T03:17:09.931870+0000","last_became_peered":"2024-09-12T03:17:09.931870+0000","last_unstale":"2024-09-12T03:18:41.293169+0000","last_undegraded":"2024-09-12T03:18:41.293169+0000","last_fullsized":"2024-09-12T03:18:41.293169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:27:44.025329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227074+0000","last_change":"2024-09-12T03:17:07.833479+0000","last_active":"2024-09-12T03:18:13.227074+0000","last_peered":"2024-09-12T03:18:13.227074+0000","last_clean":"2024-09-12T03:18:13.227074+0000","last_became_active":"2024-09-12T03:17:07.833095+0000","last_became_peered":"2024-09-12T03:17:07.833095+0000","last_unstale":"2024-09-12T03:18:13.227074+0000","last_undegraded":"2024-09-12T03:18:13.227074+0000","last_fullsized":"2024-09-12T03:18:13.227074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:51:22.258980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"82'11","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283547+0000","last_change":"2024-09-12T03:17:09.922653+0000","last_active":"2024-09-12T03:18:41.283547+0000","last_peered":"2024-09-12T03:18:41.283547+0000","last_clean":"2024-09-12T03:18:41.283547+0000","last_became_active":"2024-09-12T03:17:09.921345+0000","last_became_peered":"2024-09-12T03:17:09.921345+0000","last_unstale":"2024-09-12T03:18:41.283547+0000","last_undegraded":"2024-09-12T03:18:41.283547+0000","last_fullsized":"2024-09-12T03:18:41.283547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:32:51.024862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'36","reported_seq":322,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.989166+0000","last_change":"2024-09-12T03:17:07.833269+0000","last_active":"2024-09-12T03:19:24.989166+0000","last_peered":"2024-09-12T03:19:24.989166+0000","last_clean":"2024-09-12T03:19:24.989166+0000","last_became_active":"2024-09-12T03:17:07.832886+0000","last_became_peered":"2024-09-12T03:17:07.832886+0000","last_unstale":"2024-09-12T03:19:24.989166+0000","last_undegraded":"2024-09-12T03:19:24.989166+0000","last_fullsized":"2024-09-12T03:19:24.989166+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:02.240459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65694,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":264,"num_read_kb":197,"num_write":38,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278864+0000","last_change":"2024-09-12T03:17:09.926016+0000","last_active":"2024-09-12T03:18:41.278864+0000","last_peered":"2024-09-12T03:18:41.278864+0000","last_clean":"2024-09-12T03:18:41.278864+0000","last_became_active":"2024-09-12T03:17:09.925686+0000","last_became_peered":"2024-09-12T03:17:09.925686+0000","last_unstale":"2024-09-12T03:18:41.278864+0000","last_undegraded":"2024-09-12T03:18:41.278864+0000","last_fullsized":"2024-09-12T03:18:41.278864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:33:31.691949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'9","reported_seq":33,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221679+0000","last_change":"2024-09-12T03:17:07.831287+0000","last_active":"2024-09-12T03:18:13.221679+0000","last_peered":"2024-09-12T03:18:13.221679+0000","last_clean":"2024-09-12T03:18:13.221679+0000","last_became_active":"2024-09-12T03:17:07.830960+0000","last_became_peered":"2024-09-12T03:17:07.830960+0000","last_unstale":"2024-09-12T03:18:13.221679+0000","last_undegraded":"2024-09-12T03:18:13.221679+0000","last_fullsized":"2024-09-12T03:18:13.221679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:12:06.277918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.281254+0000","last_change":"2024-09-12T03:17:09.919032+0000","last_active":"2024-09-12T03:18:41.281254+0000","last_peered":"2024-09-12T03:18:41.281254+0000","last_clean":"2024-09-12T03:18:41.281254+0000","last_became_active":"2024-09-12T03:17:09.918663+0000","last_became_peered":"2024-09-12T03:17:09.918663+0000","last_unstale":"2024-09-12T03:18:41.281254+0000","last_undegraded":"2024-09-12T03:18:41.281254+0000","last_fullsized":"2024-09-12T03:18:41.281254+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:51:22.450770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.225737+0000","last_change":"2024-09-12T03:17:07.829748+0000","last_active":"2024-09-12T03:18:13.225737+0000","last_peered":"2024-09-12T03:18:13.225737+0000","last_clean":"2024-09-12T03:18:13.225737+0000","last_became_active":"2024-09-12T03:17:07.829285+0000","last_became_peered":"2024-09-12T03:17:07.829285+0000","last_unstale":"2024-09-12T03:18:13.225737+0000","last_undegraded":"2024-09-12T03:18:13.225737+0000","last_fullsized":"2024-09-12T03:18:13.225737+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:51:45.581971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"82'15","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282886+0000","last_change":"2024-09-12T03:17:09.931644+0000","last_active":"2024-09-12T03:18:41.282886+0000","last_peered":"2024-09-12T03:18:41.282886+0000","last_clean":"2024-09-12T03:18:41.282886+0000","last_became_active":"2024-09-12T03:17:09.931329+0000","last_became_peered":"2024-09-12T03:17:09.931329+0000","last_unstale":"2024-09-12T03:18:41.282886+0000","last_undegraded":"2024-09-12T03:18:41.282886+0000","last_fullsized":"2024-09-12T03:18:41.282886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:08:06.447066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"82'7","reported_seq":162,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:27.002695+0000","last_change":"2024-09-12T03:17:07.833942+0000","last_active":"2024-09-12T03:19:27.002695+0000","last_peered":"2024-09-12T03:19:27.002695+0000","last_clean":"2024-09-12T03:19:27.002695+0000","last_became_active":"2024-09-12T03:17:07.833625+0000","last_became_peered":"2024-09-12T03:17:07.833625+0000","last_unstale":"2024-09-12T03:19:27.002695+0000","last_undegraded":"2024-09-12T03:19:27.002695+0000","last_fullsized":"2024-09-12T03:19:27.002695+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:28.386119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"82'6","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.270905+0000","last_change":"2024-09-12T03:18:13.225722+0000","last_active":"2024-09-12T03:18:41.270905+0000","last_peered":"2024-09-12T03:18:41.270905+0000","last_clean":"2024-09-12T03:18:41.270905+0000","last_became_active":"2024-09-12T03:18:13.225351+0000","last_became_peered":"2024-09-12T03:18:13.225351+0000","last_unstale":"2024-09-12T03:18:41.270905+0000","last_undegraded":"2024-09-12T03:18:41.270905+0000","last_fullsized":"2024-09-12T03:18:41.270905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:15:54.376130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223232+0000","last_change":"2024-09-12T03:17:07.829213+0000","last_active":"2024-09-12T03:18:13.223232+0000","last_peered":"2024-09-12T03:18:13.223232+0000","last_clean":"2024-09-12T03:18:13.223232+0000","last_became_active":"2024-09-12T03:17:07.828647+0000","last_became_peered":"2024-09-12T03:17:07.828647+0000","last_unstale":"2024-09-12T03:18:13.223232+0000","last_undegraded":"2024-09-12T03:18:13.223232+0000","last_fullsized":"2024-09-12T03:18:13.223232+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:11:47.599811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"82'6","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276204+0000","last_change":"2024-09-12T03:17:09.924397+0000","last_active":"2024-09-12T03:18:41.276204+0000","last_peered":"2024-09-12T03:18:41.276204+0000","last_clean":"2024-09-12T03:18:41.276204+0000","last_became_active":"2024-09-12T03:17:09.923749+0000","last_became_peered":"2024-09-12T03:17:09.923749+0000","last_unstale":"2024-09-12T03:18:41.276204+0000","last_undegraded":"2024-09-12T03:18:41.276204+0000","last_fullsized":"2024-09-12T03:18:41.276204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:11:01.516471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.235239+0000","last_change":"2024-09-12T03:18:13.234800+0000","last_active":"2024-09-12T03:18:13.235239+0000","last_peered":"2024-09-12T03:18:13.235239+0000","last_clean":"2024-09-12T03:18:13.235239+0000","last_became_active":"2024-09-12T03:18:13.229361+0000","last_became_peered":"2024-09-12T03:18:13.229361+0000","last_unstale":"2024-09-12T03:18:13.235239+0000","last_undegraded":"2024-09-12T03:18:13.235239+0000","last_fullsized":"2024-09-12T03:18:13.235239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:36:07.371564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276871+0000","last_change":"2024-09-12T03:17:09.925110+0000","last_active":"2024-09-12T03:18:41.276871+0000","last_peered":"2024-09-12T03:18:41.276871+0000","last_clean":"2024-09-12T03:18:41.276871+0000","last_became_active":"2024-09-12T03:17:09.924768+0000","last_became_peered":"2024-09-12T03:17:09.924768+0000","last_unstale":"2024-09-12T03:18:41.276871+0000","last_undegraded":"2024-09-12T03:18:41.276871+0000","last_fullsized":"2024-09-12T03:18:41.276871+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:43:57.905977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"80'5","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.220643+0000","last_change":"2024-09-12T03:17:07.830258+0000","last_active":"2024-09-12T03:18:13.220643+0000","last_peered":"2024-09-12T03:18:13.220643+0000","last_clean":"2024-09-12T03:18:13.220643+0000","last_became_active":"2024-09-12T03:17:07.829434+0000","last_became_peered":"2024-09-12T03:17:07.829434+0000","last_unstale":"2024-09-12T03:18:13.220643+0000","last_undegraded":"2024-09-12T03:18:13.220643+0000","last_fullsized":"2024-09-12T03:18:13.220643+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:39:50.823252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.279803+0000","last_change":"2024-09-12T03:17:09.924628+0000","last_active":"2024-09-12T03:18:41.279803+0000","last_peered":"2024-09-12T03:18:41.279803+0000","last_clean":"2024-09-12T03:18:41.279803+0000","last_became_active":"2024-09-12T03:17:09.924297+0000","last_became_peered":"2024-09-12T03:17:09.924297+0000","last_unstale":"2024-09-12T03:18:41.279803+0000","last_undegraded":"2024-09-12T03:18:41.279803+0000","last_fullsized":"2024-09-12T03:18:41.279803+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:52:23.619606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'3","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227246+0000","last_change":"2024-09-12T03:17:07.832624+0000","last_active":"2024-09-12T03:18:13.227246+0000","last_peered":"2024-09-12T03:18:13.227246+0000","last_clean":"2024-09-12T03:18:13.227246+0000","last_became_active":"2024-09-12T03:17:07.832276+0000","last_became_peered":"2024-09-12T03:17:07.832276+0000","last_unstale":"2024-09-12T03:18:13.227246+0000","last_undegraded":"2024-09-12T03:18:13.227246+0000","last_fullsized":"2024-09-12T03:18:13.227246+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:17:09.180190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"82'7","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.271928+0000","last_change":"2024-09-12T03:18:13.235706+0000","last_active":"2024-09-12T03:18:41.271928+0000","last_peered":"2024-09-12T03:18:41.271928+0000","last_clean":"2024-09-12T03:18:41.271928+0000","last_became_active":"2024-09-12T03:18:13.227629+0000","last_became_peered":"2024-09-12T03:18:13.227629+0000","last_unstale":"2024-09-12T03:18:41.271928+0000","last_undegraded":"2024-09-12T03:18:41.271928+0000","last_fullsized":"2024-09-12T03:18:41.271928+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:24:31.276607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'5","reported_seq":93,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:25.373058+0000","last_change":"2024-09-12T03:18:13.235968+0000","last_active":"2024-09-12T03:19:25.373058+0000","last_peered":"2024-09-12T03:19:25.373058+0000","last_clean":"2024-09-12T03:19:25.373058+0000","last_became_active":"2024-09-12T03:18:13.228737+0000","last_became_peered":"2024-09-12T03:18:13.228737+0000","last_unstale":"2024-09-12T03:19:25.373058+0000","last_undegraded":"2024-09-12T03:19:25.373058+0000","last_fullsized":"2024-09-12T03:19:25.373058+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:32:42.530619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":75,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1170,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.280805+0000","last_change":"2024-09-12T03:17:09.929273+0000","last_active":"2024-09-12T03:18:41.280805+0000","last_peered":"2024-09-12T03:18:41.280805+0000","last_clean":"2024-09-12T03:18:41.280805+0000","last_became_active":"2024-09-12T03:17:09.928976+0000","last_became_peered":"2024-09-12T03:17:09.928976+0000","last_unstale":"2024-09-12T03:18:41.280805+0000","last_undegraded":"2024-09-12T03:18:41.280805+0000","last_fullsized":"2024-09-12T03:18:41.280805+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:27:35.976779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":339,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223843+0000","last_change":"2024-09-12T03:15:53.079272+0000","last_active":"2024-09-12T03:18:13.223843+0000","last_peered":"2024-09-12T03:18:13.223843+0000","last_clean":"2024-09-12T03:18:13.223843+0000","last_became_active":"2024-09-12T03:15:53.078917+0000","last_became_peered":"2024-09-12T03:15:53.078917+0000","last_unstale":"2024-09-12T03:18:13.223843+0000","last_undegraded":"2024-09-12T03:18:13.223843+0000","last_fullsized":"2024-09-12T03:18:13.223843+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:12:25.705158+0000","last_clean_scrub_stamp":"2024-09-12T03:12:25.705158+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:04:58.884248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224003+0000","last_change":"2024-09-12T03:17:07.832910+0000","last_active":"2024-09-12T03:18:13.224003+0000","last_peered":"2024-09-12T03:18:13.224003+0000","last_clean":"2024-09-12T03:18:13.224003+0000","last_became_active":"2024-09-12T03:17:07.832353+0000","last_became_peered":"2024-09-12T03:17:07.832353+0000","last_unstale":"2024-09-12T03:18:13.224003+0000","last_undegraded":"2024-09-12T03:18:13.224003+0000","last_fullsized":"2024-09-12T03:18:13.224003+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:16:04.469111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282907+0000","last_change":"2024-09-12T03:17:09.927432+0000","last_active":"2024-09-12T03:18:41.282907+0000","last_peered":"2024-09-12T03:18:41.282907+0000","last_clean":"2024-09-12T03:18:41.282907+0000","last_became_active":"2024-09-12T03:17:09.927094+0000","last_became_peered":"2024-09-12T03:17:09.927094+0000","last_unstale":"2024-09-12T03:18:41.282907+0000","last_undegraded":"2024-09-12T03:18:41.282907+0000","last_fullsized":"2024-09-12T03:18:41.282907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:25:12.142940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"80'4","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223773+0000","last_change":"2024-09-12T03:17:07.834450+0000","last_active":"2024-09-12T03:18:13.223773+0000","last_peered":"2024-09-12T03:18:13.223773+0000","last_clean":"2024-09-12T03:18:13.223773+0000","last_became_active":"2024-09-12T03:17:07.834110+0000","last_became_peered":"2024-09-12T03:17:07.834110+0000","last_unstale":"2024-09-12T03:18:13.223773+0000","last_undegraded":"2024-09-12T03:18:13.223773+0000","last_fullsized":"2024-09-12T03:18:13.223773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:06:53.285205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283494+0000","last_change":"2024-09-12T03:17:09.921721+0000","last_active":"2024-09-12T03:18:41.283494+0000","last_peered":"2024-09-12T03:18:41.283494+0000","last_clean":"2024-09-12T03:18:41.283494+0000","last_became_active":"2024-09-12T03:17:09.921420+0000","last_became_peered":"2024-09-12T03:17:09.921420+0000","last_unstale":"2024-09-12T03:18:41.283494+0000","last_undegraded":"2024-09-12T03:18:41.283494+0000","last_fullsized":"2024-09-12T03:18:41.283494+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:21:05.851955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"82'6","reported_seq":153,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:20.993281+0000","last_change":"2024-09-12T03:17:07.828779+0000","last_active":"2024-09-12T03:19:20.993281+0000","last_peered":"2024-09-12T03:19:20.993281+0000","last_clean":"2024-09-12T03:19:20.993281+0000","last_became_active":"2024-09-12T03:17:07.828453+0000","last_became_peered":"2024-09-12T03:17:07.828453+0000","last_unstale":"2024-09-12T03:19:20.993281+0000","last_undegraded":"2024-09-12T03:19:20.993281+0000","last_fullsized":"2024-09-12T03:19:20.993281+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:33:33.166216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.278306+0000","last_change":"2024-09-12T03:17:09.921677+0000","last_active":"2024-09-12T03:18:41.278306+0000","last_peered":"2024-09-12T03:18:41.278306+0000","last_clean":"2024-09-12T03:18:41.278306+0000","last_became_active":"2024-09-12T03:17:09.921350+0000","last_became_peered":"2024-09-12T03:17:09.921350+0000","last_unstale":"2024-09-12T03:18:41.278306+0000","last_undegraded":"2024-09-12T03:18:41.278306+0000","last_fullsized":"2024-09-12T03:18:41.278306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:14.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"82'6","reported_seq":157,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:26.011012+0000","last_change":"2024-09-12T03:17:07.832432+0000","last_active":"2024-09-12T03:19:26.011012+0000","last_peered":"2024-09-12T03:19:26.011012+0000","last_clean":"2024-09-12T03:19:26.011012+0000","last_became_active":"2024-09-12T03:17:07.832128+0000","last_became_peered":"2024-09-12T03:17:07.832128+0000","last_unstale":"2024-09-12T03:19:26.011012+0000","last_undegraded":"2024-09-12T03:19:26.011012+0000","last_fullsized":"2024-09-12T03:19:26.011012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:54:55.613163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":896,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.276594+0000","last_change":"2024-09-12T03:17:09.935124+0000","last_active":"2024-09-12T03:18:41.276594+0000","last_peered":"2024-09-12T03:18:41.276594+0000","last_clean":"2024-09-12T03:18:41.276594+0000","last_became_active":"2024-09-12T03:17:09.934620+0000","last_became_peered":"2024-09-12T03:17:09.934620+0000","last_unstale":"2024-09-12T03:18:41.276594+0000","last_undegraded":"2024-09-12T03:18:41.276594+0000","last_fullsized":"2024-09-12T03:18:41.276594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:43:17.819413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"80'1","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.223060+0000","last_change":"2024-09-12T03:17:07.828817+0000","last_active":"2024-09-12T03:18:13.223060+0000","last_peered":"2024-09-12T03:18:13.223060+0000","last_clean":"2024-09-12T03:18:13.223060+0000","last_became_active":"2024-09-12T03:17:07.828498+0000","last_became_peered":"2024-09-12T03:17:07.828498+0000","last_unstale":"2024-09-12T03:18:13.223060+0000","last_undegraded":"2024-09-12T03:18:13.223060+0000","last_fullsized":"2024-09-12T03:18:13.223060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:10.195139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.277898+0000","last_change":"2024-09-12T03:17:09.927733+0000","last_active":"2024-09-12T03:18:41.277898+0000","last_peered":"2024-09-12T03:18:41.277898+0000","last_clean":"2024-09-12T03:18:41.277898+0000","last_became_active":"2024-09-12T03:17:09.927442+0000","last_became_peered":"2024-09-12T03:17:09.927442+0000","last_unstale":"2024-09-12T03:18:41.277898+0000","last_undegraded":"2024-09-12T03:18:41.277898+0000","last_fullsized":"2024-09-12T03:18:41.277898+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:55:36.182670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"80'4","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224168+0000","last_change":"2024-09-12T03:17:07.830319+0000","last_active":"2024-09-12T03:18:13.224168+0000","last_peered":"2024-09-12T03:18:13.224168+0000","last_clean":"2024-09-12T03:18:13.224168+0000","last_became_active":"2024-09-12T03:17:07.829980+0000","last_became_peered":"2024-09-12T03:17:07.829980+0000","last_unstale":"2024-09-12T03:18:13.224168+0000","last_undegraded":"2024-09-12T03:18:13.224168+0000","last_fullsized":"2024-09-12T03:18:13.224168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:29:51.530243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282537+0000","last_change":"2024-09-12T03:17:09.918286+0000","last_active":"2024-09-12T03:18:41.282537+0000","last_peered":"2024-09-12T03:18:41.282537+0000","last_clean":"2024-09-12T03:18:41.282537+0000","last_became_active":"2024-09-12T03:17:09.917987+0000","last_became_peered":"2024-09-12T03:17:09.917987+0000","last_unstale":"2024-09-12T03:18:41.282537+0000","last_undegraded":"2024-09-12T03:18:41.282537+0000","last_fullsized":"2024-09-12T03:18:41.282537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:19:47.254705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'17","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:12.931837+0000","last_change":"2024-09-12T03:17:07.827865+0000","last_active":"2024-09-12T03:19:12.931837+0000","last_peered":"2024-09-12T03:19:12.931837+0000","last_clean":"2024-09-12T03:19:12.931837+0000","last_became_active":"2024-09-12T03:17:07.827561+0000","last_became_peered":"2024-09-12T03:17:07.827561+0000","last_unstale":"2024-09-12T03:19:12.931837+0000","last_undegraded":"2024-09-12T03:19:12.931837+0000","last_fullsized":"2024-09-12T03:19:12.931837+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:20:42.223462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285295+0000","last_change":"2024-09-12T03:17:09.923821+0000","last_active":"2024-09-12T03:18:41.285295+0000","last_peered":"2024-09-12T03:18:41.285295+0000","last_clean":"2024-09-12T03:18:41.285295+0000","last_became_active":"2024-09-12T03:17:09.923119+0000","last_became_peered":"2024-09-12T03:17:09.923119+0000","last_unstale":"2024-09-12T03:18:41.285295+0000","last_undegraded":"2024-09-12T03:18:41.285295+0000","last_fullsized":"2024-09-12T03:18:41.285295+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:32:21.148185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'7","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224395+0000","last_change":"2024-09-12T03:17:07.981839+0000","last_active":"2024-09-12T03:18:13.224395+0000","last_peered":"2024-09-12T03:18:13.224395+0000","last_clean":"2024-09-12T03:18:13.224395+0000","last_became_active":"2024-09-12T03:17:07.981512+0000","last_became_peered":"2024-09-12T03:17:07.981512+0000","last_unstale":"2024-09-12T03:18:13.224395+0000","last_undegraded":"2024-09-12T03:18:13.224395+0000","last_fullsized":"2024-09-12T03:18:13.224395+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:13:31.840686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"82'7","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.289155+0000","last_change":"2024-09-12T03:17:09.921209+0000","last_active":"2024-09-12T03:18:41.289155+0000","last_peered":"2024-09-12T03:18:41.289155+0000","last_clean":"2024-09-12T03:18:41.289155+0000","last_became_active":"2024-09-12T03:17:09.920896+0000","last_became_peered":"2024-09-12T03:17:09.920896+0000","last_unstale":"2024-09-12T03:18:41.289155+0000","last_undegraded":"2024-09-12T03:18:41.289155+0000","last_fullsized":"2024-09-12T03:18:41.289155+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:09:05.748411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.230666+0000","last_change":"2024-09-12T03:17:07.830705+0000","last_active":"2024-09-12T03:18:13.230666+0000","last_peered":"2024-09-12T03:18:13.230666+0000","last_clean":"2024-09-12T03:18:13.230666+0000","last_became_active":"2024-09-12T03:17:07.830287+0000","last_became_peered":"2024-09-12T03:17:07.830287+0000","last_unstale":"2024-09-12T03:18:13.230666+0000","last_undegraded":"2024-09-12T03:18:13.230666+0000","last_fullsized":"2024-09-12T03:18:13.230666+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:39.405782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"82'2","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.275190+0000","last_change":"2024-09-12T03:17:09.929772+0000","last_active":"2024-09-12T03:18:41.275190+0000","last_peered":"2024-09-12T03:18:41.275190+0000","last_clean":"2024-09-12T03:18:41.275190+0000","last_became_active":"2024-09-12T03:17:09.929165+0000","last_became_peered":"2024-09-12T03:17:09.929165+0000","last_unstale":"2024-09-12T03:18:41.275190+0000","last_undegraded":"2024-09-12T03:18:41.275190+0000","last_fullsized":"2024-09-12T03:18:41.275190+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:26:54.966841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'7","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.227726+0000","last_change":"2024-09-12T03:17:07.832480+0000","last_active":"2024-09-12T03:18:13.227726+0000","last_peered":"2024-09-12T03:18:13.227726+0000","last_clean":"2024-09-12T03:18:13.227726+0000","last_became_active":"2024-09-12T03:17:07.832149+0000","last_became_peered":"2024-09-12T03:17:07.832149+0000","last_unstale":"2024-09-12T03:18:13.227726+0000","last_undegraded":"2024-09-12T03:18:13.227726+0000","last_fullsized":"2024-09-12T03:18:13.227726+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:20.104571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"82'9","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285539+0000","last_change":"2024-09-12T03:17:09.922698+0000","last_active":"2024-09-12T03:18:41.285539+0000","last_peered":"2024-09-12T03:18:41.285539+0000","last_clean":"2024-09-12T03:18:41.285539+0000","last_became_active":"2024-09-12T03:17:09.921899+0000","last_became_peered":"2024-09-12T03:17:09.921899+0000","last_unstale":"2024-09-12T03:18:41.285539+0000","last_undegraded":"2024-09-12T03:18:41.285539+0000","last_fullsized":"2024-09-12T03:18:41.285539+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:08:03.620895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'3","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.221452+0000","last_change":"2024-09-12T03:17:07.827843+0000","last_active":"2024-09-12T03:18:13.221452+0000","last_peered":"2024-09-12T03:18:13.221452+0000","last_clean":"2024-09-12T03:18:13.221452+0000","last_became_active":"2024-09-12T03:17:07.827515+0000","last_became_peered":"2024-09-12T03:17:07.827515+0000","last_unstale":"2024-09-12T03:18:13.221452+0000","last_undegraded":"2024-09-12T03:18:13.221452+0000","last_fullsized":"2024-09-12T03:18:13.221452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:06:33.508032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"82'8","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.285008+0000","last_change":"2024-09-12T03:17:09.930510+0000","last_active":"2024-09-12T03:18:41.285008+0000","last_peered":"2024-09-12T03:18:41.285008+0000","last_clean":"2024-09-12T03:18:41.285008+0000","last_became_active":"2024-09-12T03:17:09.929472+0000","last_became_peered":"2024-09-12T03:17:09.929472+0000","last_unstale":"2024-09-12T03:18:41.285008+0000","last_undegraded":"2024-09-12T03:18:41.285008+0000","last_fullsized":"2024-09-12T03:18:41.285008+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:38:40.119522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":40,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.236604+0000","last_change":"2024-09-12T03:18:13.236108+0000","last_active":"2024-09-12T03:18:13.236604+0000","last_peered":"2024-09-12T03:18:13.236604+0000","last_clean":"2024-09-12T03:18:13.236604+0000","last_became_active":"2024-09-12T03:18:13.231651+0000","last_became_peered":"2024-09-12T03:18:13.231651+0000","last_unstale":"2024-09-12T03:18:13.236604+0000","last_undegraded":"2024-09-12T03:18:13.236604+0000","last_fullsized":"2024-09-12T03:18:13.236604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:40:18.593143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282965+0000","last_change":"2024-09-12T03:17:09.929336+0000","last_active":"2024-09-12T03:18:41.282965+0000","last_peered":"2024-09-12T03:18:41.282965+0000","last_clean":"2024-09-12T03:18:41.282965+0000","last_became_active":"2024-09-12T03:17:09.928864+0000","last_became_peered":"2024-09-12T03:17:09.928864+0000","last_unstale":"2024-09-12T03:18:41.282965+0000","last_undegraded":"2024-09-12T03:18:41.282965+0000","last_fullsized":"2024-09-12T03:18:41.282965+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:40:33.782703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"82'15","reported_seq":38,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.282603+0000","last_change":"2024-09-12T03:18:13.231726+0000","last_active":"2024-09-12T03:18:41.282603+0000","last_peered":"2024-09-12T03:18:41.282603+0000","last_clean":"2024-09-12T03:18:41.282603+0000","last_became_active":"2024-09-12T03:18:13.231325+0000","last_became_peered":"2024-09-12T03:18:13.231325+0000","last_unstale":"2024-09-12T03:18:41.282603+0000","last_undegraded":"2024-09-12T03:18:41.282603+0000","last_fullsized":"2024-09-12T03:18:41.282603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:43:28.578702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"80'5","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.241139+0000","last_change":"2024-09-12T03:18:13.240760+0000","last_active":"2024-09-12T03:18:13.241139+0000","last_peered":"2024-09-12T03:18:13.241139+0000","last_clean":"2024-09-12T03:18:13.241139+0000","last_became_active":"2024-09-12T03:18:13.231130+0000","last_became_peered":"2024-09-12T03:18:13.231130+0000","last_unstale":"2024-09-12T03:18:13.241139+0000","last_undegraded":"2024-09-12T03:18:13.241139+0000","last_fullsized":"2024-09-12T03:18:13.241139+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:04:26.031038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"82'9","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.284492+0000","last_change":"2024-09-12T03:17:09.930725+0000","last_active":"2024-09-12T03:18:41.284492+0000","last_peered":"2024-09-12T03:18:41.284492+0000","last_clean":"2024-09-12T03:18:41.284492+0000","last_became_active":"2024-09-12T03:17:09.930395+0000","last_became_peered":"2024-09-12T03:17:09.930395+0000","last_unstale":"2024-09-12T03:18:41.284492+0000","last_undegraded":"2024-09-12T03:18:41.284492+0000","last_fullsized":"2024-09-12T03:18:41.284492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:13:20.663539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"80'12","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.224011+0000","last_change":"2024-09-12T03:17:07.834521+0000","last_active":"2024-09-12T03:18:13.224011+0000","last_peered":"2024-09-12T03:18:13.224011+0000","last_clean":"2024-09-12T03:18:13.224011+0000","last_became_active":"2024-09-12T03:17:07.833985+0000","last_became_peered":"2024-09-12T03:17:07.833985+0000","last_unstale":"2024-09-12T03:18:13.224011+0000","last_undegraded":"2024-09-12T03:18:13.224011+0000","last_fullsized":"2024-09-12T03:18:13.224011+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:41:38.487515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"82'6","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.293617+0000","last_change":"2024-09-12T03:17:09.931362+0000","last_active":"2024-09-12T03:18:41.293617+0000","last_peered":"2024-09-12T03:18:41.293617+0000","last_clean":"2024-09-12T03:18:41.293617+0000","last_became_active":"2024-09-12T03:17:09.930901+0000","last_became_peered":"2024-09-12T03:17:09.930901+0000","last_unstale":"2024-09-12T03:18:41.293617+0000","last_undegraded":"2024-09-12T03:18:41.293617+0000","last_fullsized":"2024-09-12T03:18:41.293617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:45:28.246511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:13.228479+0000","last_change":"2024-09-12T03:17:07.829092+0000","last_active":"2024-09-12T03:18:13.228479+0000","last_peered":"2024-09-12T03:18:13.228479+0000","last_clean":"2024-09-12T03:18:13.228479+0000","last_became_active":"2024-09-12T03:17:07.828762+0000","last_became_peered":"2024-09-12T03:17:07.828762+0000","last_unstale":"2024-09-12T03:18:13.228479+0000","last_undegraded":"2024-09-12T03:18:13.228479+0000","last_fullsized":"2024-09-12T03:18:13.228479+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:22:39.906704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"82'5","reported_seq":23,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:18:41.283560+0000","last_change":"2024-09-12T03:17:09.926101+0000","last_active":"2024-09-12T03:18:41.283560+0000","last_peered":"2024-09-12T03:18:41.283560+0000","last_clean":"2024-09-12T03:18:41.283560+0000","last_became_active":"2024-09-12T03:17:09.925781+0000","last_became_peered":"2024-09-12T03:17:09.925781+0000","last_unstale":"2024-09-12T03:18:41.283560+0000","last_undegraded":"2024-09-12T03:18:41.283560+0000","last_fullsized":"2024-09-12T03:18:41.283560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:08.894426+0000","last_clean_scrub_stamp":"2024-09-12T03:17:08.894426+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:43:23.922949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"82'14","reported_seq":163,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-12T03:19:24.008069+0000","last_change":"2024-09-12T03:17:07.830346+0000","last_active":"2024-09-12T03:19:24.008069+0000","last_peered":"2024-09-12T03:19:24.008069+0000","last_clean":"2024-09-12T03:19:24.008069+0000","last_became_active":"2024-09-12T03:17:07.829865+0000","last_became_peered":"2024-09-12T03:17:07.829865+0000","last_unstale":"2024-09-12T03:19:24.008069+0000","last_undegraded":"2024-09-12T03:19:24.008069+0000","last_fullsized":"2024-09-12T03:19:24.008069+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T03:17:06.806294+0000","last_clean_scrub_stamp":"2024-09-12T03:17:06.806294+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:48:12.411951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":126,"num_read_kb":126,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73492,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1245,"num_read_kb":726,"num_write":259,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1774,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":220476,"data_compressed":5427,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":282081,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":74,"seq":317827579946,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32072,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713080,"statfs":{"total":95995035648,"available":95962193920,"internally_reserved":0,"allocated":909312,"data_stored":438533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":67,"seq":287762808877,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1095186,"data_compressed":37683,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":62,"seq":266287972403,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":437976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":437633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462974,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":1031740,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717240,"statfs":{"total":95995035648,"available":95966453760,"internally_reserved":0,"allocated":909312,"data_stored":437884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855432,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":1030576,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":439141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":437284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444120,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":503767,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":504491,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":929792,"data_stored":438915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34237,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35874,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67754,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67030,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":67964,"data_compressed":1809,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T03:19:28.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:28.631 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 -- 172.21.15.19:0/265508483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 msgr2=0x7f3a7410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 --2- 172.21.15.19:0/265508483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a7410ea90 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009ca0 tx=0x7f3a6c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 -- 172.21.15.19:0/265508483 shutdown_connections 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 --2- 172.21.15.19:0/265508483 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a7410ea90 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 --2- 172.21.15.19:0/265508483 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 0x7f3a741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 --2- 172.21.15.19:0/265508483 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 0x7f3a741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 -- 172.21.15.19:0/265508483 >> 172.21.15.19:0/265508483 conn(0x7f3a740fd8c0 msgr2=0x7f3a740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 -- 172.21.15.19:0/265508483 shutdown_connections 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.123+0000 7f3a7891a640 1 -- 172.21.15.19:0/265508483 wait complete. 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 Processor -- start 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 -- start start 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 0x7f3a740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 0x7f3a74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74111fb0 con 0x7f3a74106f10 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74111e30 con 0x7f3a74101de0 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.124+0000 7f3a7891a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3a74112130 con 0x7f3a74102790 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34488/0 (socket says 172.21.15.19:34488) 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 -- 172.21.15.19:0/615632626 learned_addr learned my addr 172.21.15.19:0/615632626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a6bfff640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 0x7f3a74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a7259c640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 0x7f3a740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 -- 172.21.15.19:0/615632626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 msgr2=0x7f3a74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 0x7f3a74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 -- 172.21.15.19:0/615632626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 msgr2=0x7f3a740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 0x7f3a740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 -- 172.21.15.19:0/615632626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a741a6980 con 0x7f3a74106f10 2024-09-12T03:19:29.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a72d9d640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c031a20 tx=0x7f3a6c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c003c30 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a6c003dd0 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a741a6c10 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c02fcf0 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.125+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a741a71a0 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.126+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a741021c0 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.128+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a6c004b40 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.129+0000 7f3a6b7fe640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 0x7f3a600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.129+0000 7f3a7259c640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 0x7f3a600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.129+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3a6c0f04a0 con 0x7f3a74106f10 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.129+0000 7f3a7259c640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 0x7f3a600a50c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009890 tx=0x7f3a64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:29.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.132+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a6c0bd4a0 con 0x7f3a74106f10 2024-09-12T03:19:29.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.342+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a74075a20 con 0x7f3a74106f10 2024-09-12T03:19:29.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:29 smithi019 ceph-mon[30953]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:19:29.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:29 smithi019 ceph-mon[30953]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:29.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.343+0000 7f3a6b7fe640 1 -- 172.21.15.19:0/615632626 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f3a74075a20 con 0x7f3a74106f10 2024-09-12T03:19:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:29.343 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:19:29.344 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.345+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 msgr2=0x7f3a600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.345+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 0x7f3a600a50c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009890 tx=0x7f3a64008040 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.345+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 msgr2=0x7f3a74074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.345+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c031a20 tx=0x7f3a6c031a50 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 shutdown_connections 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3a600a2c00 0x7f3a600a50c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3a74106f10 0x7f3a74074f70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3a74102790 0x7f3a74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 --2- 172.21.15.19:0/615632626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3a74101de0 0x7f3a740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 >> 172.21.15.19:0/615632626 conn(0x7f3a740fd8c0 msgr2=0x7f3a740fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:29.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 shutdown_connections 2024-09-12T03:19:29.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:29.346+0000 7f3a7891a640 1 -- 172.21.15.19:0/615632626 wait complete. 2024-09-12T03:19:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:29 smithi121 ceph-mon[34963]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:19:29.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:29 smithi121 ceph-mon[34963]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:29 smithi135 ceph-mon[35474]: pgmap v350: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:19:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:29 smithi135 ceph-mon[35474]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T03:19:30.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:30.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:30 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/615632626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:30.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:30 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/615632626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:30 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/615632626' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 -- 172.21.15.19:0/1946480134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048100e20 msgr2=0x7f6048101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1946480134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048100e20 0x7f6048101200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6038009900 tx=0x7f603802a480 comp rx=0 tx=0).stop 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 -- 172.21.15.19:0/1946480134 shutdown_connections 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1946480134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048102ef0 0x7f604810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1946480134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102570 0x7f60481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1946480134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048100e20 0x7f6048101200 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 -- 172.21.15.19:0/1946480134 >> 172.21.15.19:0/1946480134 conn(0x7f60480fc7d0 msgr2=0x7f60480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 -- 172.21.15.19:0/1946480134 shutdown_connections 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.824+0000 7f604e6c6640 1 -- 172.21.15.19:0/1946480134 wait complete. 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 Processor -- start 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 -- start start 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 0x7f6048197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102ef0 0x7f604819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6048110f50 con 0x7f6048102570 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6048110dd0 con 0x7f6048100e20 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f604e6c6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60481110d0 con 0x7f6048102ef0 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f603ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f603ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40218/0 (socket says 172.21.15.19:40218) 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.825+0000 7f603ffff640 1 -- 172.21.15.19:0/1139208907 learned_addr learned my addr 172.21.15.19:0/1139208907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f6047fff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 0x7f6048197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f604cc63640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102ef0 0x7f604819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 -- 172.21.15.19:0/1139208907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102ef0 msgr2=0x7f604819bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102ef0 0x7f604819bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 -- 172.21.15.19:0/1139208907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 msgr2=0x7f6048197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 0x7f6048197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 -- 172.21.15.19:0/1139208907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604819c340 con 0x7f6048102570 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f6047fff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 0x7f6048197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f603ffff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6030007b50 tx=0x7f603000e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030007c40 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6030007de0 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604819c630 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603000ae60 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.826+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6048069f70 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.827+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6048101200 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.828+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f603000a440 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.829+0000 7f6045ffb640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 0x7f60340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.829+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f60300d0640 con 0x7f6048102570 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.829+0000 7f6047fff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 0x7f60340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.830+0000 7f6047fff640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 0x7f60340a5040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6038005ab0 tx=0x7f6038005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:30.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:30.833+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6030023060 con 0x7f6048102570 2024-09-12T03:19:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.046+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f604806a1d0 con 0x7f6048102570 2024-09-12T03:19:31.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.046+0000 7f6045ffb640 1 -- 172.21.15.19:0/1139208907 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f603009d5e0 con 0x7f6048102570 2024-09-12T03:19:31.047 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:31.047 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:19:31.047 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:19:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 msgr2=0x7f60340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 0x7f60340a5040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6038005ab0 tx=0x7f6038005a20 comp rx=0 tx=0).stop 2024-09-12T03:19:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 msgr2=0x7f60481978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6030007b50 tx=0x7f603000e420 comp rx=0 tx=0).stop 2024-09-12T03:19:31.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 shutdown_connections 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6048102ef0 0x7f604819bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f60340a2b80 0x7f60340a5040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6048102570 0x7f60481978d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 --2- 172.21.15.19:0/1139208907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6048100e20 0x7f6048197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 >> 172.21.15.19:0/1139208907 conn(0x7f60480fc7d0 msgr2=0x7f60480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 shutdown_connections 2024-09-12T03:19:31.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:31.050+0000 7f604e6c6640 1 -- 172.21.15.19:0/1139208907 wait complete. 2024-09-12T03:19:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:31 smithi121 ceph-mon[34963]: pgmap v351: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T03:19:31.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:31 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1139208907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:31 smithi019 ceph-mon[30953]: pgmap v351: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T03:19:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:31 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1139208907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:31 smithi135 ceph-mon[35474]: pgmap v351: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T03:19:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:31 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1139208907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:31.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:19:32.070 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- 172.21.15.19:0/108752848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718106a80 msgr2=0x7f9718106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 --2- 172.21.15.19:0/108752848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718106a80 0x7f9718106e60 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9708009930 tx=0x7f970802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:32.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- 172.21.15.19:0/108752848 shutdown_connections 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 --2- 172.21.15.19:0/108752848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718101420 0x7f971810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 --2- 172.21.15.19:0/108752848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718100a80 0x7f9718100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 --2- 172.21.15.19:0/108752848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718106a80 0x7f9718106e60 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- 172.21.15.19:0/108752848 >> 172.21.15.19:0/108752848 conn(0x7f97180fc7a0 msgr2=0x7f97180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- 172.21.15.19:0/108752848 shutdown_connections 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- 172.21.15.19:0/108752848 wait complete. 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 Processor -- start 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.557+0000 7f971d137640 1 -- start start 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718101420 0x7f97181989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718106a80 0x7f971819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9718110e80 con 0x7f9718100a80 2024-09-12T03:19:32.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9718110d00 con 0x7f9718106a80 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971d137640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9718111000 con 0x7f9718101420 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40250/0 (socket says 172.21.15.19:40250) 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 -- 172.21.15.19:0/422225906 learned_addr learned my addr 172.21.15.19:0/422225906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f971759e640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718106a80 0x7f971819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f970ffff640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718101420 0x7f97181989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 -- 172.21.15.19:0/422225906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718101420 msgr2=0x7f97181989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718101420 0x7f97181989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 -- 172.21.15.19:0/422225906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718106a80 msgr2=0x7f971819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718106a80 0x7f971819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 -- 172.21.15.19:0/422225906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971819d440 con 0x7f9718100a80 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.558+0000 7f9716d9d640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9708009230 tx=0x7f9708004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:32.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.559+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970803f030 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.559+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9708003ad0 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.559+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971819d6d0 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.559+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9708035070 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.559+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97181a5010 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.560+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97181021c0 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.561+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9708042030 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.562+0000 7f9714d99640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 0x7f97040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.562+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97080ef960 con 0x7f9718100a80 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.562+0000 7f970ffff640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 0x7f97040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.562+0000 7f970ffff640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 0x7f97040a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9700005fd0 tx=0x7f9700005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:32.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.565+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f97080bc8d0 con 0x7f9718100a80 2024-09-12T03:19:32.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.740+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97181a51e0 con 0x7f9718100a80 2024-09-12T03:19:32.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.742+0000 7f9714d99640 1 -- 172.21.15.19:0/422225906 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7f97080bc1d0 con 0x7f9718100a80 2024-09-12T03:19:32.742 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:32.742 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:19:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 msgr2=0x7f97040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 0x7f97040a5040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9700005fd0 tx=0x7f9700005850 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 msgr2=0x7f9718198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9708009230 tx=0x7f9708004270 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 shutdown_connections 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9718106a80 0x7f971819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9718101420 0x7f97181989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97040a2b80 0x7f97040a5040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 --2- 172.21.15.19:0/422225906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9718100a80 0x7f9718198490 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 >> 172.21.15.19:0/422225906 conn(0x7f97180fc7a0 msgr2=0x7f97181048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 shutdown_connections 2024-09-12T03:19:32.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:32.744+0000 7f971d137640 1 -- 172.21.15.19:0/422225906 wait complete. 2024-09-12T03:19:33.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:33 smithi019 ceph-mon[30953]: pgmap v352: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T03:19:33.112 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:33 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/422225906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:33 smithi121 ceph-mon[34963]: pgmap v352: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T03:19:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:33 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/422225906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:33.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T03:19:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:33 smithi135 ceph-mon[35474]: pgmap v352: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T03:19:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:33 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/422225906' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:19:33.733 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:34.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:34 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:19:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:34 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:19:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:34 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:19:34.883 INFO:teuthology.orchestra.run.smithi019.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T03:19:34.883 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T03:19:34.883 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T03:19:34.884 DEBUG:tasks.quiescer:Yielding 2024-09-12T03:19:34.884 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T03:19:34.898 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:19:34.899 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T03:19:34.899 INFO:tasks.workunit:timeout=3h 2024-09-12T03:19:34.899 INFO:tasks.workunit:cleanup=True 2024-09-12T03:19:34.899 DEBUG:teuthology.orchestra.run.smithi019:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:34.900 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:34.918 INFO:teuthology.orchestra.run.smithi019.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 60h/96d Inode: 1099511628283 Links: 2 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-12 03:17:28.784911466 +0000 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-12 03:17:28.784911466 +0000 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-12 03:19:12.543653751 +0000 2024-09-12T03:19:34.919 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-12 03:17:28.784911466 +0000 2024-09-12T03:19:34.919 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T03:19:34.919 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T03:19:34.993 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T03:19:35.049 INFO:tasks.workunit.client.0.smithi019.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T03:19:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:35 smithi121 ceph-mon[34963]: pgmap v353: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-12T03:19:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:35 smithi019 ceph-mon[30953]: pgmap v353: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-12T03:19:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:35 smithi135 ceph-mon[35474]: pgmap v353: 97 pgs: 97 active+clean; 649 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-09-12T03:19:35.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 -- 172.21.15.19:0/3750165140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac102d30 msgr2=0x7f26ac103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3750165140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac102d30 0x7f26ac103110 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009930 tx=0x7f26a802a420 comp rx=0 tx=0).stop 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 -- 172.21.15.19:0/3750165140 shutdown_connections 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3750165140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3750165140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac1036e0 0x7f26ac107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3750165140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac102d30 0x7f26ac103110 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 -- 172.21.15.19:0/3750165140 >> 172.21.15.19:0/3750165140 conn(0x7f26ac0fc7d0 msgr2=0x7f26ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.691+0000 7f26b3a80640 1 -- 172.21.15.19:0/3750165140 shutdown_connections 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.692+0000 7f26b3a80640 1 -- 172.21.15.19:0/3750165140 wait complete. 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.692+0000 7f26b3a80640 1 Processor -- start 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.692+0000 7f26b3a80640 1 -- start start 2024-09-12T03:19:35.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac102d30 0x7f26ac198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac19ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac1120a0 con 0x7f26ac1036e0 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac111f20 con 0x7f26ac102d30 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b3a80640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26ac112220 con 0x7f26ac1082a0 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40268/0 (socket says 172.21.15.19:40268) 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 -- 172.21.15.19:0/3971640686 learned_addr learned my addr 172.21.15.19:0/3971640686 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b181c640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac102d30 0x7f26ac198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b201d640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac19ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 -- 172.21.15.19:0/3971640686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 msgr2=0x7f26ac19ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac19ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 -- 172.21.15.19:0/3971640686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac102d30 msgr2=0x7f26ac198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac102d30 0x7f26ac198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 -- 172.21.15.19:0/3971640686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26ac19d4f0 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b201d640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.693+0000 7f26b101b640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c8c0 tx=0x7f269c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.694+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c007bb0 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.694+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f269c007d50 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.694+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26ac19d7e0 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.694+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c018830 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.694+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26ac1a50c0 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.695+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26ac06a850 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.696+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f269c0189f0 con 0x7f26ac1036e0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.697+0000 7f26a2ffd640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 0x7f26980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.697+0000 7f26b181c640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 0x7f26980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:35.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.697+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f269c0cf350 con 0x7f26ac1036e0 2024-09-12T03:19:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.697+0000 7f26b181c640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 0x7f26980a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:35.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.701+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f269c014030 con 0x7f26ac1036e0 2024-09-12T03:19:35.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.914+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26ac103110 con 0x7f26ac1036e0 2024-09-12T03:19:35.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.915+0000 7f26a2ffd640 1 -- 172.21.15.19:0/3971640686 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f26ac103110 con 0x7f26ac1036e0 2024-09-12T03:19:35.915 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:19:35.915 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:19:35.916 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:19:35.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 msgr2=0x7f26980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 0x7f26980a50c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009900 tx=0x7f26a80058d0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 msgr2=0x7f26ac198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c8c0 tx=0x7f269c00cd80 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 shutdown_connections 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26ac1082a0 0x7f26ac19ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26980a2c00 0x7f26980a50c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26ac1036e0 0x7f26ac198a80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 --2- 172.21.15.19:0/3971640686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26ac102d30 0x7f26ac198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 >> 172.21.15.19:0/3971640686 conn(0x7f26ac0fc7d0 msgr2=0x7f26ac105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 shutdown_connections 2024-09-12T03:19:35.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:35.917+0000 7f26b3a80640 1 -- 172.21.15.19:0/3971640686 wait complete. 2024-09-12T03:19:36.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:36 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3971640686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:36.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:36 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3971640686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:36 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3971640686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:19:36.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T03:17:10.527386+0000', 'modified': '2024-09-12T03:19:13.230767+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24467, 'mds_2': 24463, 'mds_3': 24421, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14577': {'gid': 14577, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6839/2689902455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6838', 'nonce': 2689902455}, {'type': 'v1', 'addr': '172.21.15.19:6839', 'nonce': 2689902455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14589': {'gid': 14589, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.19:6841/9500015', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6840', 'nonce': 9500015}, {'type': 'v1', 'addr': '172.21.15.19:6841', 'nonce': 9500015}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.121:6833/3300883065', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6832', 'nonce': 3300883065}, {'type': 'v1', 'addr': '172.21.15.121:6833', 'nonce': 3300883065}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'l', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 15, 'addr': '172.21.15.135:6833/990973415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6832', 'nonce': 990973415}, {'type': 'v1', 'addr': '172.21.15.135:6833', 'nonce': 990973415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6835/3114415604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6834', 'nonce': 3114415604}, {'type': 'v1', 'addr': '172.21.15.121:6835', 'nonce': 3114415604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.121:6837/826498871', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 826498871}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 826498871}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24457': {'gid': 24457, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.135:6835/3494935988', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6834', 'nonce': 3494935988}, {'type': 'v1', 'addr': '172.21.15.135:6835', 'nonce': 3494935988}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24463': {'gid': 24463, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.135:6837/1827195054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6836', 'nonce': 1827195054}, {'type': 'v1', 'addr': '172.21.15.135:6837', 'nonce': 1827195054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24467': {'gid': 24467, 'name': 'b', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.121:6839/1305484715', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6838', 'nonce': 1305484715}, {'type': 'v1', 'addr': '172.21.15.121:6839', 'nonce': 1305484715}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.135:6839/1449185077', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.135:6838', 'nonce': 1449185077}, {'type': 'v1', 'addr': '172.21.15.135:6839', 'nonce': 1449185077}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24487, 'qdb_cluster': [24407, 24421, 24463, 24467, 24487]} 2024-09-12T03:19:36.652 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T03:19:36.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.k status 2024-09-12T03:19:36.954 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:37 smithi121 ceph-mon[34963]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:19:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:37 smithi019 ceph-mon[30953]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:19:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:37 smithi135 ceph-mon[35474]: pgmap v354: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 -- 172.21.15.19:0/1624071559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 msgr2=0x7f08dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 --2- 172.21.15.19:0/1624071559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc10ea90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009ca0 tx=0x7f08d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 -- 172.21.15.19:0/1624071559 shutdown_connections 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 --2- 172.21.15.19:0/1624071559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc10ea90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 --2- 172.21.15.19:0/1624071559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc102790 0x7f08dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 --2- 172.21.15.19:0/1624071559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc101de0 0x7f08dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 -- 172.21.15.19:0/1624071559 >> 172.21.15.19:0/1624071559 conn(0x7f08dc0fd8c0 msgr2=0x7f08dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 -- 172.21.15.19:0/1624071559 shutdown_connections 2024-09-12T03:19:37.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.500+0000 7f08e0d56640 1 -- 172.21.15.19:0/1624071559 wait complete. 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.501+0000 7f08e0d56640 1 Processor -- start 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.501+0000 7f08e0d56640 1 -- start start 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.501+0000 7f08e0d56640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 0x7f08dc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc102790 0x7f08dc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc112010 con 0x7f08dc106f10 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc111e90 con 0x7f08dc102790 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc112190 con 0x7f08dc101de0 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40290/0 (socket says 172.21.15.19:40290) 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 -- 172.21.15.19:0/2731395379 learned_addr learned my addr 172.21.15.19:0/2731395379 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:37.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc102790 0x7f08dc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08da59c640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 0x7f08dc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 -- 172.21.15.19:0/2731395379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 msgr2=0x7f08dc198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 0x7f08dc198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 -- 172.21.15.19:0/2731395379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc102790 msgr2=0x7f08dc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc102790 0x7f08dc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 -- 172.21.15.19:0/2731395379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc19d540 con 0x7f08dc106f10 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08da59c640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 0x7f08dc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08dad9d640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08d4031a60 tx=0x7f08d4031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08cb7fe640 1 -- 172.21.15.19:0/2731395379 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d4003c30 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08cb7fe640 1 -- 172.21.15.19:0/2731395379 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d4003dd0 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc19d7d0 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08cb7fe640 1 -- 172.21.15.19:0/2731395379 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d40047e0 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.502+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc1a5110 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.505+0000 7f08cb7fe640 1 -- 172.21.15.19:0/2731395379 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08d402f790 con 0x7f08dc106f10 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.505+0000 7f08e312f740 1 Processor -- start 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.505+0000 7f08e312f740 1 -- start start 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.505+0000 7f08e312f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0962db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08e312f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf096d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08e312f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08e312f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcf08c9090 con 0x55bcf096dbb0 2024-09-12T03:19:37.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08cb7fe640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 0x7f08c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08e312f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0870a70 con 0x55bcf09629d0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08e312f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcf08c5050 con 0x55bcf09632f0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08dad9d640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 0x7f08c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40302/0 (socket says 172.21.15.19:40302) 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 -- 172.21.15.19:0/2537954247 learned_addr learned my addr 172.21.15.19:0/2537954247 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08da59c640 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf096d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08cb7fe640 1 -- 172.21.15.19:0/2731395379 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f08d40f0510 con 0x7f08dc106f10 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0962db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 -- 172.21.15.19:0/2537954247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 msgr2=0x55bcf096d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf096d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 -- 172.21.15.19:0/2537954247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 msgr2=0x55bcf0962db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0962db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 -- 172.21.15.19:0/2537954247 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf087d130 con 0x55bcf096dbb0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae00 tx=0x7f08d000a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865986a3a5c8eb1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08dad9d640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 0x7f08c40a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08d40bca80 tx=0x7f08d4087df0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.506+0000 7f08c8ff9640 1 -- 172.21.15.19:0/2537954247 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d001b070 con 0x55bcf096dbb0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08c8ff9640 1 -- 172.21.15.19:0/2537954247 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d000fd70 con 0x55bcf096dbb0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08c8ff9640 1 -- 172.21.15.19:0/2537954247 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d00146e0 con 0x55bcf096dbb0 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 -- 172.21.15.19:0/2537954247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 msgr2=0x55bcf096ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae00 tx=0x7f08d000a490 comp rx=0 tx=0).stop 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 -- 172.21.15.19:0/2537954247 shutdown_connections 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf096ffa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf096d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 --2- 172.21.15.19:0/2537954247 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0962db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 -- 172.21.15.19:0/2537954247 >> 172.21.15.19:0/2537954247 conn(0x55bcf095ef60 msgr2=0x55bcf095f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 -- 172.21.15.19:0/2537954247 shutdown_connections 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.507+0000 7f08e312f740 1 -- 172.21.15.19:0/2537954247 wait complete. 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 -- start start 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 Processor -- start 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09632f0 0x55bcf096a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf096dbb0 0x55bcf096ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0970780 con 0x55bcf09629d0 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0883f20 con 0x55bcf09632f0 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcf0970900 con 0x55bcf096dbb0 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08e312f740 10 client.0 _ll_register_callbacks cb 0x55bcf084d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40310/0 (socket says 172.21.15.19:40310) 2024-09-12T03:19:37.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 -- 172.21.15.19:0/545185920 learned_addr learned my addr 172.21.15.19:0/545185920 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08da59c640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09632f0 0x55bcf096a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf096dbb0 0x55bcf096ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 -- 172.21.15.19:0/545185920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf096dbb0 msgr2=0x55bcf096ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.508+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf096dbb0 0x55bcf096ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08d9d9b640 1 -- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09632f0 msgr2=0x55bcf096a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09632f0 0x55bcf096a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08d9d9b640 1 -- 172.21.15.19:0/545185920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf063a2b0 con 0x55bcf09629d0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f08d0004810 tx=0x7f08d0016ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf02fbbbbddce50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0029060 con 0x55bcf09629d0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f08d0016b30 con 0x55bcf09629d0 2024-09-12T03:19:37.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d000fd70 con 0x55bcf09629d0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08d001b040 con 0x55bcf09629d0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 10 client.15366 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcf097c130 con 0x55bcf09629d0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d000fd70 con 0x55bcf09629d0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08d0021070 con 0x55bcf09629d0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 10 client.15366 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 20 client.15366 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.509+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf097e300 0x55bcf097f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.510+0000 7f08e312f740 4 client.15366 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.510+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcf08d6600 con 0x55bcf097e300 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.510+0000 7f08da59c640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf097e300 0x55bcf097f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.510+0000 7f08da59c640 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf097e300 0x55bcf097f770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d49384ecefad1e0c server_cookie=e6b398d11720de31 in_seq=0 out_seq=0 2024-09-12T03:19:37.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.510+0000 7f08c1ffb640 10 client.15366 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08c1ffb640 1 -- 172.21.15.19:0/545185920 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcf08d6600 con 0x55bcf097e300 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08c1ffb640 10 client.15366 handle_command_reply: tid=0 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 client.15366 shutdown 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 msgr2=0x55bcf096d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f08d0004810 tx=0x7f08d0016ee0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 shutdown_connections 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf097e300 0x55bcf097f770 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf096dbb0 0x55bcf096ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09632f0 0x55bcf096a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08c1ffb640 0 client.15366 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 --2- 172.21.15.19:0/545185920 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf09629d0 0x55bcf096d4f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 >> 172.21.15.19:0/545185920 conn(0x55bcf095ef60 msgr2=0x55bcf0964c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 shutdown_connections 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 1 -- 172.21.15.19:0/545185920 wait complete. 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.511+0000 7f08e312f740 20 client.15366 trim_cache size 0 max 16384 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 Processor -- start 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 -- start start 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf096c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097dab0 con 0x55bcf096dbb0 2024-09-12T03:19:37.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097e180 con 0x55bcf09629d0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.529+0000 7f08e312f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097e850 con 0x55bcf09632f0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40324/0 (socket says 172.21.15.19:40324) 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 -- 172.21.15.19:0/596589163 learned_addr learned my addr 172.21.15.19:0/596589163 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08da59c640 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf096c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 -- 172.21.15.19:0/596589163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 msgr2=0x55bcf0967ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 -- 172.21.15.19:0/596589163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 msgr2=0x55bcf096c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf096c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 -- 172.21.15.19:0/596589163 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcf09701d0 con 0x55bcf096dbb0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f08d0014640 tx=0x7f08d0018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f568e7f9a2c707 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08c3fff640 1 -- 172.21.15.19:0/596589163 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0034020 con 0x55bcf096dbb0 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08c3fff640 1 -- 172.21.15.19:0/596589163 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0023380 con 0x55bcf096dbb0 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.530+0000 7f08c3fff640 1 -- 172.21.15.19:0/596589163 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0029070 con 0x55bcf096dbb0 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- 172.21.15.19:0/596589163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 msgr2=0x55bcf0968020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f08d0014640 tx=0x7f08d0018040 comp rx=0 tx=0).stop 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- 172.21.15.19:0/596589163 shutdown_connections 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0968020 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 --2- 172.21.15.19:0/596589163 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf096c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- 172.21.15.19:0/596589163 >> 172.21.15.19:0/596589163 conn(0x55bcf095ef60 msgr2=0x55bcf096bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- 172.21.15.19:0/596589163 shutdown_connections 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- 172.21.15.19:0/596589163 wait complete. 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 -- start start 2024-09-12T03:19:37.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.531+0000 7f08e312f740 1 Processor -- start 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0967230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097f5f0 con 0x55bcf096dbb0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097ef20 con 0x55bcf09629d0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bcf097fcc0 con 0x55bcf09632f0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08e312f740 10 client.0 _ll_register_callbacks cb 0x7f08d0019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40334/0 (socket says 172.21.15.19:40334) 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 -- 172.21.15.19:0/318269291 learned_addr learned my addr 172.21.15.19:0/318269291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0967230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08da59c640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 -- 172.21.15.19:0/318269291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 msgr2=0x55bcf0967770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 -- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 msgr2=0x55bcf0967230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0967230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08d9d9b640 1 -- 172.21.15.19:0/318269291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcf09870a0 con 0x55bcf096dbb0 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.532+0000 7f08da59c640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08d9d9b640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae00 tx=0x7f08d000aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c88b366136e14a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:37.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0029070 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f08d000fa40 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08d0023380 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f08d0034070 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 10 client.15372 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcf0988380 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08d0023380 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f08d0021040 con 0x55bcf096dbb0 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 10 client.15372 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 10 client.15372 resolve_mds: resolved name 'k' to daemon mds.k 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 20 client.15372 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf09ecdc0 0x55bcf09ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 4 client.15372 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:37.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.533+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "status"}) -- 0x55bcf08d0b10 con 0x55bcf09ecdc0 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08da59c640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf09ecdc0 0x55bcf09ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08da59c640 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf09ecdc0 0x55bcf09ebad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=76ad9863a32ab93d server_cookie=d893eabacbe2f9b3 in_seq=0 out_seq=0 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08c1ffb640 10 client.15372 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08c1ffb640 1 -- 172.21.15.19:0/318269291 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55bcf08d0b10 con 0x55bcf09ecdc0 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08c1ffb640 10 client.15372 handle_command_reply: tid=0 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 client.15372 shutdown 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 msgr2=0x55bcf0986a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f08d000ae00 tx=0x7f08d000aba0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 shutdown_connections 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bcf096dbb0 0x55bcf0986a30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55bcf09ecdc0 0x55bcf09ebad0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bcf09632f0 0x55bcf0967770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 --2- 172.21.15.19:0/318269291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bcf09629d0 0x55bcf0967230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08c1ffb640 0 client.15372 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.534+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 >> 172.21.15.19:0/318269291 conn(0x55bcf095ef60 msgr2=0x55bcf095e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 shutdown_connections 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e312f740 1 -- 172.21.15.19:0/318269291 wait complete. 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 4, 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24407, 2024-09-12T03:19:37.566 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 136.71305540899999, 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 189.400052902 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e312f740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 msgr2=0x7f08c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 0x7f08c40a5040 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f08d40bca80 tx=0x7f08d4087df0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 msgr2=0x7f08dc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f08d4031a60 tx=0x7f08d4031a90 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 shutdown_connections 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f08c40a2b80 0x7f08c40a5040 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f08dc106f10 0x7f08dc19ce60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f08dc102790 0x7f08dc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 --2- 172.21.15.19:0/2731395379 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f08dc101de0 0x7f08dc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 >> 172.21.15.19:0/2731395379 conn(0x7f08dc0fd8c0 msgr2=0x7f08dc0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 shutdown_connections 2024-09-12T03:19:37.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:37.535+0000 7f08e0d56640 1 -- 172.21.15.19:0/2731395379 wait complete. 2024-09-12T03:19:38.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.l status 2024-09-12T03:19:38.562 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 -- 172.21.15.19:0/3109465440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 msgr2=0x7f56141067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 --2- 172.21.15.19:0/3109465440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f56141067e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5604009900 tx=0x7f560402a410 comp rx=0 tx=0).stop 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 -- 172.21.15.19:0/3109465440 shutdown_connections 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 --2- 172.21.15.19:0/3109465440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 0x7f561410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 --2- 172.21.15.19:0/3109465440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f56141067e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 --2- 172.21.15.19:0/3109465440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 -- 172.21.15.19:0/3109465440 >> 172.21.15.19:0/3109465440 conn(0x7f56140fd8c0 msgr2=0x7f56140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 -- 172.21.15.19:0/3109465440 shutdown_connections 2024-09-12T03:19:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.009+0000 7f561bdcb640 1 -- 172.21.15.19:0/3109465440 wait complete. 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.010+0000 7f561bdcb640 1 Processor -- start 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.010+0000 7f561bdcb640 1 -- start start 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 0x7f5614074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111f70 con 0x7f5614102790 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5614111df0 con 0x7f5614106f10 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561bdcb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f56141120f0 con 0x7f5614101de0 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40362/0 (socket says 172.21.15.19:40362) 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 -- 172.21.15.19:0/1289946580 learned_addr learned my addr 172.21.15.19:0/1289946580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f561a368640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 0x7f5614074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619b67640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 -- 172.21.15.19:0/1289946580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 msgr2=0x7f56140744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56140744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 -- 172.21.15.19:0/1289946580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 msgr2=0x7f5614074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 0x7f5614074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 -- 172.21.15.19:0/1289946580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141a6970 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619b67640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.011+0000 7f5619366640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5604006010 tx=0x7f56040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.012+0000 7f560affd640 1 -- 172.21.15.19:0/1289946580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f560403f030 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.012+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56141a6c00 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.012+0000 7f560affd640 1 -- 172.21.15.19:0/1289946580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5604003ce0 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.012+0000 7f560affd640 1 -- 172.21.15.19:0/1289946580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604035070 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.012+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56141a7130 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.014+0000 7f560affd640 1 -- 172.21.15.19:0/1289946580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5604042030 con 0x7f5614102790 2024-09-12T03:19:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f560affd640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f560affd640 1 -- 172.21.15.19:0/1289946580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f56040ef230 con 0x7f5614102790 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f5619b67640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f561e1a4740 1 Processor -- start 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f5619b67640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 0x7f56000a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f56100058c0 tx=0x7f5610005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.015+0000 7f561e1a4740 1 -- start start 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead31e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eead347b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eeac85090 con 0x562eead271e0 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562eeac2ca70 con 0x562eead27b00 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561e1a4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562eeac81050 con 0x562eead323c0 2024-09-12T03:19:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40368/0 (socket says 172.21.15.19:40368) 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 -- 172.21.15.19:0/209950 learned_addr learned my addr 172.21.15.19:0/209950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619366640 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead31e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f561a368640 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eead347b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 -- 172.21.15.19:0/209950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 msgr2=0x562eead347b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eead347b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 -- 172.21.15.19:0/209950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 msgr2=0x562eead31e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead31e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 -- 172.21.15.19:0/209950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eeac39130 con 0x562eead271e0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.016+0000 7f5619b67640 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5610005ca0 tx=0x7f5610012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2648a225bbf8b9e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f55fffff640 1 -- 172.21.15.19:0/209950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610022070 con 0x562eead271e0 2024-09-12T03:19:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f55fffff640 1 -- 172.21.15.19:0/209950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561001eb30 con 0x562eead271e0 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f55fffff640 1 -- 172.21.15.19:0/209950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610010bb0 con 0x562eead271e0 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 -- 172.21.15.19:0/209950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 msgr2=0x562eead275c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5610005ca0 tx=0x7f5610012000 comp rx=0 tx=0).stop 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 -- 172.21.15.19:0/209950 shutdown_connections 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eead347b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead31e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 --2- 172.21.15.19:0/209950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead275c0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.017+0000 7f561e1a4740 1 -- 172.21.15.19:0/209950 >> 172.21.15.19:0/209950 conn(0x562eead23770 msgr2=0x562eead23b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- 172.21.15.19:0/209950 shutdown_connections 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- 172.21.15.19:0/209950 wait complete. 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- start start 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 Processor -- start 2024-09-12T03:19:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead27b00 0x562eead2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eead34f90 con 0x562eead271e0 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562eeac3ff20 con 0x562eead323c0 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562eead35110 con 0x562eead27b00 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.018+0000 7f561e1a4740 10 client.0 _ll_register_callbacks cb 0x562eeac09030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40382/0 (socket says 172.21.15.19:40382) 2024-09-12T03:19:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 -- 172.21.15.19:0/3252419088 learned_addr learned my addr 172.21.15.19:0/3252419088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f561a368640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619366640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead27b00 0x562eead2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 -- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead27b00 msgr2=0x562eead2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead27b00 0x562eead2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 -- 172.21.15.19:0/3252419088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 msgr2=0x562eead2f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 -- 172.21.15.19:0/3252419088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eea9f62b0 con 0x562eead271e0 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f5619b67640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5610010730 tx=0x7f5610010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df320a8d1fe0ead6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f55fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610035060 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.019+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f561001e450 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561001eb30 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5610022070 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 10 client.15384 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562eead40880 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561001eb30 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f561001ece0 con 0x562eead271e0 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 10 client.15384 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 10 client.15384 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:19:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 20 client.15384 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eead42ae0 0x562eead41d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 4 client.15384 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.020+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eeac92600 con 0x562eead42ae0 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.021+0000 7f5619366640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eead42ae0 0x562eead41d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.021+0000 7f5619366640 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eead42ae0 0x562eead41d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=da57c5ad078a1b76 server_cookie=bad7db8f77c2b2d3 in_seq=0 out_seq=0 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.021+0000 7f55fd7fa640 10 client.15384 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3252419088 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eeac92600 con 0x562eead42ae0 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f55fd7fa640 10 client.15384 handle_command_reply: tid=0 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 client.15384 shutdown 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 msgr2=0x562eead31c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5610010730 tx=0x7f5610010b40 comp rx=0 tx=0).stop 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 shutdown_connections 2024-09-12T03:19:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eead42ae0 0x562eead41d40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead27b00 0x562eead2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f55fd7fa640 0 client.15384 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3252419088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead31c30 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 >> 172.21.15.19:0/3252419088 conn(0x562eead23770 msgr2=0x562eead1a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 shutdown_connections 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 1 -- 172.21.15.19:0/3252419088 wait complete. 2024-09-12T03:19:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.022+0000 7f561e1a4740 20 client.15384 trim_cache size 0 max 16384 2024-09-12T03:19:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 Processor -- start 2024-09-12T03:19:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 -- start start 2024-09-12T03:19:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead271e0 0x562eead431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eead42290 con 0x562eead27b00 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562eead42960 con 0x562eead323c0 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561e1a4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562eead43030 con 0x562eead271e0 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f5619366640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f5619366640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40384/0 (socket says 172.21.15.19:40384) 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f5619366640 1 -- 172.21.15.19:0/3510491874 learned_addr learned my addr 172.21.15.19:0/3510491874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f561a368640 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.041+0000 7f5619b67640 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead271e0 0x562eead431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 -- 172.21.15.19:0/3510491874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead271e0 msgr2=0x562eead431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead271e0 0x562eead431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 -- 172.21.15.19:0/3510491874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 msgr2=0x562eead2c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 -- 172.21.15.19:0/3510491874 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eead2d4d0 con 0x562eead27b00 2024-09-12T03:19:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f5619366640 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f56040b8530 tx=0x7f5604046040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce6eb98855296a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f55ff7fe640 1 -- 172.21.15.19:0/3510491874 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604076070 con 0x562eead27b00 2024-09-12T03:19:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f55ff7fe640 1 -- 172.21.15.19:0/3510491874 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56040bc260 con 0x562eead27b00 2024-09-12T03:19:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.042+0000 7f55ff7fe640 1 -- 172.21.15.19:0/3510491874 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5604049040 con 0x562eead27b00 2024-09-12T03:19:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- 172.21.15.19:0/3510491874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 msgr2=0x562eead19ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f56040b8530 tx=0x7f5604046040 comp rx=0 tx=0).stop 2024-09-12T03:19:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- 172.21.15.19:0/3510491874 shutdown_connections 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead323c0 0x562eead2c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead27b00 0x562eead19ef0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3510491874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead271e0 0x562eead431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- 172.21.15.19:0/3510491874 >> 172.21.15.19:0/3510491874 conn(0x562eead23770 msgr2=0x562eead197e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- 172.21.15.19:0/3510491874 shutdown_connections 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- 172.21.15.19:0/3510491874 wait complete. 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 -- start start 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.043+0000 7f561e1a4740 1 Processor -- start 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eeadab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562eead43dd0 con 0x562eead271e0 2024-09-12T03:19:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562eead43700 con 0x562eead27b00 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562eead444a0 con 0x562eead323c0 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561e1a4740 10 client.0 _ll_register_callbacks cb 0x7f5604005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40394/0 (socket says 172.21.15.19:40394) 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 -- 172.21.15.19:0/3112889486 learned_addr learned my addr 172.21.15.19:0/3112889486 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619366640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f561a368640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eeadab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 -- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 msgr2=0x562eeadab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eeadab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 -- 172.21.15.19:0/3112889486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 msgr2=0x562eead47da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead47da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.044+0000 7f5619b67640 1 -- 172.21.15.19:0/3112889486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eeadab820 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f5619b67640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5610010010 tx=0x7f56100108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3266e9993b49c9df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5610040020 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f56100049f0 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56040bc260 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5610015040 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f561e1a4740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562eeadacae0 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56040bc260 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.045+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f56100050d0 con 0x562eead271e0 2024-09-12T03:19:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 10 client.15390 resolve_mds: resolved name 'l' to daemon mds.l 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 20 client.15390 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eeadb1570 0x562eeadb0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 4 client.15390 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "status"}) -- 0x562eeac8cb10 con 0x562eeadb1570 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f5619366640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eeadb1570 0x562eeadb0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f5619366640 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eeadb1570 0x562eeadb0280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a91b89f6f992010d server_cookie=fc1ebb9a1ae61ee9 in_seq=0 out_seq=0 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.046+0000 7f55fd7fa640 10 client.15390 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f55fd7fa640 1 -- 172.21.15.19:0/3112889486 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562eeac8cb10 con 0x562eeadb1570 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f55fd7fa640 10 client.15390 handle_command_reply: tid=0 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 client.15390 shutdown 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 msgr2=0x562eead47860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5610010010 tx=0x7f56100108a0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 shutdown_connections 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562eeadb1570 0x562eeadb0280 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562eead323c0 0x562eeadab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562eead27b00 0x562eead47da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f55fd7fa640 0 client.15390 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 --2- 172.21.15.19:0/3112889486 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562eead271e0 0x562eead47860 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 >> 172.21.15.19:0/3112889486 conn(0x562eead23770 msgr2=0x562eead1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 shutdown_connections 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 1 -- 172.21.15.19:0/3112889486 wait complete. 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.047+0000 7f561e1a4740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T03:19:39.052 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 3, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24421, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 139.23205849499999, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 193.58954379299999 2024-09-12T03:19:39.053 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 msgr2=0x7f56000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 0x7f56000a5040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f56100058c0 tx=0x7f5610005850 comp rx=0 tx=0).stop 2024-09-12T03:19:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 msgr2=0x7f5614074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5604006010 tx=0x7f56040042f0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 shutdown_connections 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5614106f10 0x7f5614074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56000a2b80 0x7f56000a5040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5614102790 0x7f5614074a30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 --2- 172.21.15.19:0/1289946580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5614101de0 0x7f56140744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 >> 172.21.15.19:0/1289946580 conn(0x7f56140fd8c0 msgr2=0x7f56140fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 shutdown_connections 2024-09-12T03:19:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:39.048+0000 7f561bdcb640 1 -- 172.21.15.19:0/1289946580 wait complete. 2024-09-12T03:19:39.291 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:39 smithi019 ceph-mon[30953]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:19:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:39 smithi121 ceph-mon[34963]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:19:39.687 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:39 smithi135 ceph-mon[35474]: pgmap v355: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T03:19:39.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T03:19:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.563+0000 7f10b06c9640 1 -- 172.21.15.19:0/1658442214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8101420 msgr2=0x7f10a810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.563+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1658442214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8101420 0x7f10a810d950 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009cd0 tx=0x7f10a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- 172.21.15.19:0/1658442214 shutdown_connections 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1658442214 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8101420 0x7f10a810d950 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1658442214 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8100a80 0x7f10a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1658442214 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8106a80 0x7f10a8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- 172.21.15.19:0/1658442214 >> 172.21.15.19:0/1658442214 conn(0x7f10a80fc7a0 msgr2=0x7f10a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- 172.21.15.19:0/1658442214 shutdown_connections 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- 172.21.15.19:0/1658442214 wait complete. 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 Processor -- start 2024-09-12T03:19:40.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- start start 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8100a80 0x7f10a8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8101420 0x7f10a81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8110e80 con 0x7f10a8106a80 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8110d00 con 0x7f10a8100a80 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.564+0000 7f10b06c9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10a8111000 con 0x7f10a8101420 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41754/0 (socket says 172.21.15.19:41754) 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 -- 172.21.15.19:0/1922558414 learned_addr learned my addr 172.21.15.19:0/1922558414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10adc64640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8101420 0x7f10a81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10ae465640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8100a80 0x7f10a8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 -- 172.21.15.19:0/1922558414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8101420 msgr2=0x7f10a81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8101420 0x7f10a81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 -- 172.21.15.19:0/1922558414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8100a80 msgr2=0x7f10a8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8100a80 0x7f10a8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 -- 172.21.15.19:0/1922558414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a81a4850 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10aec66640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005cc0 tx=0x7f10a0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f109f7fe640 1 -- 172.21.15.19:0/1922558414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004040 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f109f7fe640 1 -- 172.21.15.19:0/1922558414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a00046c0 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a81a4a20 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f109f7fe640 1 -- 172.21.15.19:0/1922558414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0046480 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.565+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a81a4f00 con 0x7f10a8106a80 2024-09-12T03:19:40.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 Processor -- start 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f109f7fe640 1 -- 172.21.15.19:0/1922558414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10a00305a0 con 0x7f10a8106a80 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 -- start start 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6bc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826f618090 con 0x55826f6bcbb0 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55826f5bfa70 con 0x55826f6b19d0 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55826f614050 con 0x55826f6b22f0 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.568+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41758/0 (socket says 172.21.15.19:41758) 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 -- 172.21.15.19:0/2007125056 learned_addr learned my addr 172.21.15.19:0/2007125056 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:40.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10ae465640 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f109f7fe640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 0x7f10940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 -- 172.21.15.19:0/2007125056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 msgr2=0x55826f6bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6bc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 -- 172.21.15.19:0/2007125056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 msgr2=0x55826f6b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 -- 172.21.15.19:0/2007125056 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826f5cc130 con 0x55826f6bcbb0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10aec66640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 0x7f10940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f109f7fe640 1 -- 172.21.15.19:0/1922558414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f10a002e030 con 0x7f10a8106a80 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f10adc64640 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f10980125a0 tx=0x7f109800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91d0cab7fe4cadb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f109cff9640 1 -- 172.21.15.19:0/2007125056 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098004250 con 0x55826f6bcbb0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f109cff9640 1 -- 172.21.15.19:0/2007125056 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10980043f0 con 0x55826f6bcbb0 2024-09-12T03:19:40.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.569+0000 7f109cff9640 1 -- 172.21.15.19:0/2007125056 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098004e10 con 0x55826f6bcbb0 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2007125056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 msgr2=0x55826f6befa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10aec66640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 0x7f10940a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f10a00bcde0 tx=0x7f10a008ee60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f10980125a0 tx=0x7f109800b870 comp rx=0 tx=0).stop 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2007125056 shutdown_connections 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6befa0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6bc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2007125056 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2007125056 >> 172.21.15.19:0/2007125056 conn(0x55826f6adf60 msgr2=0x55826f6ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2007125056 shutdown_connections 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2007125056 wait complete. 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.570+0000 7f10b2aa2740 1 -- start start 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 Processor -- start 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6b9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826f6bf780 con 0x55826f6b19d0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55826f5d2f20 con 0x55826f6bcbb0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55826f6bf900 con 0x55826f6b22f0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10b2aa2740 10 client.0 _ll_register_callbacks cb 0x55826f59c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41768/0 (socket says 172.21.15.19:41768) 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 -- 172.21.15.19:0/2867678495 learned_addr learned my addr 172.21.15.19:0/2867678495 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6b9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10ae465640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 -- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 msgr2=0x55826f6b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 -- 172.21.15.19:0/2867678495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 msgr2=0x55826f6b9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6b9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10adc64640 1 -- 172.21.15.19:0/2867678495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826f3892b0 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.571+0000 7f10ae465640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10adc64640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1098012ef0 tx=0x7f1098004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ac56c26c390d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098024070 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1098005270 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10980043f0 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1098027040 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 10 client.15402 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55826f6cb0d0 con 0x55826f6b19d0 2024-09-12T03:19:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10980043f0 con 0x55826f6b19d0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f10980314e0 con 0x55826f6b19d0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 10 client.15402 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 10 client.15402 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 20 client.15402 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f6cd4a0 0x55826f6ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 4 client.15402 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.572+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55826f625600 con 0x55826f6cd4a0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.573+0000 7f10ae465640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f6cd4a0 0x55826f6ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.573+0000 7f1091ffb640 10 client.15402 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.573+0000 7f10ae465640 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f6cd4a0 0x55826f6ce910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ba4bba649a5d63b server_cookie=b31a48e4b8e9061a in_seq=0 out_seq=0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f1091ffb640 1 -- 172.21.15.19:0/2867678495 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55826f625600 con 0x55826f6cd4a0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f1091ffb640 10 client.15402 handle_command_reply: tid=0 2024-09-12T03:19:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 client.15402 shutdown 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 msgr2=0x55826f6bc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1098012ef0 tx=0x7f1098004ad0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 shutdown_connections 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f6cd4a0 0x55826f6ce910 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6b9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f1091ffb640 0 client.15402 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/2867678495 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b19d0 0x55826f6bc410 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 >> 172.21.15.19:0/2867678495 conn(0x55826f6adf60 msgr2=0x55826f6b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 shutdown_connections 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 1 -- 172.21.15.19:0/2867678495 wait complete. 2024-09-12T03:19:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.574+0000 7f10b2aa2740 20 client.15402 trim_cache size 0 max 16384 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.592+0000 7f10b2aa2740 1 Processor -- start 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.592+0000 7f10b2aa2740 1 -- start start 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b19d0 0x55826f6ce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826f6ccc50 con 0x55826f6b22f0 2024-09-12T03:19:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55826f6cd320 con 0x55826f6bcbb0 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55826f6cd9f0 con 0x55826f6b19d0 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41774/0 (socket says 172.21.15.19:41774) 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 -- 172.21.15.19:0/1262223182 learned_addr learned my addr 172.21.15.19:0/1262223182 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10adc64640 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 -- 172.21.15.19:0/1262223182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b19d0 msgr2=0x55826f6ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b19d0 0x55826f6ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 -- 172.21.15.19:0/1262223182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 msgr2=0x55826f6ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 -- 172.21.15.19:0/1262223182 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55826f6bc780 con 0x55826f6b22f0 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f10ae465640 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005fd0 tx=0x7f10a400f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8840a07074fd931 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f1093fff640 1 -- 172.21.15.19:0/1262223182 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a40142d0 con 0x55826f6b22f0 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f1093fff640 1 -- 172.21.15.19:0/1262223182 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a401b810 con 0x55826f6b22f0 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.593+0000 7f1093fff640 1 -- 172.21.15.19:0/1262223182 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a4012b70 con 0x55826f6b22f0 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- 172.21.15.19:0/1262223182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 msgr2=0x55826f6ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f10a4005fd0 tx=0x7f10a400f3c0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- 172.21.15.19:0/1262223182 shutdown_connections 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6bcbb0 0x55826f6ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6b22f0 0x55826f6ce240 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/1262223182 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b19d0 0x55826f6ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- 172.21.15.19:0/1262223182 >> 172.21.15.19:0/1262223182 conn(0x55826f6adf60 msgr2=0x55826f6ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- 172.21.15.19:0/1262223182 shutdown_connections 2024-09-12T03:19:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- 172.21.15.19:0/1262223182 wait complete. 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.594+0000 7f10b2aa2740 1 -- start start 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 Processor -- start 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55826f6ce790 con 0x55826f6bcbb0 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55826f6ce0c0 con 0x55826f6b19d0 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55826f6cee60 con 0x55826f6b22f0 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10b2aa2740 10 client.0 _ll_register_callbacks cb 0x7f10980123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41778/0 (socket says 172.21.15.19:41778) 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 -- 172.21.15.19:0/3973562225 learned_addr learned my addr 172.21.15.19:0/3973562225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:40.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10ae465640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 -- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 msgr2=0x55826f6b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 -- 172.21.15.19:0/3973562225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 msgr2=0x55826f6b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.595+0000 7f10adc64640 1 -- 172.21.15.19:0/3973562225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55826f6d60a0 con 0x55826f6bcbb0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10adc64640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f109800b8b0 tx=0x7f1098000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2add9922c96b9abd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098024070 con 0x55826f6bcbb0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a401b810 con 0x55826f6bcbb0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1098005070 con 0x55826f6bcbb0 2024-09-12T03:19:40.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1098005270 con 0x55826f6bcbb0 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 10 client.15408 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55826f6d7340 con 0x55826f6bcbb0 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10a401b810 con 0x55826f6bcbb0 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1098045070 con 0x55826f6bcbb0 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 10 client.15408 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 20 client.15408 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f73bf90 0x55826f73aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:40.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 4 client.15408 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.596+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "status"}) -- 0x55826f61fb10 con 0x55826f73bf90 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f10ae465640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f73bf90 0x55826f73aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f10ae465640 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f73bf90 0x55826f73aca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f4c3573a361df39d server_cookie=c02811998c99f877 in_seq=0 out_seq=0 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f1091ffb640 10 client.15408 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f1091ffb640 1 -- 172.21.15.19:0/3973562225 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55826f61fb10 con 0x55826f73bf90 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f1091ffb640 10 client.15408 handle_command_reply: tid=0 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f10b2aa2740 1 client.15408 shutdown 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 msgr2=0x55826f6d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.597+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f109800b8b0 tx=0x7f1098000bc0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 shutdown_connections 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55826f73bf90 0x55826f73aca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55826f6bcbb0 0x55826f6d5b60 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55826f6b22f0 0x55826f6b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 --2- 172.21.15.19:0/3973562225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55826f6b19d0 0x55826f6b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 >> 172.21.15.19:0/3973562225 conn(0x55826f6adf60 msgr2=0x55826f6b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f1091ffb640 0 client.15408 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 shutdown_connections 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 1 -- 172.21.15.19:0/3973562225 wait complete. 2024-09-12T03:19:40.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b2aa2740 20 client.15408 trim_cache size 0 max 16384 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 2, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24463, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 141.64149748299999, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:19:40.611 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 177.58965178099999 2024-09-12T03:19:40.612 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:40.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 msgr2=0x7f10940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 0x7f10940a50c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f10a00bcde0 tx=0x7f10a008ee60 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 msgr2=0x7f10a81a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f10a0005cc0 tx=0x7f10a0003970 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 shutdown_connections 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10940a2c00 0x7f10940a50c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10a8106a80 0x7f10a81a4310 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10a8101420 0x7f10a81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.598+0000 7f10b06c9640 1 --2- 172.21.15.19:0/1922558414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10a8100a80 0x7f10a8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.599+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 >> 172.21.15.19:0/1922558414 conn(0x7f10a80fc7a0 msgr2=0x7f10a80fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.599+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 shutdown_connections 2024-09-12T03:19:40.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:40.599+0000 7f10b06c9640 1 -- 172.21.15.19:0/1922558414 wait complete. 2024-09-12T03:19:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:41 smithi121 ceph-mon[34963]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T03:19:41.411 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:41 smithi019 ceph-mon[30953]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T03:19:41.442 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T03:19:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:41 smithi135 ceph-mon[35474]: pgmap v356: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-12T03:19:41.702 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:42.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.176+0000 7fc444198640 1 -- 172.21.15.19:0/143432677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c101de0 msgr2=0x7fc43c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.176+0000 7fc444198640 1 --2- 172.21.15.19:0/143432677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c101de0 0x7fc43c1021c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc438009900 tx=0x7fc43802a410 comp rx=0 tx=0).stop 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- 172.21.15.19:0/143432677 shutdown_connections 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- 172.21.15.19:0/143432677 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c106f10 0x7fc43c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- 172.21.15.19:0/143432677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c102790 0x7fc43c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- 172.21.15.19:0/143432677 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c101de0 0x7fc43c1021c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- 172.21.15.19:0/143432677 >> 172.21.15.19:0/143432677 conn(0x7fc43c0fd8c0 msgr2=0x7fc43c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- 172.21.15.19:0/143432677 shutdown_connections 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- 172.21.15.19:0/143432677 wait complete. 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 Processor -- start 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- start start 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c101de0 0x7fc43c198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c102790 0x7fc43c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c111f30 con 0x7fc43c106f10 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.177+0000 7fc444198640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c111db0 con 0x7fc43c101de0 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc444198640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c1120b0 con 0x7fc43c102790 2024-09-12T03:19:42.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41804/0 (socket says 172.21.15.19:41804) 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 -- 172.21.15.19:0/1302814961 learned_addr learned my addr 172.21.15.19:0/1302814961 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc441f34640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c101de0 0x7fc43c198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc441733640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c102790 0x7fc43c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 -- 172.21.15.19:0/1302814961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c102790 msgr2=0x7fc43c1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c102790 0x7fc43c1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 -- 172.21.15.19:0/1302814961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c101de0 msgr2=0x7fc43c198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c101de0 0x7fc43c198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 -- 172.21.15.19:0/1302814961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c19d440 con 0x7fc43c106f10 2024-09-12T03:19:42.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc442735640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc434009fd0 tx=0x7fc43400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc432ffd640 1 -- 172.21.15.19:0/1302814961 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434017070 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc432ffd640 1 -- 172.21.15.19:0/1302814961 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc434003e50 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c19d730 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc432ffd640 1 -- 172.21.15.19:0/1302814961 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340106c0 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.178+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c1a5010 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.181+0000 7fc446571740 1 Processor -- start 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.181+0000 7fc432ffd640 1 -- 172.21.15.19:0/1302814961 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc434003970 con 0x7fc43c106f10 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 -- start start 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6832c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c68443b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609c6799090 con 0x5609c68331d0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609c6740a70 con 0x5609c68328b0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc446571740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609c6795050 con 0x5609c6841fc0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41816/0 (socket says 172.21.15.19:41816) 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 -- 172.21.15.19:0/914730423 learned_addr learned my addr 172.21.15.19:0/914730423 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441733640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6832c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441733640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c68443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 -- 172.21.15.19:0/914730423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 msgr2=0x5609c68443b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c68443b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 -- 172.21.15.19:0/914730423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 msgr2=0x5609c6832c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6832c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441f34640 1 -- 172.21.15.19:0/914730423 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c674d130 con 0x5609c68331d0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc432ffd640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 0x7fc4280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.182+0000 7fc441733640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c68443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc441f34640 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc438009900 tx=0x7fc438005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68455deda31f625a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc432ffd640 1 -- 172.21.15.19:0/1302814961 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc4340ceda0 con 0x7fc43c106f10 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc441733640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 0x7fc4280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc427fff640 1 -- 172.21.15.19:0/914730423 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438040070 con 0x5609c68331d0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc427fff640 1 -- 172.21.15.19:0/914730423 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4380319f0 con 0x5609c68331d0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.183+0000 7fc427fff640 1 -- 172.21.15.19:0/914730423 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438035040 con 0x5609c68331d0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- 172.21.15.19:0/914730423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 msgr2=0x5609c6841a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc441733640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 0x7fc4280a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5609c68302e0 tx=0x7fc42c004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc438009900 tx=0x7fc438005e80 comp rx=0 tx=0).stop 2024-09-12T03:19:42.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- 172.21.15.19:0/914730423 shutdown_connections 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c68443b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c6841a80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 --2- 172.21.15.19:0/914730423 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6832c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- 172.21.15.19:0/914730423 >> 172.21.15.19:0/914730423 conn(0x5609c682ef60 msgr2=0x5609c682f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- 172.21.15.19:0/914730423 shutdown_connections 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- 172.21.15.19:0/914730423 wait complete. 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 -- start start 2024-09-12T03:19:42.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.184+0000 7fc446571740 1 Processor -- start 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6841790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c683f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609c6844b90 con 0x5609c68331d0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609c6753f20 con 0x5609c68328b0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609c6844d10 con 0x5609c6841fc0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc446571740 10 client.0 _ll_register_callbacks cb 0x5609c671d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41832/0 (socket says 172.21.15.19:41832) 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 -- 172.21.15.19:0/3670488323 learned_addr learned my addr 172.21.15.19:0/3670488323 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441f34640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6841790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc442735640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c683f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 -- 172.21.15.19:0/3670488323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 msgr2=0x5609c683f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c683f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 -- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 msgr2=0x5609c6841790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6841790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 -- 172.21.15.19:0/3670488323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c650a2b0 con 0x5609c68331d0 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc442735640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c683f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:42.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc441733640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00fb80 tx=0x7fc42c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19599e843005fb75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.185+0000 7fc4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c01ccd0 con 0x5609c68331d0 2024-09-12T03:19:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc42c027480 con 0x5609c68331d0 2024-09-12T03:19:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4380319f0 con 0x5609c68331d0 2024-09-12T03:19:42.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc42c027680 con 0x5609c68331d0 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 10 client.15420 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609c6850420 con 0x5609c68331d0 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4380319f0 con 0x5609c68331d0 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc42c01e7b0 con 0x5609c68331d0 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 10 client.15420 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 20 client.15420 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c6852a80 0x5609c6853ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 4 client.15420 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.186+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609c67a6600 con 0x5609c6852a80 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.187+0000 7fc441f34640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c6852a80 0x5609c6853ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.187+0000 7fc441f34640 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c6852a80 0x5609c6853ef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d399d374e1af685c server_cookie=bb1ec1b054041ba6 in_seq=0 out_seq=0 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.187+0000 7fc4257fa640 10 client.15420 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc4257fa640 1 -- 172.21.15.19:0/3670488323 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609c67a6600 con 0x5609c6852a80 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc4257fa640 10 client.15420 handle_command_reply: tid=0 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 client.15420 shutdown 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 msgr2=0x5609c683eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc42c00fb80 tx=0x7fc42c005850 comp rx=0 tx=0).stop 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 shutdown_connections 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c6852a80 0x5609c6853ef0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c683f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c683eb30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc4257fa640 0 client.15420 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 --2- 172.21.15.19:0/3670488323 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68328b0 0x5609c6841790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 >> 172.21.15.19:0/3670488323 conn(0x5609c682ef60 msgr2=0x5609c6834cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 shutdown_connections 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 1 -- 172.21.15.19:0/3670488323 wait complete. 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.188+0000 7fc446571740 20 client.15420 trim_cache size 0 max 16384 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.206+0000 7fc446571740 1 Processor -- start 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 -- start start 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c68328b0 0x5609c6853ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c6841fc0 0x5609c683d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609c6852230 con 0x5609c68331d0 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609c6852900 con 0x5609c6841fc0 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc446571740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609c6852fd0 con 0x5609c68328b0 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41840/0 (socket says 172.21.15.19:41840) 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 -- 172.21.15.19:0/4012958910 learned_addr learned my addr 172.21.15.19:0/4012958910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc442735640 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c6841fc0 0x5609c683d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441f34640 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c68328b0 0x5609c6853ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 -- 172.21.15.19:0/4012958910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c68328b0 msgr2=0x5609c6853ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c68328b0 0x5609c6853ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 -- 172.21.15.19:0/4012958910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c6841fc0 msgr2=0x5609c683d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c6841fc0 0x5609c683d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.207+0000 7fc441733640 1 -- 172.21.15.19:0/4012958910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609c6833000 con 0x5609c68331d0 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.208+0000 7fc441733640 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc42c016040 tx=0x7fc42c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ac101583fb667d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.208+0000 7fc4277fe640 1 -- 172.21.15.19:0/4012958910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c041070 con 0x5609c68331d0 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.208+0000 7fc4277fe640 1 -- 172.21.15.19:0/4012958910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42c01c710 con 0x5609c68331d0 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.208+0000 7fc4277fe640 1 -- 172.21.15.19:0/4012958910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c027a80 con 0x5609c68331d0 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- 172.21.15.19:0/4012958910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 msgr2=0x5609c68523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc42c016040 tx=0x7fc42c00ef30 comp rx=0 tx=0).stop 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- 172.21.15.19:0/4012958910 shutdown_connections 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c6841fc0 0x5609c683d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68331d0 0x5609c68523b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 --2- 172.21.15.19:0/4012958910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c68328b0 0x5609c6853ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- 172.21.15.19:0/4012958910 >> 172.21.15.19:0/4012958910 conn(0x5609c682ef60 msgr2=0x5609c6829ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- 172.21.15.19:0/4012958910 shutdown_connections 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- 172.21.15.19:0/4012958910 wait complete. 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.209+0000 7fc446571740 1 -- start start 2024-09-12T03:19:42.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 Processor -- start 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68331d0 0x5609c685c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c6855690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5609c6853d70 con 0x5609c68328b0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5609c68536a0 con 0x5609c68331d0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5609c6854440 con 0x5609c6841fc0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc446571740 10 client.0 _ll_register_callbacks cb 0x7fc42c00e520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc441f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc441f34640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41852/0 (socket says 172.21.15.19:41852) 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc441f34640 1 -- 172.21.15.19:0/1887988995 learned_addr learned my addr 172.21.15.19:0/1887988995 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc441733640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68331d0 0x5609c685c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.210+0000 7fc442735640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c6855690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 -- 172.21.15.19:0/1887988995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 msgr2=0x5609c6855690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c6855690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 -- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68331d0 msgr2=0x5609c685c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68331d0 0x5609c685c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 -- 172.21.15.19:0/1887988995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609c6855ec0 con 0x5609c68328b0 2024-09-12T03:19:42.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc441f34640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc438000c00 tx=0x7fc438031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d718d783e5d6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438040040 con 0x5609c68328b0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc438004450 con 0x5609c68328b0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc42c01c710 con 0x5609c68328b0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc438035040 con 0x5609c68328b0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc446571740 10 client.15426 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5609c685a8e0 con 0x5609c68328b0 2024-09-12T03:19:42.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc42c01c710 con 0x5609c68328b0 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.211+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc438004ce0 con 0x5609c68328b0 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 10 client.15426 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 20 client.15426 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c68c1310 0x5609c68c0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 4 client.15426 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:42.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "status"}) -- 0x5609c67a0b10 con 0x5609c68c1310 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc441733640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c68c1310 0x5609c68c0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc441733640 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c68c1310 0x5609c68c0020 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e0647cb448ffee7 server_cookie=fca28be8b49881a1 in_seq=0 out_seq=0 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc4257fa640 10 client.15426 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc4257fa640 1 -- 172.21.15.19:0/1887988995 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5609c67a0b10 con 0x5609c68c1310 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc4257fa640 10 client.15426 handle_command_reply: tid=0 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.212+0000 7fc446571740 1 client.15426 shutdown 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 msgr2=0x5609c685c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc438000c00 tx=0x7fc438031a30 comp rx=0 tx=0).stop 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 shutdown_connections 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5609c68c1310 0x5609c68c0020 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5609c6841fc0 0x5609c6855690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5609c68331d0 0x5609c685c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc4257fa640 0 client.15426 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 --2- 172.21.15.19:0/1887988995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5609c68328b0 0x5609c685c360 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 >> 172.21.15.19:0/1887988995 conn(0x5609c682ef60 msgr2=0x5609c683d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 shutdown_connections 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 1 -- 172.21.15.19:0/1887988995 wait complete. 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc446571740 20 client.15426 trim_cache size 0 max 16384 2024-09-12T03:19:42.233 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 1, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24467, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 144.265011501, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 163.614862313 2024-09-12T03:19:42.234 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 msgr2=0x7fc4280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 0x7fc4280a5040 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x5609c68302e0 tx=0x7fc42c004730 comp rx=0 tx=0).stop 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 msgr2=0x7fc43c19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.213+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc434009fd0 tx=0x7fc43400b260 comp rx=0 tx=0).stop 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 shutdown_connections 2024-09-12T03:19:42.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4280a2b80 0x7fc4280a5040 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc43c106f10 0x7fc43c19cd60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc43c102790 0x7fc43c1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 --2- 172.21.15.19:0/1302814961 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc43c101de0 0x7fc43c198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 >> 172.21.15.19:0/1302814961 conn(0x7fc43c0fd8c0 msgr2=0x7fc43c0fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 shutdown_connections 2024-09-12T03:19:42.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:42.214+0000 7fc444198640 1 -- 172.21.15.19:0/1302814961 wait complete. 2024-09-12T03:19:42.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T03:19:43.161 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:43 smithi121 ceph-mon[34963]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:43 smithi019 ceph-mon[30953]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:43 smithi135 ceph-mon[35474]: pgmap v357: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:43.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 -- 172.21.15.19:0/622644318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 msgr2=0x7fc27010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 --2- 172.21.15.19:0/622644318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27010e610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc268009ca0 tx=0x7fc26802a950 comp rx=0 tx=0).stop 2024-09-12T03:19:43.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 -- 172.21.15.19:0/622644318 shutdown_connections 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 --2- 172.21.15.19:0/622644318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27010e610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 --2- 172.21.15.19:0/622644318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 0x7fc27010bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 --2- 172.21.15.19:0/622644318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc2701031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 -- 172.21.15.19:0/622644318 >> 172.21.15.19:0/622644318 conn(0x7fc2700fc7a0 msgr2=0x7fc2700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 -- 172.21.15.19:0/622644318 shutdown_connections 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.632+0000 7fc275bfb640 1 -- 172.21.15.19:0/622644318 wait complete. 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 Processor -- start 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 -- start start 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc270198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 0x7fc2701988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc270111f70 con 0x7fc27010c180 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc270111df0 con 0x7fc270103700 2024-09-12T03:19:43.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc275bfb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701120f0 con 0x7fc270102de0 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41880/0 (socket says 172.21.15.19:41880) 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc26ffff640 1 -- 172.21.15.19:0/1486976821 learned_addr learned my addr 172.21.15.19:0/1486976821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc270198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.633+0000 7fc26effd640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 0x7fc2701988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 -- 172.21.15.19:0/1486976821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 msgr2=0x7fc270198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc270198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 -- 172.21.15.19:0/1486976821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 msgr2=0x7fc2701988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 0x7fc2701988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 -- 172.21.15.19:0/1486976821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27019d310 con 0x7fc27010c180 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc270198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:43.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26ffff640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc26802ae70 tx=0x7fc26802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26cff9640 1 -- 172.21.15.19:0/1486976821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268034040 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27019d5a0 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26cff9640 1 -- 172.21.15.19:0/1486976821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc268030920 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc26cff9640 1 -- 172.21.15.19:0/1486976821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc268045480 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.634+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a4ee0 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.636+0000 7fc26cff9640 1 -- 172.21.15.19:0/1486976821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc268037020 con 0x7fc27010c180 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.636+0000 7fc277fd4740 1 Processor -- start 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 -- start start 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415ca2f0 0x5646415d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564641530090 con 0x5646415c99d0 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646414d7a70 con 0x5646415ca2f0 2024-09-12T03:19:43.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc277fd4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56464152c050 con 0x5646415d4bb0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41892/0 (socket says 172.21.15.19:41892) 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 -- 172.21.15.19:0/301572301 learned_addr learned my addr 172.21.15.19:0/301572301 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415ca2f0 0x5646415d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 -- 172.21.15.19:0/301572301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 msgr2=0x5646415d6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 -- 172.21.15.19:0/301572301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415ca2f0 msgr2=0x5646415d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415ca2f0 0x5646415d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 -- 172.21.15.19:0/301572301 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646414e4130 con 0x5646415c99d0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26cff9640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 0x7fc2580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.637+0000 7fc26effd640 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc26400cc40 tx=0x7fc264010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23fb3d9bf75a11c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.638+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 0x7fc2580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.638+0000 7fc26cff9640 1 -- 172.21.15.19:0/1486976821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc2680effb0 con 0x7fc27010c180 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.638+0000 7fc255ffb640 1 -- 172.21.15.19:0/301572301 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264016070 con 0x5646415c99d0 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.638+0000 7fc255ffb640 1 -- 172.21.15.19:0/301572301 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc264012b60 con 0x5646415c99d0 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.638+0000 7fc255ffb640 1 -- 172.21.15.19:0/301572301 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264011b80 con 0x5646415c99d0 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- 172.21.15.19:0/301572301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 msgr2=0x5646415c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc26400cc40 tx=0x7fc264010740 comp rx=0 tx=0).stop 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 0x7fc2580a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5646415c6dc0 tx=0x7fc26000b710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- 172.21.15.19:0/301572301 shutdown_connections 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415ca2f0 0x5646415d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 --2- 172.21.15.19:0/301572301 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415c99d0 0x5646415c9db0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- 172.21.15.19:0/301572301 >> 172.21.15.19:0/301572301 conn(0x5646415c5f60 msgr2=0x5646415c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- 172.21.15.19:0/301572301 shutdown_connections 2024-09-12T03:19:43.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- 172.21.15.19:0/301572301 wait complete. 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 -- start start 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.639+0000 7fc277fd4740 1 Processor -- start 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646415d7780 con 0x5646415ca2f0 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646414eaf20 con 0x5646415c99d0 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646415d7900 con 0x5646415d4bb0 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 10 client.0 _ll_register_callbacks cb 0x5646414b4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41900/0 (socket says 172.21.15.19:41900) 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 -- 172.21.15.19:0/2393198459 learned_addr learned my addr 172.21.15.19:0/2393198459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:43.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26ffff640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 -- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 msgr2=0x5646415d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 -- 172.21.15.19:0/2393198459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 msgr2=0x5646415d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 -- 172.21.15.19:0/2393198459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646412a12b0 con 0x5646415ca2f0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc26effd640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc264011720 tx=0x7fc26400c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8af55281b44e031 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc24effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264016030 con 0x5646415ca2f0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc264012b60 con 0x5646415ca2f0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc264011900 con 0x5646415ca2f0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.640+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc264011b00 con 0x5646415ca2f0 2024-09-12T03:19:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5646415e3090 con 0x5646415ca2f0 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc264012b60 con 0x5646415ca2f0 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc264004750 con 0x5646415ca2f0 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 10 client.15438 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 20 client.15438 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646415e5320 0x5646415e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 4 client.15438 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.641+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56464153d600 con 0x5646415e5320 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.642+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646415e5320 0x5646415e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.642+0000 7fc24effd640 10 client.15438 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.642+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646415e5320 0x5646415e6790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e875b322ecb471b3 server_cookie=7b2eb22500aa344b in_seq=0 out_seq=0 2024-09-12T03:19:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc24effd640 1 -- 172.21.15.19:0/2393198459 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56464153d600 con 0x5646415e5320 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc24effd640 10 client.15438 handle_command_reply: tid=0 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 client.15438 shutdown 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 msgr2=0x5646415d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc264011720 tx=0x7fc26400c3d0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 shutdown_connections 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415d1780 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646415e5320 0x5646415e6790 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2393198459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc24effd640 0 client.15438 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 >> 172.21.15.19:0/2393198459 conn(0x5646415c5f60 msgr2=0x5646415cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 shutdown_connections 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 1 -- 172.21.15.19:0/2393198459 wait complete. 2024-09-12T03:19:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.643+0000 7fc277fd4740 20 client.15438 trim_cache size 0 max 16384 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 Processor -- start 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 -- start start 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 0x5646415e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 0x5646415ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646415e4ad0 con 0x5646415ca2f0 2024-09-12T03:19:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646415e51a0 con 0x5646415d4bb0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc277fd4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646415e5870 con 0x5646415c99d0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41914/0 (socket says 172.21.15.19:41914) 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.661+0000 7fc26effd640 1 -- 172.21.15.19:0/907389268 learned_addr learned my addr 172.21.15.19:0/907389268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 0x5646415e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26ffff640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 0x5646415ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 -- 172.21.15.19:0/907389268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 msgr2=0x5646415e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 0x5646415e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 -- 172.21.15.19:0/907389268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 msgr2=0x5646415ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 0x5646415ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 -- 172.21.15.19:0/907389268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646415c6be0 con 0x5646415ca2f0 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 0x5646415e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:43.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26ffff640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 0x5646415ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc26effd640 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc264011770 tx=0x7fc264012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db152f8d2b8cdae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc2557fa640 1 -- 172.21.15.19:0/907389268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264016070 con 0x5646415ca2f0 2024-09-12T03:19:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc2557fa640 1 -- 172.21.15.19:0/907389268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26401f040 con 0x5646415ca2f0 2024-09-12T03:19:43.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.662+0000 7fc2557fa640 1 -- 172.21.15.19:0/907389268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264011040 con 0x5646415ca2f0 2024-09-12T03:19:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- 172.21.15.19:0/907389268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 msgr2=0x5646415c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc264011770 tx=0x7fc264012920 comp rx=0 tx=0).stop 2024-09-12T03:19:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- 172.21.15.19:0/907389268 shutdown_connections 2024-09-12T03:19:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415d4bb0 0x5646415ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415c6340 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 --2- 172.21.15.19:0/907389268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415c99d0 0x5646415e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- 172.21.15.19:0/907389268 >> 172.21.15.19:0/907389268 conn(0x5646415c5f60 msgr2=0x5646415d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- 172.21.15.19:0/907389268 shutdown_connections 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- 172.21.15.19:0/907389268 wait complete. 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.663+0000 7fc277fd4740 1 -- start start 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 Processor -- start 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415eeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415ea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646415e6610 con 0x5646415ca2f0 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646415e5f40 con 0x5646415c99d0 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646415e6ce0 con 0x5646415d4bb0 2024-09-12T03:19:43.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc277fd4740 10 client.0 _ll_register_callbacks cb 0x7fc26400cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415eeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41926/0 (socket says 172.21.15.19:41926) 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 -- 172.21.15.19:0/2667471201 learned_addr learned my addr 172.21.15.19:0/2667471201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26ffff640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415ea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 -- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 msgr2=0x5646415ea750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415ea750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 -- 172.21.15.19:0/2667471201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 msgr2=0x5646415eeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415eeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.664+0000 7fc26effd640 1 -- 172.21.15.19:0/2667471201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646415eac90 con 0x5646415ca2f0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc26effd640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc264011fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdfcf2525b221a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:43.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc24effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26403f060 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc264011040 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc26401f040 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc264016040 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc277fd4740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56464164f780 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.665+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc26401f040 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fc264004210 con 0x5646415ca2f0 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 20 client.15444 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564641653ec0 0x564641652bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:43.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 4 client.15444 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "status"}) -- 0x564641537b10 con 0x564641653ec0 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564641653ec0 0x564641652bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc24effd640 10 client.15444 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.666+0000 7fc26f7fe640 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564641653ec0 0x564641652bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1bd6bdc0ef62939 server_cookie=1197d63348809916 in_seq=0 out_seq=0 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc24effd640 1 -- 172.21.15.19:0/2667471201 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564641537b10 con 0x564641653ec0 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc24effd640 10 client.15444 handle_command_reply: tid=0 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 client.15444 shutdown 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 msgr2=0x5646415ea210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc264000c00 tx=0x7fc264011fb0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 shutdown_connections 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646415d4bb0 0x5646415ea750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646415ca2f0 0x5646415ea210 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564641653ec0 0x564641652bd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 --2- 172.21.15.19:0/2667471201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646415c99d0 0x5646415eeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc24effd640 0 client.15444 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 >> 172.21.15.19:0/2667471201 conn(0x5646415c5f60 msgr2=0x5646415d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 shutdown_connections 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 1 -- 172.21.15.19:0/2667471201 wait complete. 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.667+0000 7fc277fd4740 20 client.15444 trim_cache size 0 max 16384 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_fsid": "45fb56a2-70b4-11ef-bcea-c7b262605968", 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stdout: "whoami": 0, 2024-09-12T03:19:43.672 INFO:teuthology.orchestra.run.smithi019.stdout: "id": 24487, 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "want_state": "up:active", 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "up:active", 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "fs_name": "cephfs", 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rank_uptime": 152.75912464999999, 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "mdsmap_epoch": 28, 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch": 82, 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_epoch_barrier": 82, 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 175.20389667200001 2024-09-12T03:19:43.673 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 msgr2=0x7fc2580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 0x7fc2580a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x5646415c6dc0 tx=0x7fc26000b710 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 msgr2=0x7fc27019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc26802ae70 tx=0x7fc26802f5b0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 shutdown_connections 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc2580a2b80 0x7fc2580a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc27010c180 0x7fc27019cc30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc270103700 0x7fc2701988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 --2- 172.21.15.19:0/1486976821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc270102de0 0x7fc270198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:43.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 >> 172.21.15.19:0/1486976821 conn(0x7fc2700fc7a0 msgr2=0x7fc2700fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 shutdown_connections 2024-09-12T03:19:43.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:43.668+0000 7fc275bfb640 1 -- 172.21.15.19:0/1486976821 wait complete. 2024-09-12T03:19:44.379 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T03:19:45.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:45 smithi121 ceph-mon[34963]: pgmap v358: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:45 smithi019 ceph-mon[30953]: pgmap v358: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:45 smithi135 ceph-mon[35474]: pgmap v358: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:47.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:47 smithi121 ceph-mon[34963]: pgmap v359: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:47 smithi019 ceph-mon[30953]: pgmap v359: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:47 smithi135 ceph-mon[35474]: pgmap v359: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T03:19:49.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:49 smithi121 ceph-mon[34963]: pgmap v360: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:19:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:49 smithi019 ceph-mon[30953]: pgmap v360: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:19:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:49 smithi135 ceph-mon[35474]: pgmap v360: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 341 B/s wr, 10 op/s 2024-09-12T03:19:51.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:51 smithi121 ceph-mon[34963]: pgmap v361: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:19:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:51 smithi019 ceph-mon[30953]: pgmap v361: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:19:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:51 smithi135 ceph-mon[35474]: pgmap v361: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:19:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:53 smithi019 ceph-mon[30953]: pgmap v362: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:19:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:53 smithi135 ceph-mon[35474]: pgmap v362: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:19:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:53 smithi121 ceph-mon[34963]: pgmap v362: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:19:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:55 smithi019 ceph-mon[30953]: pgmap v363: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:19:55.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:19:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:55 smithi135 ceph-mon[35474]: pgmap v363: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:19:55.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:55 smithi121 ceph-mon[34963]: pgmap v363: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:19:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:19:56.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.235+0000 7f4729139640 1 -- 172.21.15.19:0/2277189549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47241027c0 msgr2=0x7f4724106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.235+0000 7f4729139640 1 --2- 172.21.15.19:0/2277189549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47241027c0 0x7f4724106810 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4718009900 tx=0x7f471802a410 comp rx=0 tx=0).stop 2024-09-12T03:19:56.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 -- 172.21.15.19:0/2277189549 shutdown_connections 2024-09-12T03:19:56.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 --2- 172.21.15.19:0/2277189549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724106f40 0x7f472410eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 --2- 172.21.15.19:0/2277189549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f47241027c0 0x7f4724106810 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 --2- 172.21.15.19:0/2277189549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4724101e10 0x7f47241021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 -- 172.21.15.19:0/2277189549 >> 172.21.15.19:0/2277189549 conn(0x7f47240fd8d0 msgr2=0x7f47240ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 -- 172.21.15.19:0/2277189549 shutdown_connections 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 -- 172.21.15.19:0/2277189549 wait complete. 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.236+0000 7f4729139640 1 Processor -- start 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 -- start start 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 0x7f47241984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47241027c0 0x7f4724198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 0x7f472419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4724111fa0 con 0x7f4724106f40 2024-09-12T03:19:56.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4724111e20 con 0x7f4724101e10 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4729139640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4724112120 con 0x7f47241027c0 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 0x7f472419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 0x7f472419cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55320/0 (socket says 172.21.15.19:55320) 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f472359e640 1 -- 172.21.15.19:0/1249986133 learned_addr learned my addr 172.21.15.19:0/1249986133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f4722d9d640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 0x7f47241984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.237+0000 7f472259c640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47241027c0 0x7f4724198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 -- 172.21.15.19:0/1249986133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47241027c0 msgr2=0x7f4724198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47241027c0 0x7f4724198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 -- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 msgr2=0x7f472419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 0x7f472419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 -- 172.21.15.19:0/1249986133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472419d470 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4722d9d640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 0x7f47241984c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f471400d6d0 tx=0x7f471400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f470bfff640 1 -- 172.21.15.19:0/1249986133 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f470bfff640 1 -- 172.21.15.19:0/1249986133 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4714009820 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f470bfff640 1 -- 172.21.15.19:0/1249986133 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714004d00 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472419d760 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.238+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47241a5040 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.241+0000 7f470bfff640 1 -- 172.21.15.19:0/1249986133 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4714005220 con 0x7f4724101e10 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.241+0000 7f472b512740 1 Processor -- start 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.241+0000 7f472b512740 1 -- start start 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba6356870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356db0 0x55fba6361130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 0x55fba6363a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fba62bc2e0 con 0x55fba6356490 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fba6263ce0 con 0x55fba6361670 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472b512740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fba62b82a0 con 0x55fba6356db0 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f470bfff640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 0x7f470c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba6356870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba6356870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55328/0 (socket says 172.21.15.19:55328) 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472359e640 1 -- 172.21.15.19:0/65036192 learned_addr learned my addr 172.21.15.19:0/65036192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 0x55fba6363a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f470bfff640 1 -- 172.21.15.19:0/1249986133 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47140ceb00 con 0x7f4724101e10 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f4722d9d640 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356db0 0x55fba6361130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472359e640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 0x7f470c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 -- 172.21.15.19:0/65036192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356db0 msgr2=0x55fba6361130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356db0 0x55fba6361130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 -- 172.21.15.19:0/65036192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 msgr2=0x55fba6356870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba6356870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.242+0000 7f472259c640 1 -- 172.21.15.19:0/65036192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba6248440 con 0x55fba6361670 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.243+0000 7f472359e640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 0x7f470c0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f471c012df0 tx=0x7f471c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.243+0000 7f472259c640 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 0x55fba6363a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4718005940 tx=0x7f4718005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e78f03911e2dfa92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.243+0000 7f47097fa640 1 -- 172.21.15.19:0/65036192 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718040070 con 0x55fba6361670 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.243+0000 7f47097fa640 1 -- 172.21.15.19:0/65036192 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47180319f0 con 0x55fba6361670 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.243+0000 7f47097fa640 1 -- 172.21.15.19:0/65036192 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718035040 con 0x55fba6361670 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- 172.21.15.19:0/65036192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 msgr2=0x55fba6363a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 0x55fba6363a60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4718005940 tx=0x7f4718005e80 comp rx=0 tx=0).stop 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- 172.21.15.19:0/65036192 shutdown_connections 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6361670 0x55fba6363a60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356db0 0x55fba6361130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 --2- 172.21.15.19:0/65036192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba6356870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- 172.21.15.19:0/65036192 >> 172.21.15.19:0/65036192 conn(0x55fba6352a20 msgr2=0x55fba6352e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- 172.21.15.19:0/65036192 shutdown_connections 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- 172.21.15.19:0/65036192 wait complete. 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.244+0000 7f472b512740 1 -- start start 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 Processor -- start 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356db0 0x55fba635e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba635e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fba6368650 con 0x55fba6356db0 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356db0 0x55fba635e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fba6277050 con 0x55fba6356490 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f4722d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fba63687d0 con 0x55fba6361670 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f4722d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37256/0 (socket says 172.21.15.19:37256) 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f4722d9d640 1 -- 172.21.15.19:0/4268400468 learned_addr learned my addr 172.21.15.19:0/4268400468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472b512740 10 client.0 _ll_register_callbacks cb 0x55fba62b7160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:56.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.245+0000 7f472359e640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba635e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 -- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 msgr2=0x55fba635e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba635e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 -- 172.21.15.19:0/4268400468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356db0 msgr2=0x55fba635e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356db0 0x55fba635e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 -- 172.21.15.19:0/4268400468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba602d3c0 con 0x55fba6356490 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4722d9d640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f471400d6a0 tx=0x7f471409b030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34cdc1c48da3921 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4706ff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714020040 con 0x55fba6356490 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f47140b4260 con 0x55fba6356490 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47180319f0 con 0x55fba6356490 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f471409f6c0 con 0x55fba6356490 2024-09-12T03:19:56.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f472b512740 10 client.25043 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.246+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fba6373fb0 con 0x55fba6356490 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47180319f0 con 0x55fba6356490 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f47140099d0 con 0x55fba6356490 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 10 client.25043 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 10 client.25043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 20 client.25043 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63753c0 0x55fba63777b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 4 client.25043 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.247+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fba62c9850 con 0x55fba63753c0 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.248+0000 7f472259c640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63753c0 0x55fba63777b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.248+0000 7f472259c640 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63753c0 0x55fba63777b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68d8c82a3d190dee server_cookie=1a16a1901f3801af in_seq=0 out_seq=0 2024-09-12T03:19:56.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.248+0000 7f4706ff5640 10 client.25043 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f4706ff5640 1 -- 172.21.15.19:0/4268400468 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fba62c9850 con 0x55fba63753c0 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f4706ff5640 10 client.25043 handle_command_reply: tid=0 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 client.25043 shutdown 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 msgr2=0x55fba6360f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f471400d6a0 tx=0x7f471409b030 comp rx=0 tx=0).stop 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 shutdown_connections 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba635e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356db0 0x55fba635e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63753c0 0x55fba63777b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 --2- 172.21.15.19:0/4268400468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356490 0x55fba6360f80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f4706ff5640 0 client.25043 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 >> 172.21.15.19:0/4268400468 conn(0x55fba6352a20 msgr2=0x55fba6358750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 shutdown_connections 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 1 -- 172.21.15.19:0/4268400468 wait complete. 2024-09-12T03:19:56.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.249+0000 7f472b512740 20 client.25043 trim_cache size 0 max 16384 2024-09-12T03:19:56.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 Processor -- start 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 -- start start 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba637d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ab2c0 con 0x55fba6356490 2024-09-12T03:19:56.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ace00 con 0x55fba6356db0 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.269+0000 7f472b512740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ab990 con 0x55fba6361670 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4722d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4722d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55346/0 (socket says 172.21.15.19:55346) 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4722d9d640 1 -- 172.21.15.19:0/3942195945 learned_addr learned my addr 172.21.15.19:0/3942195945 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:56.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37266/0 (socket says 172.21.15.19:37266) 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 -- 172.21.15.19:0/3942195945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 msgr2=0x55fba637d5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba637d5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 -- 172.21.15.19:0/3942195945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 msgr2=0x55fba63718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 -- 172.21.15.19:0/3942195945 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fba63562c0 con 0x55fba6356db0 2024-09-12T03:19:56.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4722d9d640 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f472259c640 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f471802a930 tx=0x7f4718047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f01bef23b3761330 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4708ff9640 1 -- 172.21.15.19:0/3942195945 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718040040 con 0x55fba6356db0 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4708ff9640 1 -- 172.21.15.19:0/3942195945 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47180054e0 con 0x55fba6356db0 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.270+0000 7f4708ff9640 1 -- 172.21.15.19:0/3942195945 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4718035040 con 0x55fba6356db0 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 -- 172.21.15.19:0/3942195945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 msgr2=0x55fba637d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f471802a930 tx=0x7f4718047040 comp rx=0 tx=0).stop 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 -- 172.21.15.19:0/3942195945 shutdown_connections 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6361670 0x55fba637d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba637d080 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 --2- 172.21.15.19:0/3942195945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6356490 0x55fba63718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 -- 172.21.15.19:0/3942195945 >> 172.21.15.19:0/3942195945 conn(0x55fba6352a20 msgr2=0x55fba635cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 -- 172.21.15.19:0/3942195945 shutdown_connections 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.271+0000 7f472b512740 1 -- 172.21.15.19:0/3942195945 wait complete. 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 -- start start 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 Processor -- start 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356490 0x55fba6369b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba636a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ac730 con 0x55fba6361670 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ac060 con 0x55fba6356db0 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f47241a92a0 con 0x55fba6356490 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472b512740 10 client.0 _ll_register_callbacks cb 0x7f471409b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55350/0 (socket says 172.21.15.19:55350) 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.272+0000 7f472359e640 1 -- 172.21.15.19:0/3776355100 learned_addr learned my addr 172.21.15.19:0/3776355100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472259c640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba636a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f4722d9d640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356490 0x55fba6369b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 -- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356490 msgr2=0x55fba6369b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356490 0x55fba6369b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 -- 172.21.15.19:0/3776355100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 msgr2=0x55fba636a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba636a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 -- 172.21.15.19:0/3776355100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fba634e000 con 0x55fba6361670 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472359e640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f471c009370 tx=0x7f471c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d9f4c7ef5f542 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f4706ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00ec80 con 0x55fba6361670 2024-09-12T03:19:56.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47180054e0 con 0x55fba6361670 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.273+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f471c00ee20 con 0x55fba6361670 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f471c003970 con 0x55fba6361670 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fba63e0600 con 0x55fba6361670 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47180054e0 con 0x55fba6361670 2024-09-12T03:19:56.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f471c02b400 con 0x55fba6361670 2024-09-12T03:19:56.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:19:56.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:19:56.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 20 client.15462 populate_metadata read hostname 'smithi019' 2024-09-12T03:19:56.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63e0d30 0x55fba63e3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:19:56.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 4 client.15462 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.274+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fba635eb00 con 0x55fba63e0d30 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f4722d9d640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63e0d30 0x55fba63e3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f4706ff5640 10 client.15462 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f4722d9d640 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63e0d30 0x55fba63e3150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee038a609cea97d server_cookie=4419c8a96123c84d in_seq=0 out_seq=0 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f4706ff5640 1 -- 172.21.15.19:0/3776355100 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fba635eb00 con 0x55fba63e0d30 2024-09-12T03:19:56.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f4706ff5640 10 client.15462 handle_command_reply: tid=0 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.275+0000 7f472b512740 1 client.15462 shutdown 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 msgr2=0x55fba63df050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f471c009370 tx=0x7f471c004a40 comp rx=0 tx=0).stop 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 shutdown_connections 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fba6361670 0x55fba63df050 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fba63e0d30 0x55fba63e3150 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fba6356db0 0x55fba636a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 --2- 172.21.15.19:0/3776355100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fba6356490 0x55fba6369b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 >> 172.21.15.19:0/3776355100 conn(0x55fba6352a20 msgr2=0x55fba6351380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f4706ff5640 0 client.15462 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:19:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 shutdown_connections 2024-09-12T03:19:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 1 -- 172.21.15.19:0/3776355100 wait complete. 2024-09-12T03:19:56.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.276+0000 7f472b512740 20 client.15462 trim_cache size 0 max 16384 2024-09-12T03:19:56.296 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:19:56.296 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:19:56.296 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:19:56.297 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:19:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 msgr2=0x7f470c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 0x7f470c0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f471c012df0 tx=0x7f471c004ab0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 msgr2=0x7f47241984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:19:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 0x7f47241984c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f471400d6d0 tx=0x7f471400db90 comp rx=0 tx=0).stop 2024-09-12T03:19:56.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 shutdown_connections 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f470c0a2c00 0x7f470c0a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4724106f40 0x7f472419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f47241027c0 0x7f4724198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 --2- 172.21.15.19:0/1249986133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4724101e10 0x7f47241984c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 >> 172.21.15.19:0/1249986133 conn(0x7f47240fd8d0 msgr2=0x7f47240fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 shutdown_connections 2024-09-12T03:19:56.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:19:56.277+0000 7f4729139640 1 -- 172.21.15.19:0/1249986133 wait complete. 2024-09-12T03:19:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:57 smithi019 ceph-mon[30953]: pgmap v364: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:57 smithi135 ceph-mon[35474]: pgmap v364: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:57 smithi121 ceph-mon[34963]: pgmap v364: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:19:59.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:19:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:59.380 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T03:19:59.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:19:59.471 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:19:59 smithi019 ceph-mon[30953]: pgmap v365: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T03:19:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:19:59 smithi135 ceph-mon[35474]: pgmap v365: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T03:19:59.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:19:59 smithi121 ceph-mon[34963]: pgmap v365: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 9 op/s 2024-09-12T03:19:59.656 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:00.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 -- 172.21.15.19:0/1633855274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 msgr2=0x7fca00110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 --2- 172.21.15.19:0/1633855274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca00110900 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009ca0 tx=0x7fc9f802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 -- 172.21.15.19:0/1633855274 shutdown_connections 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 --2- 172.21.15.19:0/1633855274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca00110900 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 --2- 172.21.15.19:0/1633855274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca00101380 0x7fca0010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 --2- 172.21.15.19:0/1633855274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca001009d0 0x7fca00100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 -- 172.21.15.19:0/1633855274 >> 172.21.15.19:0/1633855274 conn(0x7fca000fc7d0 msgr2=0x7fca000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 -- 172.21.15.19:0/1633855274 shutdown_connections 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.163+0000 7fca05a84640 1 -- 172.21.15.19:0/1633855274 wait complete. 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.164+0000 7fca05a84640 1 Processor -- start 2024-09-12T03:20:00.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.164+0000 7fca05a84640 1 -- start start 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 0x7fca0019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca00101380 0x7fca0019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fca00114250 con 0x7fca0010e4f0 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fca001140d0 con 0x7fca001009d0 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fca05a84640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fca001143d0 con 0x7fca00101380 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55372/0 (socket says 172.21.15.19:55372) 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9fffff640 1 -- 172.21.15.19:0/2570951021 learned_addr learned my addr 172.21.15.19:0/2570951021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 0x7fca0019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9feffd640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca00101380 0x7fca0019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 -- 172.21.15.19:0/2570951021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca00101380 msgr2=0x7fca0019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca00101380 0x7fca0019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 -- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 msgr2=0x7fca0019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 -- 172.21.15.19:0/2570951021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0019f6c0 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9fffff640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.165+0000 7fc9ff7fe640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 0x7fca0019a710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e9f0 tx=0x7fc9f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.168+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000ccc0 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.168+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9f0005020 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.168+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f00049e0 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.168+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0019f9b0 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.168+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001a7290 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.169+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca00100db0 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.170+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9f0005560 con 0x7fca001009d0 2024-09-12T03:20:00.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.171+0000 7fc9fcff9640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 0x7fc9e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.171+0000 7fc9feffd640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 0x7fc9e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.171+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc9f00cefa0 con 0x7fca001009d0 2024-09-12T03:20:00.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.172+0000 7fc9feffd640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 0x7fc9e80a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca0010ed90 tx=0x7fc9f4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.176+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9f00051d0 con 0x7fca001009d0 2024-09-12T03:20:00.183 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:00.351 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:00 smithi019 ceph-mon[30953]: overall HEALTH_OK 2024-09-12T03:20:00.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.407+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca0019ba70 con 0x7fca001009d0 2024-09-12T03:20:00.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.408+0000 7fc9fcff9640 1 -- 172.21.15.19:0/2570951021 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7fc9f009beb0 con 0x7fca001009d0 2024-09-12T03:20:00.409 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:20:00.409 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:20:00.409 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:20:00.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 msgr2=0x7fc9e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 0x7fc9e80a50c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fca0010ed90 tx=0x7fc9f4005f90 comp rx=0 tx=0).stop 2024-09-12T03:20:00.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 msgr2=0x7fca0019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 0x7fca0019a710 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000e9f0 tx=0x7fc9f000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 shutdown_connections 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc9e80a2c00 0x7fc9e80a50c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fca0010e4f0 0x7fca0019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fca00101380 0x7fca0019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 --2- 172.21.15.19:0/2570951021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fca001009d0 0x7fca0019a710 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 >> 172.21.15.19:0/2570951021 conn(0x7fca000fc7d0 msgr2=0x7fca000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 shutdown_connections 2024-09-12T03:20:00.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.411+0000 7fca05a84640 1 -- 172.21.15.19:0/2570951021 wait complete. 2024-09-12T03:20:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:00 smithi135 ceph-mon[35474]: overall HEALTH_OK 2024-09-12T03:20:00.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:00 smithi121 ceph-mon[34963]: overall HEALTH_OK 2024-09-12T03:20:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 -- 172.21.15.19:0/1528627698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 msgr2=0x7f8e7410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 --2- 172.21.15.19:0/1528627698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e7410ea90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009ca0 tx=0x7f8e6c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 -- 172.21.15.19:0/1528627698 shutdown_connections 2024-09-12T03:20:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 --2- 172.21.15.19:0/1528627698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e7410ea90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 --2- 172.21.15.19:0/1528627698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74102790 0x7f8e741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 --2- 172.21.15.19:0/1528627698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74101de0 0x7f8e741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 -- 172.21.15.19:0/1528627698 >> 172.21.15.19:0/1528627698 conn(0x7f8e740fd8e0 msgr2=0x7f8e740ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.924+0000 7f8e79642640 1 -- 172.21.15.19:0/1528627698 shutdown_connections 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.925+0000 7f8e79642640 1 -- 172.21.15.19:0/1528627698 wait complete. 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.925+0000 7f8e79642640 1 Processor -- start 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.925+0000 7f8e79642640 1 -- start start 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74101de0 0x7f8e740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74102790 0x7f8e74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111f90 con 0x7f8e74106f10 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111e10 con 0x7f8e74101de0 2024-09-12T03:20:00.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e79642640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74112110 con 0x7f8e74102790 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33672/0 (socket says 172.21.15.19:33672) 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e72ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74101de0 0x7f8e740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 -- 172.21.15.19:0/3666804126 learned_addr learned my addr 172.21.15.19:0/3666804126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e6bfff640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74102790 0x7f8e74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 -- 172.21.15.19:0/3666804126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74102790 msgr2=0x7f8e74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74102790 0x7f8e74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 -- 172.21.15.19:0/3666804126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74101de0 msgr2=0x7f8e740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74101de0 0x7f8e740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 -- 172.21.15.19:0/3666804126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e741a69e0 con 0x7f8e74106f10 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.926+0000 7f8e737fe640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c031a20 tx=0x7f8e6c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.927+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c003b70 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.927+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e741a6c70 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.927+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e6c003d10 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.927+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e6c02fc80 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.927+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a7200 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.928+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e741021c0 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.930+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e6c0046a0 con 0x7f8e74106f10 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.930+0000 7f8e70ff9640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 0x7f8e600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.931+0000 7f8e72ffd640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 0x7f8e600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:00.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.931+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8e6c0f0340 con 0x7f8e74106f10 2024-09-12T03:20:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.931+0000 7f8e72ffd640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 0x7f8e600a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009990 tx=0x7f8e64009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:00.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:00.935+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e6c0bd3d0 con 0x7f8e74106f10 2024-09-12T03:20:01.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.158+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e7410a0f0 con 0x7f8e74106f10 2024-09-12T03:20:01.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.159+0000 7f8e70ff9640 1 -- 172.21.15.19:0/3666804126 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f8e6c0bccd0 con 0x7f8e74106f10 2024-09-12T03:20:01.159 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:20:01.159 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:20:01.160 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.161+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 msgr2=0x7f8e600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 0x7f8e600a5040 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009990 tx=0x7f8e64009110 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 msgr2=0x7f8e74074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c031a20 tx=0x7f8e6c031a50 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 shutdown_connections 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8e600a2b80 0x7f8e600a5040 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8e74106f10 0x7f8e74074f70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8e74102790 0x7f8e74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 --2- 172.21.15.19:0/3666804126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8e74101de0 0x7f8e740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 >> 172.21.15.19:0/3666804126 conn(0x7f8e740fd8e0 msgr2=0x7f8e740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:01.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 shutdown_connections 2024-09-12T03:20:01.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:01.162+0000 7f8e79642640 1 -- 172.21.15.19:0/3666804126 wait complete. 2024-09-12T03:20:01.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:01 smithi019 ceph-mon[30953]: pgmap v366: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-12T03:20:01.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2570951021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:01.403 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3666804126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:01.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:20:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:01 smithi135 ceph-mon[35474]: pgmap v366: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-12T03:20:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2570951021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3666804126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:01.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:01 smithi121 ceph-mon[34963]: pgmap v366: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-09-12T03:20:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2570951021' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3666804126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:02.184 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T03:20:02.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T03:20:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:02.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 -- 172.21.15.19:0/3367460417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790100aa0 msgr2=0x7f279010ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 --2- 172.21.15.19:0/3367460417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790100aa0 0x7f279010ea20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ca0 tx=0x7f278802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 -- 172.21.15.19:0/3367460417 shutdown_connections 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 --2- 172.21.15.19:0/3367460417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790100aa0 0x7f279010ea20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 --2- 172.21.15.19:0/3367460417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f2790100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 --2- 172.21.15.19:0/3367460417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790106b80 0x7f2790106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 -- 172.21.15.19:0/3367460417 >> 172.21.15.19:0/3367460417 conn(0x7f2790077230 msgr2=0x7f27900fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 -- 172.21.15.19:0/3367460417 shutdown_connections 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 -- 172.21.15.19:0/3367460417 wait complete. 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.827+0000 7f2794b74640 1 Processor -- start 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 -- start start 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790100aa0 0x7f279019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2790112050 con 0x7f2790106b80 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2790111ed0 con 0x7f2790100100 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f2794b74640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27901121d0 con 0x7f2790100aa0 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:57554/0 (socket says 172.21.15.19:57554) 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33688/0 (socket says 172.21.15.19:33688) 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 -- 172.21.15.19:0/3716329344 learned_addr learned my addr 172.21.15.19:0/3716329344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278dd9b640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790100aa0 0x7f279019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 -- 172.21.15.19:0/3716329344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790100aa0 msgr2=0x7f279019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790100aa0 0x7f279019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 -- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 msgr2=0x7f279019ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278e59c640 1 -- 172.21.15.19:0/3716329344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279019f5e0 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.828+0000 7f278ed9d640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f278e59c640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f277f7fe640 1 -- 172.21.15.19:0/3716329344 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780017070 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279019f8d0 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f277f7fe640 1 -- 172.21.15.19:0/3716329344 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2780009820 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f277f7fe640 1 -- 172.21.15.19:0/3716329344 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004cd0 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.829+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27901a71b0 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.831+0000 7f277f7fe640 1 -- 172.21.15.19:0/3716329344 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27800099d0 con 0x7f2790100100 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.831+0000 7f2796f4d740 1 Processor -- start 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 -- start start 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43bb40 0x55840d43a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d43aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55840d3a02f0 con 0x55840d445370 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55840d347cf0 con 0x55840d43bb40 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f2796f4d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55840d39c2b0 con 0x55840d43a5d0 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33704/0 (socket says 172.21.15.19:33704) 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f278dd9b640 1 -- 172.21.15.19:0/4042077912 learned_addr learned my addr 172.21.15.19:0/4042077912 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f278ed9d640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43bb40 0x55840d43a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.832+0000 7f278e59c640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d43aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 -- 172.21.15.19:0/4042077912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 msgr2=0x55840d43aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d43aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 -- 172.21.15.19:0/4042077912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43bb40 msgr2=0x55840d43a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43bb40 0x55840d43a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 -- 172.21.15.19:0/4042077912 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55840d32c3a0 con 0x55840d445370 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f277f7fe640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 0x7f27780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278e59c640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d43aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278ed9d640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 0x7f27780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f277f7fe640 1 -- 172.21.15.19:0/3716329344 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27800ce9b0 con 0x7f2790100100 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f278dd9b640 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f278400aca0 tx=0x7f278400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd916ebc37ee6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f277cff9640 1 -- 172.21.15.19:0/4042077912 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278401d070 con 0x55840d445370 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f277cff9640 1 -- 172.21.15.19:0/4042077912 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2784018b20 con 0x55840d445370 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.833+0000 7f277cff9640 1 -- 172.21.15.19:0/4042077912 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784010b90 con 0x55840d445370 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 -- 172.21.15.19:0/4042077912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 msgr2=0x55840d447760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f278ed9d640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 0x7f27780a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2788005e30 tx=0x7f2788005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f278400aca0 tx=0x7f278400f790 comp rx=0 tx=0).stop 2024-09-12T03:20:02.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 -- 172.21.15.19:0/4042077912 shutdown_connections 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d445370 0x55840d447760 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d43aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 --2- 172.21.15.19:0/4042077912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43bb40 0x55840d43a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 -- 172.21.15.19:0/4042077912 >> 172.21.15.19:0/4042077912 conn(0x55840d403310 msgr2=0x55840d436c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 -- 172.21.15.19:0/4042077912 shutdown_connections 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.834+0000 7f2796f4d740 1 -- 172.21.15.19:0/4042077912 wait complete. 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 -- start start 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 Processor -- start 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43bb40 0x55840d442040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d442650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55840d448070 con 0x55840d43a5d0 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55840d35b060 con 0x55840d445370 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55840d4481f0 con 0x55840d43bb40 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f278e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f2796f4d740 10 client.0 _ll_register_callbacks cb 0x55840d0ae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f278e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33716/0 (socket says 172.21.15.19:33716) 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f278e59c640 1 -- 172.21.15.19:0/1828836999 learned_addr learned my addr 172.21.15.19:0/1828836999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43bb40 0x55840d442040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.835+0000 7f278ed9d640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d442650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 -- 172.21.15.19:0/1828836999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43bb40 msgr2=0x55840d442040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43bb40 0x55840d442040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 -- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 msgr2=0x55840d442650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d442650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 -- 172.21.15.19:0/1828836999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55840d1113c0 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f278e59c640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f278000d740 tx=0x7f278009b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a14d45d32b197ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780020070 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27800994c0 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2784018b20 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27800a3e30 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2796f4d740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55840d453ab0 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2784018b20 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.836+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f27800bb0b0 con 0x55840d43a5d0 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 10 client.15486 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:20:02.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 20 client.15486 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d454e00 0x55840d457270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 4 client.15486 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55840d3ad860 con 0x55840d454e00 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d454e00 0x55840d457270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d454e00 0x55840d457270 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=786d4494b9afd5d8 server_cookie=5a0f29534a890f9d in_seq=0 out_seq=0 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.837+0000 7f2775ffb640 10 client.15486 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2775ffb640 1 -- 172.21.15.19:0/1828836999 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55840d3ad860 con 0x55840d454e00 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2775ffb640 10 client.15486 handle_command_reply: tid=0 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 client.15486 shutdown 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 msgr2=0x55840d444d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f278000d740 tx=0x7f278009b8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 shutdown_connections 2024-09-12T03:20:02.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d442650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d454e00 0x55840d457270 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43bb40 0x55840d442040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.838+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1828836999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43a5d0 0x55840d444d90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.839+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 >> 172.21.15.19:0/1828836999 conn(0x55840d403310 msgr2=0x55840d43c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.839+0000 7f2775ffb640 0 client.15486 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.839+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 shutdown_connections 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.839+0000 7f2796f4d740 1 -- 172.21.15.19:0/1828836999 wait complete. 2024-09-12T03:20:02.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.839+0000 7f2796f4d740 20 client.15486 trim_cache size 0 max 16384 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.852+0000 7f2796f4d740 1 Processor -- start 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 -- start start 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d451310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d45cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27901af660 con 0x55840d43bb40 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27901b11a0 con 0x55840d445370 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f2796f4d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27901afd30 con 0x55840d43a5d0 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33730/0 (socket says 172.21.15.19:33730) 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 -- 172.21.15.19:0/1154300019 learned_addr learned my addr 172.21.15.19:0/1154300019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278ed9d640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d45cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278e59c640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d451310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 -- 172.21.15.19:0/1154300019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 msgr2=0x55840d451310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d451310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 -- 172.21.15.19:0/1154300019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 msgr2=0x55840d45cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d45cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278dd9b640 1 -- 172.21.15.19:0/1154300019 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55840d437960 con 0x55840d43bb40 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278ed9d640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d45cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.853+0000 7f278e59c640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d451310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.854+0000 7f278dd9b640 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f278400fc90 tx=0x7f278400a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a619c8c5b533d128 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.854+0000 7f2777fff640 1 -- 172.21.15.19:0/1154300019 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278401d040 con 0x55840d43bb40 2024-09-12T03:20:02.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.854+0000 7f2777fff640 1 -- 172.21.15.19:0/1154300019 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278402e070 con 0x55840d43bb40 2024-09-12T03:20:02.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.854+0000 7f2777fff640 1 -- 172.21.15.19:0/1154300019 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784018a10 con 0x55840d43bb40 2024-09-12T03:20:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- 172.21.15.19:0/1154300019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 msgr2=0x55840d43c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f278400fc90 tx=0x7f278400a6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- 172.21.15.19:0/1154300019 shutdown_connections 2024-09-12T03:20:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d445370 0x55840d45cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d43c820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 --2- 172.21.15.19:0/1154300019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d43a5d0 0x55840d451310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- 172.21.15.19:0/1154300019 >> 172.21.15.19:0/1154300019 conn(0x55840d403310 msgr2=0x55840d43c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- 172.21.15.19:0/1154300019 shutdown_connections 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- 172.21.15.19:0/1154300019 wait complete. 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 -- start start 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.855+0000 7f2796f4d740 1 Processor -- start 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43a5d0 0x55840d44db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d445370 0x55840d4493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27901b0ad0 con 0x55840d43bb40 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27901b0400 con 0x55840d43a5d0 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27901ad640 con 0x55840d445370 2024-09-12T03:20:02.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2796f4d740 10 client.0 _ll_register_callbacks cb 0x7f2780005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33738/0 (socket says 172.21.15.19:33738) 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 -- 172.21.15.19:0/733302014 learned_addr learned my addr 172.21.15.19:0/733302014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278e59c640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43a5d0 0x55840d44db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278ed9d640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d445370 0x55840d4493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 -- 172.21.15.19:0/733302014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d445370 msgr2=0x55840d4493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d445370 0x55840d4493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 -- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43a5d0 msgr2=0x55840d44db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43a5d0 0x55840d44db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 -- 172.21.15.19:0/733302014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55840d449bf0 con 0x55840d43bb40 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f278dd9b640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f278402d040 tx=0x7f2784005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a657688cd2832c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:02.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2775ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.856+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784037020 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2784018440 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278402e070 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f278401d070 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55840d4c82a0 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f278402e070 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2784004cb0 con 0x55840d43bb40 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 10 client.15492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 20 client.15492 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d4c8740 0x55840d4cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:02.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 4 client.15492 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.857+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55840d3a7d70 con 0x55840d4c8740 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f278e59c640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d4c8740 0x55840d4cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2775ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f278e59c640 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d4c8740 0x55840d4cab60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6507e96fc0bf389b server_cookie=e6ca95bd6132be80 in_seq=0 out_seq=0 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2775ffb640 1 -- 172.21.15.19:0/733302014 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55840d3a7d70 con 0x55840d4c8740 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2775ffb640 10 client.15492 handle_command_reply: tid=0 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 client.15492 shutdown 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 msgr2=0x55840d448e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f278402d040 tx=0x7f2784005040 comp rx=0 tx=0).stop 2024-09-12T03:20:02.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 shutdown_connections 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55840d4c8740 0x55840d4cab60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55840d445370 0x55840d4493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2775ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55840d43bb40 0x55840d448e80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 --2- 172.21.15.19:0/733302014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55840d43a5d0 0x55840d44db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 >> 172.21.15.19:0/733302014 conn(0x55840d403310 msgr2=0x55840d4354e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.858+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 shutdown_connections 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2796f4d740 1 -- 172.21.15.19:0/733302014 wait complete. 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2796f4d740 20 client.15492 trim_cache size 0 max 16384 2024-09-12T03:20:02.864 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 msgr2=0x7f27780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 0x7f27780a5040 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2788005e30 tx=0x7f2788005ef0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 msgr2=0x7f279019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:02.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f278000d770 tx=0x7f278000db70 comp rx=0 tx=0).stop 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 shutdown_connections 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27780a2b80 0x7f27780a5040 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2790106b80 0x7f279019ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2790100aa0 0x7f279019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 --2- 172.21.15.19:0/3716329344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2790100100 0x7f279019a630 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 >> 172.21.15.19:0/3716329344 conn(0x7f2790077230 msgr2=0x7f2790104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 shutdown_connections 2024-09-12T03:20:02.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:02.859+0000 7f2794b74640 1 -- 172.21.15.19:0/3716329344 wait complete. 2024-09-12T03:20:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:03 smithi135 ceph-mon[35474]: pgmap v367: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:20:03.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:03 smithi019 ceph-mon[30953]: pgmap v367: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:20:03.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:20:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:03 smithi121 ceph-mon[34963]: pgmap v367: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T03:20:03.710 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 -- 172.21.15.19:0/3940050812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 msgr2=0x7f3758101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 --2- 172.21.15.19:0/3940050812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758101e30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3744009930 tx=0x7f374402a420 comp rx=0 tx=0).stop 2024-09-12T03:20:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 -- 172.21.15.19:0/3940050812 shutdown_connections 2024-09-12T03:20:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 --2- 172.21.15.19:0/3940050812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3758102370 0x7f375810e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 --2- 172.21.15.19:0/3940050812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758101e30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 --2- 172.21.15.19:0/3940050812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37581079d0 0x7f3758107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.219+0000 7f375d9a0640 1 -- 172.21.15.19:0/3940050812 >> 172.21.15.19:0/3940050812 conn(0x7f37580fd710 msgr2=0x7f37580ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- 172.21.15.19:0/3940050812 shutdown_connections 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- 172.21.15.19:0/3940050812 wait complete. 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 Processor -- start 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- start start 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3758102370 0x7f3758198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 0x7f375819cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3758111da0 con 0x7f37581019d0 2024-09-12T03:20:04.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3758111c20 con 0x7f3758102370 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.220+0000 7f375d9a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3758111f20 con 0x7f37581079d0 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33760/0 (socket says 172.21.15.19:33760) 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 -- 172.21.15.19:0/2145865143 learned_addr learned my addr 172.21.15.19:0/2145865143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f3757fff640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3758102370 0x7f3758198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 -- 172.21.15.19:0/2145865143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 msgr2=0x7f375819cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375d19f640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 0x7f375819cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 0x7f375819cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 -- 172.21.15.19:0/2145865143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3758102370 msgr2=0x7f3758198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3758102370 0x7f3758198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 -- 172.21.15.19:0/2145865143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375819d2b0 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375d19f640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 0x7f375819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375c99e640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f374c00d950 tx=0x7f374c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f3755ffb640 1 -- 172.21.15.19:0/2145865143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c017070 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375819d5a0 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f3755ffb640 1 -- 172.21.15.19:0/2145865143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374c009c30 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.221+0000 7f3755ffb640 1 -- 172.21.15.19:0/2145865143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c00f6b0 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.222+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37581a4e80 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.224+0000 7f3755ffb640 1 -- 172.21.15.19:0/2145865143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f374c0109d0 con 0x7f37581019d0 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.224+0000 7f3760fdb740 1 Processor -- start 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3755ffb640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 0x7f37480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 -- start start 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375d19f640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 0x7f37480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3755ffb640 1 -- 172.21.15.19:0/2145865143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f374c0cee90 con 0x7f37581019d0 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b089220 0x55944b0935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b095ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55944afee2a0 con 0x55944b088900 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55944af95b10 con 0x55944b093ae0 2024-09-12T03:20:04.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3760fdb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55944afea260 con 0x55944b089220 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33772/0 (socket says 172.21.15.19:33772) 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 -- 172.21.15.19:0/2144014329 learned_addr learned my addr 172.21.15.19:0/2144014329 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f3757fff640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b089220 0x55944b0935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375d19f640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b095ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375d19f640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 0x7f37480a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3750005fd0 tx=0x7f375000f730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 -- 172.21.15.19:0/2144014329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b089220 msgr2=0x55944b0935a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b089220 0x55944b0935a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 -- 172.21.15.19:0/2144014329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 msgr2=0x55944b095ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b095ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375c99e640 1 -- 172.21.15.19:0/2144014329 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944ad5fb90 con 0x55944b088900 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.225+0000 7f375d19f640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b095ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.226+0000 7f375c99e640 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f374c097b60 tx=0x7f374c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e7256c8af9a6a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.226+0000 7f3742ffd640 1 -- 172.21.15.19:0/2144014329 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c0cce60 con 0x55944b088900 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.226+0000 7f3742ffd640 1 -- 172.21.15.19:0/2144014329 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374c09b700 con 0x55944b088900 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.226+0000 7f3742ffd640 1 -- 172.21.15.19:0/2144014329 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c020070 con 0x55944b088900 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- 172.21.15.19:0/2144014329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 msgr2=0x55944b088ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f374c097b60 tx=0x7f374c010d70 comp rx=0 tx=0).stop 2024-09-12T03:20:04.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- 172.21.15.19:0/2144014329 shutdown_connections 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b095ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b089220 0x55944b0935a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 --2- 172.21.15.19:0/2144014329 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b088900 0x55944b088ce0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- 172.21.15.19:0/2144014329 >> 172.21.15.19:0/2144014329 conn(0x55944b084e90 msgr2=0x55944b085270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- 172.21.15.19:0/2144014329 shutdown_connections 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- 172.21.15.19:0/2144014329 wait complete. 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 -- start start 2024-09-12T03:20:04.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.227+0000 7f3760fdb740 1 Processor -- start 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b088900 0x55944b090a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b093ae0 0x55944b0914a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55944b096650 con 0x55944b089220 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55944af766e0 con 0x55944b088900 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55944b0967d0 con 0x55944b093ae0 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3760fdb740 10 client.0 _ll_register_callbacks cb 0x55944af9aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33778/0 (socket says 172.21.15.19:33778) 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 -- 172.21.15.19:0/1214564605 learned_addr learned my addr 172.21.15.19:0/1214564605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:04.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f375c99e640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b088900 0x55944b090a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f375d19f640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b093ae0 0x55944b0914a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 -- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b093ae0 msgr2=0x55944b0914a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b093ae0 0x55944b0914a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 -- 172.21.15.19:0/1214564605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b088900 msgr2=0x55944b090a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b088900 0x55944b090a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 -- 172.21.15.19:0/1214564605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944b09fb90 con 0x55944b089220 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3757fff640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f374402a940 tx=0x7f3744031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd78006f4cec91fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.228+0000 7f3737fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744035040 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3744004460 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f374c09b700 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3744005020 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 10 client.15504 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55944b0a2520 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f374c09b700 con 0x55944b089220 2024-09-12T03:20:04.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f37440035f0 con 0x55944b089220 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 10 client.15504 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 20 client.15504 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b0a39e0 0x55944b0a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 4 client.15504 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.229+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55944affb9d0 con 0x55944b0a39e0 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.230+0000 7f375c99e640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b0a39e0 0x55944b0a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.230+0000 7f375c99e640 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b0a39e0 0x55944b0a5df0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875016d518d7eee9 server_cookie=d2600c3ff60bb2c8 in_seq=0 out_seq=0 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.230+0000 7f3737fff640 10 client.15504 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3737fff640 1 -- 172.21.15.19:0/1214564605 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55944affb9d0 con 0x55944b0a39e0 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3737fff640 10 client.15504 handle_command_reply: tid=0 2024-09-12T03:20:04.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 client.15504 shutdown 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 msgr2=0x55944b090f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f374402a940 tx=0x7f3744031920 comp rx=0 tx=0).stop 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 shutdown_connections 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b093ae0 0x55944b0914a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b090f60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b0a39e0 0x55944b0a5df0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 --2- 172.21.15.19:0/1214564605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b088900 0x55944b090a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 >> 172.21.15.19:0/1214564605 conn(0x55944b084e90 msgr2=0x55944b084000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3737fff640 0 client.15504 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:04.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 shutdown_connections 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 1 -- 172.21.15.19:0/1214564605 wait complete. 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.231+0000 7f3760fdb740 20 client.15504 trim_cache size 0 max 16384 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.248+0000 7f3760fdb740 1 Processor -- start 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.248+0000 7f3760fdb740 1 -- start start 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b08b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 0x55944b083ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37581ad3c0 con 0x55944b093ae0 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37581aef00 con 0x55944b089220 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3760fdb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37581ada90 con 0x55944b088900 2024-09-12T03:20:04.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33794/0 (socket says 172.21.15.19:33794) 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 -- 172.21.15.19:0/903859337 learned_addr learned my addr 172.21.15.19:0/903859337 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375c99e640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b08b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3757fff640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 0x55944b083ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 -- 172.21.15.19:0/903859337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 msgr2=0x55944b08b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b08b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 -- 172.21.15.19:0/903859337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 msgr2=0x55944b083ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 0x55944b083ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 -- 172.21.15.19:0/903859337 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55944b08ef90 con 0x55944b093ae0 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f3757fff640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 0x55944b083ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.249+0000 7f375d19f640 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37500127a0 tx=0x7f3750014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c68be954fd003b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.250+0000 7f37427fc640 1 -- 172.21.15.19:0/903859337 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002b070 con 0x55944b093ae0 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.250+0000 7f37427fc640 1 -- 172.21.15.19:0/903859337 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37500263f0 con 0x55944b093ae0 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.250+0000 7f37427fc640 1 -- 172.21.15.19:0/903859337 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375001b5f0 con 0x55944b093ae0 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.250+0000 7f3760fdb740 1 -- 172.21.15.19:0/903859337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 msgr2=0x55944b0ab6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.250+0000 7f3760fdb740 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37500127a0 tx=0x7f3750014000 comp rx=0 tx=0).stop 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 -- 172.21.15.19:0/903859337 shutdown_connections 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b093ae0 0x55944b0ab6c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b089220 0x55944b083ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 --2- 172.21.15.19:0/903859337 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b08b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 -- 172.21.15.19:0/903859337 >> 172.21.15.19:0/903859337 conn(0x55944b084e90 msgr2=0x55944b0837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 -- 172.21.15.19:0/903859337 shutdown_connections 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 -- 172.21.15.19:0/903859337 wait complete. 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 -- start start 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.251+0000 7f3760fdb740 1 Processor -- start 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b09ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b09bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b0989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37581ae830 con 0x55944b089220 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37581ae160 con 0x55944b093ae0 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37581ab3a0 con 0x55944b088900 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3760fdb740 10 client.0 _ll_register_callbacks cb 0x7f37440315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b09bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3757fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b09bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33800/0 (socket says 172.21.15.19:33800) 2024-09-12T03:20:04.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3757fff640 1 -- 172.21.15.19:0/4288140231 learned_addr learned my addr 172.21.15.19:0/4288140231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b0989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375c99e640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b09ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 -- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 msgr2=0x55944b09ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b09ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 -- 172.21.15.19:0/4288140231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 msgr2=0x55944b09bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b09bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 -- 172.21.15.19:0/4288140231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55944b099030 con 0x55944b093ae0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f375d19f640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b0989f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3750029040 tx=0x7f3750026850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39f87b77e902d773 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.252+0000 7f3737fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375002b040 con 0x55944b093ae0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3750012250 con 0x55944b093ae0 2024-09-12T03:20:04.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37500263f0 con 0x55944b093ae0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3750004a40 con 0x55944b093ae0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3760fdb740 10 client.25091 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55944b10ef30 con 0x55944b093ae0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37500263f0 con 0x55944b093ae0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f3750039030 con 0x55944b093ae0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.253+0000 7f3760fdb740 10 client.25091 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3760fdb740 10 client.25091 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3760fdb740 20 client.25091 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b10f600 0x55944b111a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3760fdb740 4 client.25091 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55944aff5d20 con 0x55944b10f600 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f375c99e640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b10f600 0x55944b111a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:04.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f375c99e640 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b10f600 0x55944b111a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98cab67d65fd610c server_cookie=ed174349f02c16b9 in_seq=0 out_seq=0 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.254+0000 7f3737fff640 10 client.25091 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3737fff640 1 -- 172.21.15.19:0/4288140231 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55944aff5d20 con 0x55944b10f600 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3737fff640 10 client.25091 handle_command_reply: tid=0 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 client.25091 shutdown 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 msgr2=0x55944b0989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b0989f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3750029040 tx=0x7f3750026850 comp rx=0 tx=0).stop 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 shutdown_connections 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55944b10f600 0x55944b111a20 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55944b093ae0 0x55944b0989f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55944b089220 0x55944b09bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 --2- 172.21.15.19:0/4288140231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55944b088900 0x55944b09ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3737fff640 0 client.25091 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 28, 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 2, 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "1a0cc1b": { 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 2, 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:20:04.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 144.5, 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:04.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 >> 172.21.15.19:0/4288140231 conn(0x55944b084e90 msgr2=0x55944b0911c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 shutdown_connections 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.266+0000 7f3760fdb740 1 -- 172.21.15.19:0/4288140231 wait complete. 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f3760fdb740 20 client.25091 trim_cache size 0 max 16384 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 msgr2=0x7f37480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 0x7f37480a50c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3750005fd0 tx=0x7f375000f730 comp rx=0 tx=0).stop 2024-09-12T03:20:04.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 msgr2=0x7f3758198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f374c00d950 tx=0x7f374c00de10 comp rx=0 tx=0).stop 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 shutdown_connections 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37581079d0 0x7f375819cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3758102370 0x7f3758198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37480a2c00 0x7f37480a50c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 --2- 172.21.15.19:0/2145865143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37581019d0 0x7f3758198300 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.267+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 >> 172.21.15.19:0/2145865143 conn(0x7f37580fd710 msgr2=0x7f37580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:04.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.268+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 shutdown_connections 2024-09-12T03:20:04.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:04.268+0000 7f375d9a0640 1 -- 172.21.15.19:0/2145865143 wait complete. 2024-09-12T03:20:04.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:05.081 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1a0cc1b', quiesce took 5.7 seconds. Will release after: 18.8 2024-09-12T03:20:05.331 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:05 smithi019 ceph-mon[30953]: pgmap v368: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-12T03:20:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:05 smithi135 ceph-mon[35474]: pgmap v368: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-12T03:20:05.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 -- 172.21.15.19:0/3771651147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c102dd0 msgr2=0x7f8c0c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 --2- 172.21.15.19:0/3771651147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c1031b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009930 tx=0x7f8c0002a420 comp rx=0 tx=0).stop 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 -- 172.21.15.19:0/3771651147 shutdown_connections 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 --2- 172.21.15.19:0/3771651147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c108300 0x7f8c0c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 --2- 172.21.15.19:0/3771651147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c103780 0x7f8c0c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 --2- 172.21.15.19:0/3771651147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c1031b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 -- 172.21.15.19:0/3771651147 >> 172.21.15.19:0/3771651147 conn(0x7f8c0c0fc7d0 msgr2=0x7f8c0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 -- 172.21.15.19:0/3771651147 shutdown_connections 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.556+0000 7f8c13119640 1 -- 172.21.15.19:0/3771651147 wait complete. 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 Processor -- start 2024-09-12T03:20:05.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 -- start start 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 0x7f8c0c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c112090 con 0x7f8c0c103780 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c111f10 con 0x7f8c0c102dd0 2024-09-12T03:20:05.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.557+0000 7f8c13119640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c112210 con 0x7f8c0c108300 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33818/0 (socket says 172.21.15.19:33818) 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 -- 172.21.15.19:0/148042719 learned_addr learned my addr 172.21.15.19:0/148042719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c116b6640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 0x7f8c0c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 -- 172.21.15.19:0/148042719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 msgr2=0x7f8c0c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 0x7f8c0c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 -- 172.21.15.19:0/148042719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 msgr2=0x7f8c0c198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 -- 172.21.15.19:0/148042719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0c19d600 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c116b6640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 0x7f8c0c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800e9f0 tx=0x7f8bf800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.558+0000 7f8c09ffb640 1 -- 172.21.15.19:0/148042719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800ccc0 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.559+0000 7f8c09ffb640 1 -- 172.21.15.19:0/148042719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf8010910 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.559+0000 7f8c09ffb640 1 -- 172.21.15.19:0/148042719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8018920 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.559+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0c19d8f0 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.559+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c0c1a51d0 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.562+0000 7f8c09ffb640 1 -- 172.21.15.19:0/148042719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8bf8010ac0 con 0x7f8c0c103780 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.562+0000 7f8c154f2740 1 Processor -- start 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 -- start start 2024-09-12T03:20:05.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 0x555d491c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d491292f0 con 0x555d491d2780 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555d490d0cf0 con 0x555d491c4b40 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c154f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d491252b0 con 0x555d491c35d0 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33834/0 (socket says 172.21.15.19:33834) 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c0bfff640 1 -- 172.21.15.19:0/1628857698 learned_addr learned my addr 172.21.15.19:0/1628857698 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c09ffb640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 0x7f8bfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 0x555d491c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c116b6640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.563+0000 7f8c09ffb640 1 -- 172.21.15.19:0/148042719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8bf80cf480 con 0x7f8c0c103780 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 -- 172.21.15.19:0/1628857698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 msgr2=0x555d491c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 -- 172.21.15.19:0/1628857698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 msgr2=0x555d491d4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c116b6640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 0x7f8bfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 -- 172.21.15.19:0/1628857698 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d490b53a0 con 0x555d491c35d0 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 0x555d491c3a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009900 tx=0x7f8c00005e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4030cce5872c927 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8c116b6640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 0x7f8bfc0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005fd0 tx=0x7f8c04004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8bf6ffd640 1 -- 172.21.15.19:0/1628857698 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003f070 con 0x555d491c35d0 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8bf6ffd640 1 -- 172.21.15.19:0/1628857698 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00004030 con 0x555d491c35d0 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.564+0000 7f8bf6ffd640 1 -- 172.21.15.19:0/1628857698 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003f070 con 0x555d491c35d0 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 -- 172.21.15.19:0/1628857698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 msgr2=0x555d491c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 0x555d491c3a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009900 tx=0x7f8c00005e60 comp rx=0 tx=0).stop 2024-09-12T03:20:05.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 -- 172.21.15.19:0/1628857698 shutdown_connections 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c35d0 0x555d491c3a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 --2- 172.21.15.19:0/1628857698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 -- 172.21.15.19:0/1628857698 >> 172.21.15.19:0/1628857698 conn(0x555d4918c310 msgr2=0x555d491bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 -- 172.21.15.19:0/1628857698 shutdown_connections 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.565+0000 7f8c154f2740 1 -- 172.21.15.19:0/1628857698 wait complete. 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 -- start start 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 Processor -- start 2024-09-12T03:20:05.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491cf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555d491d5420 con 0x555d491c35d0 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555d490e4060 con 0x555d491c4b40 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c154f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555d491d55a0 con 0x555d491d2780 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.566+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33848/0 (socket says 172.21.15.19:33848) 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2471961788 learned_addr learned my addr 172.21.15.19:0/2471961788 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:05.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c154f2740 10 client.0 _ll_register_callbacks cb 0x555d48e378c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c116b6640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491cf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 msgr2=0x555d491cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2471961788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 msgr2=0x555d491cf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491cf400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2471961788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d48e9a3c0 con 0x555d491c35d0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c00032fd0 tx=0x7f8c00004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8af2588c2d9635b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003f070 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c00032db0 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.567+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c00004030 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c00035070 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 10 client.15522 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d491e0e70 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00004030 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c000046c0 con 0x555d491c35d0 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 10 client.15522 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 20 client.15522 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:05.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d491e21c0 0x555d491e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 4 client.15522 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.568+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d49136860 con 0x555d491e21c0 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.569+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d491e21c0 0x555d491e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.569+0000 7f8bebfff640 10 client.15522 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.569+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d491e21c0 0x555d491e45b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d869c5edaedf6f46 server_cookie=39fd5fbceb3c226a in_seq=0 out_seq=0 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8bebfff640 1 -- 172.21.15.19:0/2471961788 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d49136860 con 0x555d491e21c0 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8bebfff640 10 client.15522 handle_command_reply: tid=0 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 client.15522 shutdown 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 msgr2=0x555d491d2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8c00032fd0 tx=0x7f8c00004410 comp rx=0 tx=0).stop 2024-09-12T03:20:05.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 shutdown_connections 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d491e21c0 0x555d491e45b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8bebfff640 0 client.15522 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491cf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2471961788 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491d2150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 >> 172.21.15.19:0/2471961788 conn(0x555d4918c310 msgr2=0x555d491c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 shutdown_connections 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 1 -- 172.21.15.19:0/2471961788 wait complete. 2024-09-12T03:20:05.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.570+0000 7f8c154f2740 20 client.15522 trim_cache size 0 max 16384 2024-09-12T03:20:05.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.597+0000 7f8c154f2740 1 Processor -- start 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.597+0000 7f8c154f2740 1 -- start start 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.597+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.597+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c154f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1ab350 con 0x555d491c35d0 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c154f2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1ace90 con 0x555d491c4b40 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c154f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1aba20 con 0x555d491d2780 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33858/0 (socket says 172.21.15.19:33858) 2024-09-12T03:20:05.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 -- 172.21.15.19:0/18757100 learned_addr learned my addr 172.21.15.19:0/18757100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 -- 172.21.15.19:0/18757100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 msgr2=0x555d491e9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 -- 172.21.15.19:0/18757100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 msgr2=0x555d491bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 -- 172.21.15.19:0/18757100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d491c6120 con 0x555d491c35d0 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.598+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002a940 tx=0x7f8c000051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47962db78acf6d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.599+0000 7f8bf67fc640 1 -- 172.21.15.19:0/18757100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00047070 con 0x555d491c35d0 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.599+0000 7f8bf67fc640 1 -- 172.21.15.19:0/18757100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00054030 con 0x555d491c35d0 2024-09-12T03:20:05.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.599+0000 7f8bf67fc640 1 -- 172.21.15.19:0/18757100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0003f070 con 0x555d491c35d0 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- 172.21.15.19:0/18757100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 msgr2=0x555d491de6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002a940 tx=0x7f8c000051c0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- 172.21.15.19:0/18757100 shutdown_connections 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491d2780 0x555d491e9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c4b40 0x555d491bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 --2- 172.21.15.19:0/18757100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491c35d0 0x555d491de6c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- 172.21.15.19:0/18757100 >> 172.21.15.19:0/18757100 conn(0x555d4918c310 msgr2=0x555d491be4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- 172.21.15.19:0/18757100 shutdown_connections 2024-09-12T03:20:05.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- 172.21.15.19:0/18757100 wait complete. 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.600+0000 7f8c154f2740 1 -- start start 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 Processor -- start 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 0x555d491d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1ac7c0 con 0x555d491d2780 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1ac0f0 con 0x555d491c35d0 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c0c1a9350 con 0x555d491c4b40 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.601+0000 7f8c154f2740 10 client.0 _ll_register_callbacks cb 0x7f8bf8097fb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:57672/0 (socket says 172.21.15.19:57672) 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2314415754 learned_addr learned my addr 172.21.15.19:0/2314415754 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 0x555d491d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 msgr2=0x555d491d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 0x555d491d60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2314415754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 msgr2=0x555d491d6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 -- 172.21.15.19:0/2314415754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d491d6c70 con 0x555d491c35d0 2024-09-12T03:20:05.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 0x555d491d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:05.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8c10eb5640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c00004bd0 tx=0x7f8c00005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc495f4cf8fcc195 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.602+0000 7f8bebfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c00047070 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c00054030 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c00032d30 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f8c0003f040 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 10 client.25109 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555d4924c9d0 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c00054030 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f8c00067820 con 0x555d491c35d0 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 10 client.25109 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 10 client.25109 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:20:05.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 20 client.25109 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d4924cf80 0x555d4924f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 4 client.25109 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.603+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d49130d70 con 0x555d4924cf80 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.604+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d4924cf80 0x555d4924f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.604+0000 7f8c0bfff640 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d4924cf80 0x555d4924f390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8314135f29e45bc server_cookie=c0bd0e647506ab18 in_seq=0 out_seq=0 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.604+0000 7f8bebfff640 10 client.25109 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8bebfff640 1 -- 172.21.15.19:0/2314415754 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d49130d70 con 0x555d4924cf80 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8bebfff640 10 client.25109 handle_command_reply: tid=0 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 client.25109 shutdown 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 msgr2=0x555d491dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8c00004bd0 tx=0x7f8c00005e80 comp rx=0 tx=0).stop 2024-09-12T03:20:05.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 shutdown_connections 2024-09-12T03:20:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555d4924cf80 0x555d4924f390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8bebfff640 0 client.25109 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555d491d2780 0x555d491d6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555d491c4b40 0x555d491d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.611 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 --2- 172.21.15.19:0/2314415754 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555d491c35d0 0x555d491dadc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 >> 172.21.15.19:0/2314415754 conn(0x555d4918c310 msgr2=0x555d491bed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 shutdown_connections 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 1 -- 172.21.15.19:0/2314415754 wait complete. 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.605+0000 7f8c154f2740 20 client.25109 trim_cache size 0 max 16384 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.606+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 msgr2=0x7f8bfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.606+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 0x7f8bfc0a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005fd0 tx=0x7f8c04004ab0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.606+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 msgr2=0x7f8c0c198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.606+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800e9f0 tx=0x7f8bf800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 shutdown_connections 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c0c108300 0x7f8c0c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bfc0a2b80 0x7f8bfc0a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c0c103780 0x7f8c0c198b90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 --2- 172.21.15.19:0/148042719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c0c102dd0 0x7f8c0c198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.607+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 >> 172.21.15.19:0/148042719 conn(0x7f8c0c0fc7d0 msgr2=0x7f8c0c105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.608+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 shutdown_connections 2024-09-12T03:20:05.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:05.608+0000 7f8c13119640 1 -- 172.21.15.19:0/148042719 wait complete. 2024-09-12T03:20:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:05 smithi121 ceph-mon[34963]: pgmap v368: 97 pgs: 97 active+clean; 654 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 10 op/s 2024-09-12T03:20:06.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:20:06.768 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.283+0000 7f7baa3b3640 1 -- 172.21.15.19:0/1229375111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4102db0 msgr2=0x7f7ba4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.283+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/1229375111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4102db0 0x7f7ba4103190 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009930 tx=0x7f7b9402a420 comp rx=0 tx=0).stop 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 -- 172.21.15.19:0/1229375111 shutdown_connections 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/1229375111 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba41082e0 0x7f7ba410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/1229375111 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba4103760 0x7f7ba4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/1229375111 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4102db0 0x7f7ba4103190 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 -- 172.21.15.19:0/1229375111 >> 172.21.15.19:0/1229375111 conn(0x7f7ba40fc7d0 msgr2=0x7f7ba40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 -- 172.21.15.19:0/1229375111 shutdown_connections 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 -- 172.21.15.19:0/1229375111 wait complete. 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.284+0000 7f7baa3b3640 1 Processor -- start 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 -- start start 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba4102db0 0x7f7ba4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba41082e0 0x7f7ba419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4112090 con 0x7f7ba4103760 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4111f10 con 0x7f7ba4102db0 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7baa3b3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba4112210 con 0x7f7ba41082e0 2024-09-12T03:20:07.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7ba37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7ba37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33886/0 (socket says 172.21.15.19:33886) 2024-09-12T03:20:07.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.285+0000 7f7ba37fe640 1 -- 172.21.15.19:0/2769516288 learned_addr learned my addr 172.21.15.19:0/2769516288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:07.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba4102db0 0x7f7ba4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba8950640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba41082e0 0x7f7ba419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 -- 172.21.15.19:0/2769516288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba41082e0 msgr2=0x7f7ba419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba41082e0 0x7f7ba419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 -- 172.21.15.19:0/2769516288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba4102db0 msgr2=0x7f7ba4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba4102db0 0x7f7ba4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 -- 172.21.15.19:0/2769516288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba419d4e0 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c8c0 tx=0x7f7b9800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba17fa640 1 -- 172.21.15.19:0/2769516288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98007bb0 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba17fa640 1 -- 172.21.15.19:0/2769516288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b98007d50 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7ba17fa640 1 -- 172.21.15.19:0/2769516288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b98018830 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba419d7d0 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.286+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba41a50b0 con 0x7f7ba4103760 2024-09-12T03:20:07.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7ba17fa640 1 -- 172.21.15.19:0/2769516288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7b98018a10 con 0x7f7ba4103760 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 Processor -- start 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 -- start start 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7dab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dbd72f2f0 con 0x563dbd7c95d0 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563dbd6d6cf0 con 0x563dbd7cab40 2024-09-12T03:20:07.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7bac78c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563dbd72b2b0 con 0x563dbd7d8780 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.290+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33896/0 (socket says 172.21.15.19:33896) 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 -- 172.21.15.19:0/1539328520 learned_addr learned my addr 172.21.15.19:0/1539328520 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba8950640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7dab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba17fa640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 0x7f7b8c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 -- 172.21.15.19:0/1539328520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 msgr2=0x563dbd7dab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7dab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 -- 172.21.15.19:0/1539328520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 msgr2=0x563dbd7c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 -- 172.21.15.19:0/1539328520 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dbd6bb3a0 con 0x563dbd7c95d0 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba8950640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:07.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba8950640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 0x7f7b8c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba17fa640 1 -- 172.21.15.19:0/2769516288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7b980cf210 con 0x7f7ba4103760 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.291+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b940314e0 tx=0x7f7b94004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ea0cee98477c4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7b8a7fc640 1 -- 172.21.15.19:0/1539328520 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94035030 con 0x563dbd7c95d0 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7b8a7fc640 1 -- 172.21.15.19:0/1539328520 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94003ad0 con 0x563dbd7c95d0 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7b8a7fc640 1 -- 172.21.15.19:0/1539328520 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94031c60 con 0x563dbd7c95d0 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7bac78c740 1 -- 172.21.15.19:0/1539328520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 msgr2=0x563dbd7c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f7b940314e0 tx=0x7f7b94004170 comp rx=0 tx=0).stop 2024-09-12T03:20:07.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.292+0000 7f7ba8950640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 0x7f7b8c0a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0113e0 tx=0x7f7b9c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 -- 172.21.15.19:0/1539328520 shutdown_connections 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7dab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7c95d0 0x563dbd7c9a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1539328520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 -- 172.21.15.19:0/1539328520 >> 172.21.15.19:0/1539328520 conn(0x563dbd792310 msgr2=0x563dbd7c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 -- 172.21.15.19:0/1539328520 shutdown_connections 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 -- 172.21.15.19:0/1539328520 wait complete. 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 -- start start 2024-09-12T03:20:07.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.293+0000 7f7bac78c740 1 Processor -- start 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7d59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563dbd7db420 con 0x563dbd7cab40 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563dbd6ea060 con 0x563dbd7c95d0 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563dbd7db5a0 con 0x563dbd7d8780 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7bac78c740 10 client.0 _ll_register_callbacks cb 0x563dbd43d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:57702/0 (socket says 172.21.15.19:57702) 2024-09-12T03:20:07.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 -- 172.21.15.19:0/2162398154 learned_addr learned my addr 172.21.15.19:0/2162398154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba8950640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7d59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 -- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 msgr2=0x563dbd7d59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7d59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 -- 172.21.15.19:0/2162398154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 msgr2=0x563dbd7d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba3fff640 1 -- 172.21.15.19:0/2162398154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dbd4a03c0 con 0x563dbd7c95d0 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.294+0000 7f7ba8950640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:07.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b94000c00 tx=0x7f7b940315c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237b4bbb59653cbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94035070 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7b940037f0 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b94003ad0 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b94049070 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7bac78c740 10 client.25118 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563dbd7e6e90 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94003e90 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.295+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b9404ec10 con 0x563dbd7c95d0 2024-09-12T03:20:07.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 10 client.25118 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 10 client.25118 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 20 client.25118 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd7e81e0 0x563dbd7ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 4 client.25118 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7b94003ad0 con 0x563dbd7e81e0 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd7e81e0 0x563dbd7ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7b837fe640 10 client.25118 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.296+0000 7f7ba37fe640 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd7e81e0 0x563dbd7ea5d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88b11c27249c7007 server_cookie=98a7ac286d92e556 in_seq=0 out_seq=0 2024-09-12T03:20:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.297+0000 7f7b837fe640 1 -- 172.21.15.19:0/2162398154 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f7b94003ad0 con 0x563dbd7e81e0 2024-09-12T03:20:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.297+0000 7f7b837fe640 10 client.25118 handle_command_reply: tid=0 2024-09-12T03:20:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.297+0000 7f7bac78c740 1 client.25118 shutdown 2024-09-12T03:20:07.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 msgr2=0x563dbd7d8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7b94000c00 tx=0x7f7b940315c0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 shutdown_connections 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd7e81e0 0x563dbd7ea5d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7b837fe640 0 client.25118 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7d59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 --2- 172.21.15.19:0/2162398154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7d8150 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 >> 172.21.15.19:0/2162398154 conn(0x563dbd792310 msgr2=0x563dbd7cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 shutdown_connections 2024-09-12T03:20:07.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 1 -- 172.21.15.19:0/2162398154 wait complete. 2024-09-12T03:20:07.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.298+0000 7f7bac78c740 20 client.25118 trim_cache size 0 max 16384 2024-09-12T03:20:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 Processor -- start 2024-09-12T03:20:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 -- start start 2024-09-12T03:20:07.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7efb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41ab2e0 con 0x563dbd7cab40 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41ace20 con 0x563dbd7c95d0 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.311+0000 7f7bac78c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41ab9b0 con 0x563dbd7d8780 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33916/0 (socket says 172.21.15.19:33916) 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba37fe640 1 -- 172.21.15.19:0/3913300412 learned_addr learned my addr 172.21.15.19:0/3913300412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba8950640 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7efb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 -- 172.21.15.19:0/3913300412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 msgr2=0x563dbd7efb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7efb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 -- 172.21.15.19:0/3913300412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 msgr2=0x563dbd7d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 -- 172.21.15.19:0/3913300412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563dbd7d84c0 con 0x563dbd7c95d0 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.312+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7e46c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402a940 tx=0x7f7b94004170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4fe21d79cca79c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7b89ffb640 1 -- 172.21.15.19:0/3913300412 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9405d030 con 0x563dbd7c95d0 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7b89ffb640 1 -- 172.21.15.19:0/3913300412 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b94031630 con 0x563dbd7c95d0 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7b89ffb640 1 -- 172.21.15.19:0/3913300412 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94035040 con 0x563dbd7c95d0 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 -- 172.21.15.19:0/3913300412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 msgr2=0x563dbd7e46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7e46c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7b9402a940 tx=0x7f7b94004170 comp rx=0 tx=0).stop 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 -- 172.21.15.19:0/3913300412 shutdown_connections 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7d8780 0x563dbd7efb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7cab40 0x563dbd7d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 --2- 172.21.15.19:0/3913300412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7c95d0 0x563dbd7e46c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.313+0000 7f7bac78c740 1 -- 172.21.15.19:0/3913300412 >> 172.21.15.19:0/3913300412 conn(0x563dbd792310 msgr2=0x563dbd7d3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.314+0000 7f7bac78c740 1 -- 172.21.15.19:0/3913300412 shutdown_connections 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.314+0000 7f7bac78c740 1 -- 172.21.15.19:0/3913300412 wait complete. 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.314+0000 7f7bac78c740 1 -- start start 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.314+0000 7f7bac78c740 1 Processor -- start 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 0x563dbd7d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41ac750 con 0x563dbd7d8780 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41ac080 con 0x563dbd7cab40 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba41a92c0 con 0x563dbd7c95d0 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7bac78c740 10 client.0 _ll_register_callbacks cb 0x7f7b9c008410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:07.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 0x563dbd7d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:52454/0 (socket says 172.21.15.19:52454) 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba3fff640 1 -- 172.21.15.19:0/1214468831 learned_addr learned my addr 172.21.15.19:0/1214468831 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 -- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 msgr2=0x563dbd7d2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 -- 172.21.15.19:0/1214468831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 msgr2=0x563dbd7d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba8950640 1 -- 172.21.15.19:0/1214468831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563dbd7ea940 con 0x563dbd7d8780 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.315+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7ba8950640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 0x563dbd7d2df0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c012ee0 tx=0x7f7b9c014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee4bae8d9c6a710 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7b837fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:07.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c02d060 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7b9c028680 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b94031630 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c028880 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7bac78c740 10 client.15543 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563dbd852bf0 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.316+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b9c016070 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7b9c0333d0 con 0x563dbd7d8780 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 10 client.15543 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 10 client.15543 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 20 client.15543 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd853370 0x563dbd855790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:07.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 4 client.15543 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7b94031630 con 0x563dbd853370 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd853370 0x563dbd855790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7b837fe640 10 client.15543 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7ba3fff640 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd853370 0x563dbd855790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=672e38a07a5831fa server_cookie=96f940d5b03ca644 in_seq=0 out_seq=0 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7b837fe640 1 -- 172.21.15.19:0/1214468831 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7b94031630 con 0x563dbd853370 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7b837fe640 10 client.15543 handle_command_reply: tid=0 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.317+0000 7f7bac78c740 1 client.15543 shutdown 2024-09-12T03:20:07.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 msgr2=0x563dbd7d2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 0x563dbd7d2df0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c012ee0 tx=0x7f7b9c014070 comp rx=0 tx=0).stop 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 shutdown_connections 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563dbd853370 0x563dbd855790 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563dbd7d8780 0x563dbd7d2df0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563dbd7cab40 0x563dbd7d28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 --2- 172.21.15.19:0/1214468831 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563dbd7c95d0 0x563dbd7d2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 >> 172.21.15.19:0/1214468831 conn(0x563dbd792310 msgr2=0x563dbd7d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7b837fe640 0 client.15543 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 shutdown_connections 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 1 -- 172.21.15.19:0/1214468831 wait complete. 2024-09-12T03:20:07.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7bac78c740 20 client.15543 trim_cache size 0 max 16384 2024-09-12T03:20:07.327 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:07.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 msgr2=0x7f7b8c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 0x7f7b8c0a5040 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0113e0 tx=0x7f7b9c004a40 comp rx=0 tx=0).stop 2024-09-12T03:20:07.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 msgr2=0x7f7ba4198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.318+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c8c0 tx=0x7f7b9800cd80 comp rx=0 tx=0).stop 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 shutdown_connections 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ba41082e0 0x7f7ba419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7b8c0a2b80 0x7f7b8c0a5040 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ba4103760 0x7f7ba4198a70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 --2- 172.21.15.19:0/2769516288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ba4102db0 0x7f7ba4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 >> 172.21.15.19:0/2769516288 conn(0x7f7ba40fc7d0 msgr2=0x7f7ba4106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:07.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 shutdown_connections 2024-09-12T03:20:07.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:07.319+0000 7f7baa3b3640 1 -- 172.21.15.19:0/2769516288 wait complete. 2024-09-12T03:20:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:07 smithi135 ceph-mon[35474]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:07.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:07 smithi019 ceph-mon[30953]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:07.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:07 smithi121 ceph-mon[34963]: pgmap v369: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:08.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:20:08.532 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 -- 172.21.15.19:0/4253819419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70741082a0 msgr2=0x7f707410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 --2- 172.21.15.19:0/4253819419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70741082a0 0x7f707410ec40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f706c009cd0 tx=0x7f706c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 -- 172.21.15.19:0/4253819419 shutdown_connections 2024-09-12T03:20:09.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 --2- 172.21.15.19:0/4253819419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70741082a0 0x7f707410ec40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 --2- 172.21.15.19:0/4253819419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 --2- 172.21.15.19:0/4253819419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7074102d30 0x7f7074103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.086+0000 7f7078b18640 1 -- 172.21.15.19:0/4253819419 >> 172.21.15.19:0/4253819419 conn(0x7f70740fc7d0 msgr2=0x7f70740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.087+0000 7f7078b18640 1 -- 172.21.15.19:0/4253819419 shutdown_connections 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.087+0000 7f7078b18640 1 -- 172.21.15.19:0/4253819419 wait complete. 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.087+0000 7f7078b18640 1 Processor -- start 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 -- start start 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70741082a0 0x7f707419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70741120a0 con 0x7f7074102d30 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7074111f20 con 0x7f70741082a0 2024-09-12T03:20:09.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7078b18640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7074112220 con 0x7f70741036e0 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33928/0 (socket says 172.21.15.19:33928) 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 -- 172.21.15.19:0/4118147557 learned_addr learned my addr 172.21.15.19:0/4118147557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7072d9d640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70741082a0 0x7f707419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 -- 172.21.15.19:0/4118147557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 msgr2=0x7f7074198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7071d9b640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 -- 172.21.15.19:0/4118147557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70741082a0 msgr2=0x7f707419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70741082a0 0x7f707419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f707259c640 1 -- 172.21.15.19:0/4118147557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f707419d510 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.088+0000 7f7071d9b640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f707259c640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f706400d950 tx=0x7f706400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f70637fe640 1 -- 172.21.15.19:0/4118147557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064017070 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f707419d800 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70741a50e0 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f70637fe640 1 -- 172.21.15.19:0/4118147557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7064009c30 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.089+0000 7f70637fe640 1 -- 172.21.15.19:0/4118147557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706400f6b0 con 0x7f7074102d30 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 Processor -- start 2024-09-12T03:20:09.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 -- start start 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 0x55e88bab1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e88ba172f0 con 0x55e88babc370 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e88b9becf0 con 0x55e88bab2b40 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f707aef1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e88ba132b0 con 0x55e88bab15d0 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.092+0000 7f7071d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33934/0 (socket says 172.21.15.19:33934) 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 -- 172.21.15.19:0/2340177542 learned_addr learned my addr 172.21.15.19:0/2340177542 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7072d9d640 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 0x55e88bab1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 -- 172.21.15.19:0/2340177542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 msgr2=0x55e88bab1a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:20:09.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab1a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 -- 172.21.15.19:0/2340177542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 msgr2=0x55e88bab1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 0x55e88bab1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 -- 172.21.15.19:0/2340177542 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88b9a33a0 con 0x55e88babc370 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f70637fe640 1 -- 172.21.15.19:0/4118147557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f706400f850 con 0x7f7074102d30 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7072d9d640 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 0x55e88bab1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7071d9b640 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7068009b30 tx=0x7f706800e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b64e69f4d300f6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7060ff9640 1 -- 172.21.15.19:0/2340177542 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068016070 con 0x55e88babc370 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7060ff9640 1 -- 172.21.15.19:0/2340177542 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f706800ed20 con 0x55e88babc370 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.093+0000 7f7060ff9640 1 -- 172.21.15.19:0/2340177542 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70680107c0 con 0x55e88babc370 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 -- 172.21.15.19:0/2340177542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 msgr2=0x55e88babe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7068009b30 tx=0x7f706800e2f0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 -- 172.21.15.19:0/2340177542 shutdown_connections 2024-09-12T03:20:09.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88babc370 0x55e88babe760 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 --2- 172.21.15.19:0/2340177542 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bab2b40 0x55e88bab1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f707aef1740 1 -- 172.21.15.19:0/2340177542 >> 172.21.15.19:0/2340177542 conn(0x55e88ba7a310 msgr2=0x55e88baadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.094+0000 7f70637fe640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 0x7f705c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f707aef1740 1 -- 172.21.15.19:0/2340177542 shutdown_connections 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f70637fe640 1 -- 172.21.15.19:0/4118147557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70640cef80 con 0x7f7074102d30 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f7071d9b640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 0x7f705c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f707aef1740 1 -- 172.21.15.19:0/2340177542 wait complete. 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f707aef1740 1 -- start start 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f7071d9b640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 0x7f705c0a5140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7068011d80 tx=0x7f706801d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.095+0000 7f707aef1740 1 Processor -- start 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab2b40 0x55e88babbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88bab4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e88bac3480 con 0x55e88bab15d0 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e88b9d2060 con 0x55e88babc370 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e88bac3600 con 0x55e88bab2b40 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707aef1740 10 client.0 _ll_register_callbacks cb 0x55e88b7258c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33942/0 (socket says 172.21.15.19:33942) 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 -- 172.21.15.19:0/129788935 learned_addr learned my addr 172.21.15.19:0/129788935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f7071d9b640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab2b40 0x55e88babbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f7072d9d640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88bab4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 -- 172.21.15.19:0/129788935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab2b40 msgr2=0x55e88babbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab2b40 0x55e88babbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 -- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 msgr2=0x55e88bab4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88bab4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f707259c640 1 -- 172.21.15.19:0/129788935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88b7883c0 con 0x55e88bab15d0 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.096+0000 7f7072d9d640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88bab4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:09.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f707259c640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f706409b490 tx=0x7f706409be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8a3e2edbc86897 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7064020070 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7064097e00 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f706800ed20 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f706409fb40 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f707aef1740 10 client.15555 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e88bad3320 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f706800ed20 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.097+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f70640b1ce0 con 0x55e88bab15d0 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 10 client.15555 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:20:09.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 20 client.15555 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bad4670 0x55e88bad6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 4 client.15555 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e88ba24860 con 0x55e88bad4670 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f7071d9b640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bad4670 0x55e88bad6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.098+0000 7f7071d9b640 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bad4670 0x55e88bad6a60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c8e71dd1d714319f server_cookie=d22ec9a332b81a33 in_seq=0 out_seq=0 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f7059ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f7059ffb640 1 -- 172.21.15.19:0/129788935 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e88ba24860 con 0x55e88bad4670 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f7059ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f707aef1740 1 client.15555 shutdown 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 msgr2=0x55e88babb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.099+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f706409b490 tx=0x7f706409be90 comp rx=0 tx=0).stop 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 shutdown_connections 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bad4670 0x55e88bad6a60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f7059ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88bab4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab2b40 0x55e88babbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 --2- 172.21.15.19:0/129788935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88babb8c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 >> 172.21.15.19:0/129788935 conn(0x55e88ba7a310 msgr2=0x55e88bab3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 shutdown_connections 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 1 -- 172.21.15.19:0/129788935 wait complete. 2024-09-12T03:20:09.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.100+0000 7f707aef1740 20 client.15555 trim_cache size 0 max 16384 2024-09-12T03:20:09.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 Processor -- start 2024-09-12T03:20:09.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 -- start start 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88badc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f705400e310 con 0x55e88bab2b40 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f705400fe50 con 0x55e88babc370 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707aef1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f705400e9e0 con 0x55e88bab15d0 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33958/0 (socket says 172.21.15.19:33958) 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 -- 172.21.15.19:0/579817619 learned_addr learned my addr 172.21.15.19:0/579817619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f707259c640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7072d9d640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88badc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 -- 172.21.15.19:0/579817619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 msgr2=0x55e88bab8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.119+0000 7f7071d9b640 1 -- 172.21.15.19:0/579817619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 msgr2=0x55e88badc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f7071d9b640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88badc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f7071d9b640 1 -- 172.21.15.19:0/579817619 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e88bac29a0 con 0x55e88bab2b40 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f7072d9d640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88badc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f7071d9b640 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7068005c90 tx=0x7f7068023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fc4951527347ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f705bfff640 1 -- 172.21.15.19:0/579817619 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068036070 con 0x55e88bab2b40 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f705bfff640 1 -- 172.21.15.19:0/579817619 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7068004b00 con 0x55e88bab2b40 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.120+0000 7f705bfff640 1 -- 172.21.15.19:0/579817619 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706801b9a0 con 0x55e88bab2b40 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 -- 172.21.15.19:0/579817619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 msgr2=0x55e88badc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7068005c90 tx=0x7f7068023000 comp rx=0 tx=0).stop 2024-09-12T03:20:09.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 -- 172.21.15.19:0/579817619 shutdown_connections 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88babc370 0x55e88badc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab2b40 0x55e88badc310 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 --2- 172.21.15.19:0/579817619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88bab15d0 0x55e88bab8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 -- 172.21.15.19:0/579817619 >> 172.21.15.19:0/579817619 conn(0x55e88ba7a310 msgr2=0x55e88bab83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.121+0000 7f707aef1740 1 -- 172.21.15.19:0/579817619 shutdown_connections 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.122+0000 7f707aef1740 1 -- 172.21.15.19:0/579817619 wait complete. 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.122+0000 7f707aef1740 1 -- start start 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.122+0000 7f707aef1740 1 Processor -- start 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88babc370 0x55e88bacd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33972/0 (socket says 172.21.15.19:33972) 2024-09-12T03:20:09.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707259c640 1 -- 172.21.15.19:0/3236673347 learned_addr learned my addr 172.21.15.19:0/3236673347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bad4670 0x55e88bacdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f705400f780 con 0x55e88bab15d0 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f7071d9b640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88babc370 0x55e88bacd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f705400f0b0 con 0x55e88bad4670 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f705400c2f0 con 0x55e88babc370 2024-09-12T03:20:09.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.123+0000 7f7072d9d640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bad4670 0x55e88bacdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707aef1740 10 client.0 _ll_register_callbacks cb 0x7f706409b2d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 -- 172.21.15.19:0/3236673347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88babc370 msgr2=0x55e88bacd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88babc370 0x55e88bacd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 -- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bad4670 msgr2=0x55e88bacdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bad4670 0x55e88bacdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e88bb3e530 con 0x55e88bab15d0 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707259c640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7064010e50 tx=0x7f70640a41a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240302e1f37d8db server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:09.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f7059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7068004b00 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f7059ffb640 1 -- 172.21.15.19:0/3236673347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70640b9060 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f7059ffb640 1 -- 172.21.15.19:0/3236673347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70640974e0 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f7059ffb640 1 -- 172.21.15.19:0/3236673347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7064015070 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707aef1740 10 client.15561 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.124+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55e88bb3f5f0 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f7059ffb640 1 -- 172.21.15.19:0/3236673347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f706409cac0 con 0x55e88bab15d0 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 10 client.15561 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 10 client.15561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 20 client.15561 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bb3fbb0 0x55e88bb41fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 4 client.15561 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:09.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e88ba1ed70 con 0x55e88bb3fbb0 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.125+0000 7f7071d9b640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bb3fbb0 0x55e88bb41fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f7071d9b640 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bb3fbb0 0x55e88bb41fe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9feebe4d6697d3c server_cookie=9f0e8159e664e0b6 in_seq=0 out_seq=0 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f7059ffb640 10 client.15561 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f7059ffb640 1 -- 172.21.15.19:0/3236673347 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e88ba1ed70 con 0x55e88bb3fbb0 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f7059ffb640 10 client.15561 handle_command_reply: tid=0 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 client.15561 shutdown 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 msgr2=0x55e88bacd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7064010e50 tx=0x7f70640a41a0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 shutdown_connections 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e88bb3fbb0 0x55e88bb41fe0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e88bad4670 0x55e88bacdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.126+0000 7f7059ffb640 0 client.15561 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e88babc370 0x55e88bacd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 1 --2- 172.21.15.19:0/3236673347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e88bab15d0 0x55e88bacd220 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 >> 172.21.15.19:0/3236673347 conn(0x55e88ba7a310 msgr2=0x55e88babef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 shutdown_connections 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 1 -- 172.21.15.19:0/3236673347 wait complete. 2024-09-12T03:20:09.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.127+0000 7f707aef1740 20 client.15561 trim_cache size 0 max 16384 2024-09-12T03:20:09.136 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:09.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 msgr2=0x7f705c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 0x7f705c0a5140 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7068011d80 tx=0x7f706801d040 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 msgr2=0x7f7074198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f706400d950 tx=0x7f706400de10 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 shutdown_connections 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70741082a0 0x7f707419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70741036e0 0x7f7074198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f705c0a2c80 0x7f705c0a5140 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 --2- 172.21.15.19:0/4118147557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7074102d30 0x7f7074198560 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:09.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 >> 172.21.15.19:0/4118147557 conn(0x7f70740fc7d0 msgr2=0x7f7074105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:09.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 shutdown_connections 2024-09-12T03:20:09.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:09.128+0000 7f7078b18640 1 -- 172.21.15.19:0/4118147557 wait complete. 2024-09-12T03:20:09.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:09 smithi019 ceph-mon[30953]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:09.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:09 smithi135 ceph-mon[35474]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:09.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:09 smithi121 ceph-mon[34963]: pgmap v370: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:09.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:20:10.274 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 -- 172.21.15.19:0/804961244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 msgr2=0x7f0d2010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/804961244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2010ec40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009cd0 tx=0x7f0d1802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 -- 172.21.15.19:0/804961244 shutdown_connections 2024-09-12T03:20:10.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/804961244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2010ec40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/804961244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d201036e0 0x7f0d20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/804961244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d20102d30 0x7f0d20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 -- 172.21.15.19:0/804961244 >> 172.21.15.19:0/804961244 conn(0x7f0d200fc7d0 msgr2=0x7f0d200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 -- 172.21.15.19:0/804961244 shutdown_connections 2024-09-12T03:20:10.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.787+0000 7f0d27cd2640 1 -- 172.21.15.19:0/804961244 wait complete. 2024-09-12T03:20:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 Processor -- start 2024-09-12T03:20:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 -- start start 2024-09-12T03:20:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d20102d30 0x7f0d201a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d201120d0 con 0x7f0d201082a0 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20111f50 con 0x7f0d20102d30 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.788+0000 7f0d27cd2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20112250 con 0x7f0d201036e0 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45324/0 (socket says 172.21.15.19:45324) 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 -- 172.21.15.19:0/3850556135 learned_addr learned my addr 172.21.15.19:0/3850556135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2626f640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d20102d30 0x7f0d201a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 -- 172.21.15.19:0/3850556135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d20102d30 msgr2=0x7f0d201a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d20102d30 0x7f0d201a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 -- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 msgr2=0x7f0d2019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2019a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 -- 172.21.15.19:0/3850556135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d2019acf0 con 0x7f0d201036e0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2626f640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.789+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000d740 tx=0x7f0d1000dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.790+0000 7f0d16ffd640 1 -- 172.21.15.19:0/3850556135 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10017070 con 0x7f0d201036e0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.790+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d2019afe0 con 0x7f0d201036e0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.790+0000 7f0d16ffd640 1 -- 172.21.15.19:0/3850556135 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1000de50 con 0x7f0d201036e0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.790+0000 7f0d16ffd640 1 -- 172.21.15.19:0/3850556135 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000f6b0 con 0x7f0d201036e0 2024-09-12T03:20:10.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.790+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d201a72d0 con 0x7f0d201036e0 2024-09-12T03:20:10.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.792+0000 7f0d16ffd640 1 -- 172.21.15.19:0/3850556135 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0d1001a070 con 0x7f0d201036e0 2024-09-12T03:20:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.793+0000 7f0d2a0ab740 1 Processor -- start 2024-09-12T03:20:10.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.793+0000 7f0d2a0ab740 1 -- start start 2024-09-12T03:20:10.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab316090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 0x5556ab323760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556ab27c2f0 con 0x5556ab3165d0 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5556ab223cf0 con 0x5556ab317b40 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5556ab2782b0 con 0x5556ab321370 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d25a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d25a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43372/0 (socket says 172.21.15.19:43372) 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d25a6e640 1 -- 172.21.15.19:0/2896898138 learned_addr learned my addr 172.21.15.19:0/2896898138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2626f640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab316090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 0x5556ab323760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d25a6e640 1 -- 172.21.15.19:0/2896898138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 msgr2=0x5556ab323760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.794+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 0x5556ab323760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d25a6e640 1 -- 172.21.15.19:0/2896898138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 msgr2=0x5556ab316090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab316090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d25a6e640 1 -- 172.21.15.19:0/2896898138 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ab2083a0 con 0x5556ab3165d0 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d16ffd640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 0x7f0d0c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d16ffd640 1 -- 172.21.15.19:0/3850556135 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0d100cedc0 con 0x7f0d201036e0 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009c80 tx=0x7f0d1c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7521720ff90ae904 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 0x5556ab323760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d0bfff640 1 -- 172.21.15.19:0/2896898138 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c012450 con 0x5556ab3165d0 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d2626f640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 0x7f0d0c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d0bfff640 1 -- 172.21.15.19:0/2896898138 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1c012ad0 con 0x5556ab3165d0 2024-09-12T03:20:10.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.795+0000 7f0d0bfff640 1 -- 172.21.15.19:0/2896898138 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d1c00fc70 con 0x5556ab3165d0 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2896898138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 msgr2=0x5556ab316a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c009c80 tx=0x7f0d1c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2896898138 shutdown_connections 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab321370 0x5556ab323760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab316a10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.796+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2896898138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab316090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.797+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2896898138 >> 172.21.15.19:0/2896898138 conn(0x5556ab2df310 msgr2=0x5556ab312c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.797+0000 7f0d2626f640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 0x7f0d0c0a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004af0 tx=0x7f0d18043040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.797+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2896898138 shutdown_connections 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.797+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2896898138 wait complete. 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.797+0000 7f0d2a0ab740 1 -- start start 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 Processor -- start 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab320d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab31dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 0x5556ab31e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5556ab324070 con 0x5556ab321370 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5556ab237060 con 0x5556ab317b40 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5556ab3241f0 con 0x5556ab3165d0 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2a0ab740 10 client.0 _ll_register_callbacks cb 0x5556aaf8a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2626f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 0x5556ab31e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2626f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 0x5556ab31e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43382/0 (socket says 172.21.15.19:43382) 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab31dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab320d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.798+0000 7f0d2626f640 1 -- 172.21.15.19:0/2949844972 learned_addr learned my addr 172.21.15.19:0/2949844972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:10.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 -- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 msgr2=0x5556ab320d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab320d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 -- 172.21.15.19:0/2949844972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 msgr2=0x5556ab31e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 0x5556ab31e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 -- 172.21.15.19:0/2949844972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556aafed3c0 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2526d640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab31dfe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d100cc920 tx=0x7f0d10009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821e3c2a958974cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10021050 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0d10097ca0 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1c012ad0 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d1009f9e0 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.799+0000 7f0d2a0ab740 10 client.25145 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556ab32fa90 con 0x5556ab317b40 2024-09-12T03:20:10.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d100a42a0 con 0x5556ab317b40 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d10084cc0 con 0x5556ab317b40 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 10 client.25145 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 10 client.25145 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 20 client.25145 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab331460 0x5556ab333850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 4 client.25145 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.800+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0d1c012ad0 con 0x5556ab331460 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.801+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab331460 0x5556ab333850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.801+0000 7f0d097fa640 10 client.25145 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.801+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab331460 0x5556ab333850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89597628fb557b48 server_cookie=7aeba2481c29c8ca in_seq=0 out_seq=0 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d097fa640 1 -- 172.21.15.19:0/2949844972 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0d1c012ad0 con 0x5556ab331460 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d097fa640 10 client.25145 handle_command_reply: tid=0 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 client.25145 shutdown 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 msgr2=0x5556ab31dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab31dfe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d100cc920 tx=0x7f0d10009c30 comp rx=0 tx=0).stop 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 shutdown_connections 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab321370 0x5556ab31e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab331460 0x5556ab333850 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab317b40 0x5556ab31dfe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/2949844972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab320d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d097fa640 0 client.25145 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:10.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 >> 172.21.15.19:0/2949844972 conn(0x5556ab2df310 msgr2=0x5556ab318930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.802+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 shutdown_connections 2024-09-12T03:20:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.803+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/2949844972 wait complete. 2024-09-12T03:20:10.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.803+0000 7f0d2a0ab740 20 client.25145 trim_cache size 0 max 16384 2024-09-12T03:20:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 Processor -- start 2024-09-12T03:20:10.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 -- start start 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab311bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab319660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2006faa0 con 0x5556ab317b40 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d200715e0 con 0x5556ab321370 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20070170 con 0x5556ab3165d0 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43392/0 (socket says 172.21.15.19:43392) 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2526d640 1 -- 172.21.15.19:0/4256845438 learned_addr learned my addr 172.21.15.19:0/4256845438 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.816+0000 7f0d2626f640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab319660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab311bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 -- 172.21.15.19:0/4256845438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 msgr2=0x5556ab311bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab311bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 -- 172.21.15.19:0/4256845438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 msgr2=0x5556ab319120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 -- 172.21.15.19:0/4256845438 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556ab333bc0 con 0x5556ab321370 2024-09-12T03:20:10.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2526d640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.817+0000 7f0d2626f640 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab319660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009ca0 tx=0x7f0d1804b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=441d967b548fbd05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.818+0000 7f0d0b7fe640 1 -- 172.21.15.19:0/4256845438 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18004240 con 0x5556ab321370 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.818+0000 7f0d0b7fe640 1 -- 172.21.15.19:0/4256845438 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d1805c070 con 0x5556ab321370 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.818+0000 7f0d0b7fe640 1 -- 172.21.15.19:0/4256845438 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d18030be0 con 0x5556ab321370 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/4256845438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 msgr2=0x5556ab319660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab319660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0d18009ca0 tx=0x7f0d1804b000 comp rx=0 tx=0).stop 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/4256845438 shutdown_connections 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab319660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab317b40 0x5556ab319120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/4256845438 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab3165d0 0x5556ab311bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/4256845438 >> 172.21.15.19:0/4256845438 conn(0x5556ab2df310 msgr2=0x5556ab3114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/4256845438 shutdown_connections 2024-09-12T03:20:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/4256845438 wait complete. 2024-09-12T03:20:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.819+0000 7f0d2a0ab740 1 -- start start 2024-09-12T03:20:10.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.820+0000 7f0d2a0ab740 1 Processor -- start 2024-09-12T03:20:10.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.820+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.820+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2a0ab740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab325220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20070f10 con 0x5556ab3165d0 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d25a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d20070840 con 0x5556ab321370 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2a0ab740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2006da80 con 0x5556ab317b40 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2a0ab740 10 client.0 _ll_register_callbacks cb 0x7f0d10010c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45360/0 (socket says 172.21.15.19:45360) 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d25a6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43398/0 (socket says 172.21.15.19:43398) 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 -- 172.21.15.19:0/3060615365 learned_addr learned my addr 172.21.15.19:0/3060615365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2626f640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab325220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 -- 172.21.15.19:0/3060615365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 msgr2=0x5556ab325220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab325220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 -- 172.21.15.19:0/3060615365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 msgr2=0x5556ab3299b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:10.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d2526d640 1 -- 172.21.15.19:0/3060615365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556ab325810 con 0x5556ab317b40 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.821+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d2526d640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d100afbf0 tx=0x7f0d10009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703b7ea6e307222 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d097fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d10021040 con 0x5556ab317b40 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0d10084bc0 con 0x5556ab317b40 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d1805c070 con 0x5556ab317b40 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0d1009bd30 con 0x5556ab317b40 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d2a0ab740 10 client.25138 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:10.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.822+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556ab325810 con 0x5556ab317b40 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d10015060 con 0x5556ab317b40 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f0d1009aa80 con 0x5556ab317b40 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 10 client.25138 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 10 client.25138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 20 client.25138 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:10.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab39c280 0x5556ab39e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 4 client.25138 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.823+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0d1805c070 con 0x5556ab39c280 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab39c280 0x5556ab39e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d25a6e640 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab39c280 0x5556ab39e6a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4814c26f77e329 server_cookie=345122aca4344ca in_seq=0 out_seq=0 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d097fa640 10 client.25138 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d097fa640 1 -- 172.21.15.19:0/3060615365 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0d1805c070 con 0x5556ab39c280 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d097fa640 10 client.25138 handle_command_reply: tid=0 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d2a0ab740 1 client.25138 shutdown 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 msgr2=0x5556ab324ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.824+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0d100afbf0 tx=0x7f0d10009d60 comp rx=0 tx=0).stop 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 shutdown_connections 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5556ab39c280 0x5556ab39e6a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5556ab321370 0x5556ab325220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d097fa640 0 client.25138 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5556ab317b40 0x5556ab324ce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 --2- 172.21.15.19:0/3060615365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5556ab3165d0 0x5556ab3299b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 >> 172.21.15.19:0/3060615365 conn(0x5556ab2df310 msgr2=0x5556ab311d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 shutdown_connections 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 1 -- 172.21.15.19:0/3060615365 wait complete. 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.825+0000 7f0d2a0ab740 20 client.25138 trim_cache size 0 max 16384 2024-09-12T03:20:10.829 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 msgr2=0x7f0d0c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 0x7f0d0c0a50c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d18004af0 tx=0x7f0d18043040 comp rx=0 tx=0).stop 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 msgr2=0x7f0d201a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000d740 tx=0x7f0d1000dc00 comp rx=0 tx=0).stop 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 shutdown_connections 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0d0c0a2c00 0x7f0d0c0a50c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0d201082a0 0x7f0d2019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0d201036e0 0x7f0d201a0940 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 --2- 172.21.15.19:0/3850556135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d20102d30 0x7f0d201a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.826+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 >> 172.21.15.19:0/3850556135 conn(0x7f0d200fc7d0 msgr2=0x7f0d20101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.827+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 shutdown_connections 2024-09-12T03:20:10.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:10.827+0000 7f0d27cd2640 1 -- 172.21.15.19:0/3850556135 wait complete. 2024-09-12T03:20:11.437 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:11 smithi019 ceph-mon[30953]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:11 smithi135 ceph-mon[35474]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:11.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:11 smithi121 ceph-mon[34963]: pgmap v371: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T03:20:12.673 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:20:12.673 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:20:12.986 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:13.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:13 smithi019 ceph-mon[30953]: pgmap v372: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 -- 172.21.15.19:0/1025182102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 msgr2=0x7ff28c100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 --2- 172.21.15.19:0/1025182102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c100ef0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff278009930 tx=0x7ff27802a420 comp rx=0 tx=0).stop 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 -- 172.21.15.19:0/1025182102 shutdown_connections 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 --2- 172.21.15.19:0/1025182102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c101430 0x7ff28c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 --2- 172.21.15.19:0/1025182102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c100ef0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 --2- 172.21.15.19:0/1025182102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c106a90 0x7ff28c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 -- 172.21.15.19:0/1025182102 >> 172.21.15.19:0/1025182102 conn(0x7ff28c0fc7d0 msgr2=0x7ff28c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 -- 172.21.15.19:0/1025182102 shutdown_connections 2024-09-12T03:20:13.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.541+0000 7ff292e9c640 1 -- 172.21.15.19:0/1025182102 wait complete. 2024-09-12T03:20:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.542+0000 7ff292e9c640 1 Processor -- start 2024-09-12T03:20:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.542+0000 7ff292e9c640 1 -- start start 2024-09-12T03:20:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c101430 0x7ff28c196630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 0x7ff28c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10fd80 con 0x7ff28c100a90 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10fc00 con 0x7ff28c106a90 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff292e9c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff28c10ff00 con 0x7ff28c101430 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43414/0 (socket says 172.21.15.19:43414) 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff290c38640 1 -- 172.21.15.19:0/4250043612 learned_addr learned my addr 172.21.15.19:0/4250043612 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff291439640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 0x7ff28c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.543+0000 7ff28bfff640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c101430 0x7ff28c196630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 -- 172.21.15.19:0/4250043612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c101430 msgr2=0x7ff28c196630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c101430 0x7ff28c196630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 -- 172.21.15.19:0/4250043612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 msgr2=0x7ff28c196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 0x7ff28c196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 -- 172.21.15.19:0/4250043612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28c1aad70 con 0x7ff28c100a90 2024-09-12T03:20:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff291439640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 0x7ff28c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:13.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff290c38640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ea20 tx=0x7ff28000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:13 smithi135 ceph-mon[35474]: pgmap v372: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff289ffb640 1 -- 172.21.15.19:0/4250043612 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28000cd80 con 0x7ff28c100a90 2024-09-12T03:20:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.544+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28c1ab060 con 0x7ff28c100a90 2024-09-12T03:20:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.545+0000 7ff289ffb640 1 -- 172.21.15.19:0/4250043612 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff280005500 con 0x7ff28c100a90 2024-09-12T03:20:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.545+0000 7ff289ffb640 1 -- 172.21.15.19:0/4250043612 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280010690 con 0x7ff28c100a90 2024-09-12T03:20:13.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.545+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff28c1ab570 con 0x7ff28c100a90 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.548+0000 7ff295275740 1 Processor -- start 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.548+0000 7ff295275740 1 -- start start 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39dc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f3937280 con 0x55c9f39d2480 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c9f38deb10 con 0x55c9f39dcd40 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff295275740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9f3933240 con 0x55c9f39d1b60 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff289ffb640 1 -- 172.21.15.19:0/4250043612 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2800046c0 con 0x7ff28c100a90 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff28bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33322/0 (socket says 172.21.15.19:33322) 2024-09-12T03:20:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff28bfff640 1 -- 172.21.15.19:0/3370084432 learned_addr learned my addr 172.21.15.19:0/3370084432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff291439640 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.549+0000 7ff290c38640 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39dc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 -- 172.21.15.19:0/3370084432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 msgr2=0x55c9f39d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 -- 172.21.15.19:0/3370084432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 msgr2=0x55c9f39dc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39dc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 -- 172.21.15.19:0/3370084432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f38eb1f0 con 0x55c9f39dcd40 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.550+0000 7ff28bfff640 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff27802a940 tx=0x7ff278005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0445d534fbe056 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.551+0000 7ff276ffd640 1 -- 172.21.15.19:0/3370084432 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278040070 con 0x55c9f39dcd40 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.551+0000 7ff276ffd640 1 -- 172.21.15.19:0/3370084432 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278031870 con 0x55c9f39dcd40 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.551+0000 7ff276ffd640 1 -- 172.21.15.19:0/3370084432 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278035040 con 0x55c9f39dcd40 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff289ffb640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 0x7ff27c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff291439640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 0x7ff27c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff289ffb640 1 -- 172.21.15.19:0/4250043612 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff2800d3060 con 0x7ff28c100a90 2024-09-12T03:20:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 -- 172.21.15.19:0/3370084432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 msgr2=0x55c9f39df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff27802a940 tx=0x7ff278005bd0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 -- 172.21.15.19:0/3370084432 shutdown_connections 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39df130 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39dc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 --2- 172.21.15.19:0/3370084432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.552+0000 7ff295275740 1 -- 172.21.15.19:0/3370084432 >> 172.21.15.19:0/3370084432 conn(0x55c9f39ce0f0 msgr2=0x55c9f39ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.553+0000 7ff295275740 1 -- 172.21.15.19:0/3370084432 shutdown_connections 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.553+0000 7ff295275740 1 -- 172.21.15.19:0/3370084432 wait complete. 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.553+0000 7ff295275740 1 -- start start 2024-09-12T03:20:13.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.553+0000 7ff295275740 1 Processor -- start 2024-09-12T03:20:13.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 0x7ff27c0a50b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2840099a0 tx=0x7ff284009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9f39df910 con 0x55c9f39d2480 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c9f38f1f10 con 0x55c9f39dcd40 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9f39dfa90 con 0x55c9f39d1b60 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff295275740 10 client.0 _ll_register_callbacks cb 0x55c9f38baa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33338/0 (socket says 172.21.15.19:33338) 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 -- 172.21.15.19:0/1191026439 learned_addr learned my addr 172.21.15.19:0/1191026439 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:13.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff290c38640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff28bfff640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 -- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 msgr2=0x55c9f39dc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39dc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 -- 172.21.15.19:0/1191026439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 msgr2=0x55c9f39d9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39d9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.554+0000 7ff291439640 1 -- 172.21.15.19:0/1191026439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f36a86a0 con 0x55c9f39dcd40 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff278031870 con 0x55c9f39dcd40 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff291439640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff284012010 tx=0x7ff284004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25262f3bc735c31a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff26bfff640 10 client.25163 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff26bfff640 1 -- 172.21.15.19:0/1191026439 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28402c060 con 0x55c9f39dcd40 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff26bfff640 1 -- 172.21.15.19:0/1191026439 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff284017030 con 0x55c9f39dcd40 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.555+0000 7ff26bfff640 1 -- 172.21.15.19:0/1191026439 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff28401fc80 con 0x55c9f39dcd40 2024-09-12T03:20:13.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 10 client.25163 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55c9f39eb3e0 con 0x55c9f39dcd40 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff26bfff640 1 -- 172.21.15.19:0/1191026439 <== mon.2 v2:172.21.15.121:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff284027410 con 0x55c9f39dcd40 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 10 client.25163 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 10 client.25163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 20 client.25163 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f39ec780 0x55c9f39eec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 4 client.25163 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f39447f0 con 0x55c9f39ec780 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.556+0000 7ff290c38640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f39ec780 0x55c9f39eec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.557+0000 7ff290c38640 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f39ec780 0x55c9f39eec20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e06c7a28e1e1c18b server_cookie=8cf6c9390850b19d in_seq=0 out_seq=0 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.557+0000 7ff26bfff640 10 client.25163 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff26bfff640 1 -- 172.21.15.19:0/1191026439 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9f39447f0 con 0x55c9f39ec780 2024-09-12T03:20:13.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff26bfff640 10 client.25163 handle_command_reply: tid=0 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 client.25163 shutdown 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 msgr2=0x55c9f39da240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff284012010 tx=0x7ff284004ab0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 shutdown_connections 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f39ec780 0x55c9f39eec20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff26bfff640 0 client.25163 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39dcd40 0x55c9f39da240 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d2480 0x55c9f39d9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 --2- 172.21.15.19:0/1191026439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39d1b60 0x55c9f39dc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 >> 172.21.15.19:0/1191026439 conn(0x55c9f39ce0f0 msgr2=0x55c9f39d3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.558+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 shutdown_connections 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.559+0000 7ff295275740 1 -- 172.21.15.19:0/1191026439 wait complete. 2024-09-12T03:20:13.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.559+0000 7ff295275740 20 client.25163 trim_cache size 0 max 16384 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.583+0000 7ff295275740 1 Processor -- start 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.583+0000 7ff295275740 1 -- start start 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39db960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000e370 con 0x55c9f39d1b60 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000feb0 con 0x55c9f39d2480 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff295275740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000ea40 con 0x55c9f39dcd40 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43432/0 (socket says 172.21.15.19:43432) 2024-09-12T03:20:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff290c38640 1 -- 172.21.15.19:0/1395362205 learned_addr learned my addr 172.21.15.19:0/1395362205 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.584+0000 7ff291439640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff28bfff640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39db960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 -- 172.21.15.19:0/1395362205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 msgr2=0x55c9f39db960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39db960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 -- 172.21.15.19:0/1395362205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 msgr2=0x55c9f39e8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 -- 172.21.15.19:0/1395362205 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f39d22b0 con 0x55c9f39dcd40 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff290c38640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff291439640 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39f44f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff284011010 tx=0x7ff284022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b9bc3e0104392e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.585+0000 7ff2767fc640 1 -- 172.21.15.19:0/1395362205 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28402c040 con 0x55c9f39dcd40 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff2767fc640 1 -- 172.21.15.19:0/1395362205 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278031870 con 0x55c9f39dcd40 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff2767fc640 1 -- 172.21.15.19:0/1395362205 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284004490 con 0x55c9f39dcd40 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 -- 172.21.15.19:0/1395362205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 msgr2=0x55c9f39f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39f44f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff284011010 tx=0x7ff284022000 comp rx=0 tx=0).stop 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 -- 172.21.15.19:0/1395362205 shutdown_connections 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39f44f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39db960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 --2- 172.21.15.19:0/1395362205 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.586+0000 7ff295275740 1 -- 172.21.15.19:0/1395362205 >> 172.21.15.19:0/1395362205 conn(0x55c9f39ce0f0 msgr2=0x55c9f39db250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.587+0000 7ff295275740 1 -- 172.21.15.19:0/1395362205 shutdown_connections 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.587+0000 7ff295275740 1 -- 172.21.15.19:0/1395362205 wait complete. 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.587+0000 7ff295275740 1 -- start start 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.587+0000 7ff295275740 1 Processor -- start 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39e5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000f7e0 con 0x55c9f39d1b60 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000f110 con 0x55c9f39d2480 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff27000c350 con 0x55c9f39dcd40 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff295275740 10 client.0 _ll_register_callbacks cb 0x7ff284009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43442/0 (socket says 172.21.15.19:43442) 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 -- 172.21.15.19:0/2622568993 learned_addr learned my addr 172.21.15.19:0/2622568993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff28bfff640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39e5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff291439640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 -- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 msgr2=0x55c9f39dfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39dfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 -- 172.21.15.19:0/2622568993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 msgr2=0x55c9f39e5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39e5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.588+0000 7ff290c38640 1 -- 172.21.15.19:0/2622568993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f39e0600 con 0x55c9f39d1b60 2024-09-12T03:20:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff290c38640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff280097880 tx=0x7ff280095f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf900402b1e353d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff26bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28009bb20 con 0x55c9f39d1b60 2024-09-12T03:20:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff2800049b0 con 0x55c9f39d1b60 2024-09-12T03:20:13.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff278031870 con 0x55c9f39d1b60 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff28009b420 con 0x55c9f39d1b60 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff295275740 10 client.15597 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.589+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c9f3a580e0 con 0x55c9f39d1b60 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff278031870 con 0x55c9f39d1b60 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff2800ccb70 con 0x55c9f39d1b60 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 10 client.15597 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 10 client.15597 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 20 client.15597 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f3a587e0 0x55c9f3a5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:13.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 4 client.15597 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9f393ed00 con 0x55c9f3a587e0 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.590+0000 7ff28bfff640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f3a587e0 0x55c9f3a5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.591+0000 7ff26bfff640 10 client.15597 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.591+0000 7ff28bfff640 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f3a587e0 0x55c9f3a5ac00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e171f9347d7ba010 server_cookie=c128185eda12bd6f in_seq=0 out_seq=0 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff26bfff640 1 -- 172.21.15.19:0/2622568993 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9f393ed00 con 0x55c9f3a587e0 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff26bfff640 10 client.15597 handle_command_reply: tid=0 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 client.15597 shutdown 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 msgr2=0x55c9f39e5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff280097880 tx=0x7ff280095f90 comp rx=0 tx=0).stop 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 shutdown_connections 2024-09-12T03:20:13.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9f39dcd40 0x55c9f39dfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9f3a587e0 0x55c9f3a5ac00 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9f39d2480 0x55c9f39e5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff26bfff640 0 client.15597 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 --2- 172.21.15.19:0/2622568993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9f39d1b60 0x55c9f39e5010 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.592+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 >> 172.21.15.19:0/2622568993 conn(0x55c9f39ce0f0 msgr2=0x55c9f39cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.593+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 shutdown_connections 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.593+0000 7ff295275740 1 -- 172.21.15.19:0/2622568993 wait complete. 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.593+0000 7ff295275740 20 client.15597 trim_cache size 0 max 16384 2024-09-12T03:20:13.597 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:20:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:20:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "777e1955-0afe-4589-95f3-3c7ed5ba5833", 2024-09-12T03:20:13.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:20:13.598 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:20:13.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 msgr2=0x7ff27c0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 0x7ff27c0a50b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff2840099a0 tx=0x7ff284009500 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 msgr2=0x7ff28c1960f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff28000ea20 tx=0x7ff28000ee20 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 shutdown_connections 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff28c106a90 0x7ff28c196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff28c101430 0x7ff28c196630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff27c0a2bf0 0x7ff27c0a50b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 --2- 172.21.15.19:0/4250043612 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff28c100a90 0x7ff28c1960f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 >> 172.21.15.19:0/4250043612 conn(0x7ff28c0fc7d0 msgr2=0x7ff28c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 shutdown_connections 2024-09-12T03:20:13.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:13.594+0000 7ff292e9c640 1 -- 172.21.15.19:0/4250043612 wait complete. 2024-09-12T03:20:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:13 smithi121 ceph-mon[34963]: pgmap v372: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:14.521 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:20:14.807 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 -- 172.21.15.19:0/1778249164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208102780 msgr2=0x7f92081067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 --2- 172.21.15.19:0/1778249164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208102780 0x7f92081067d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f91fc009900 tx=0x7f91fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 -- 172.21.15.19:0/1778249164 shutdown_connections 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 --2- 172.21.15.19:0/1778249164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 0x7f920810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 --2- 172.21.15.19:0/1778249164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208102780 0x7f92081067d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 --2- 172.21.15.19:0/1778249164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208101dd0 0x7f92081021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 -- 172.21.15.19:0/1778249164 >> 172.21.15.19:0/1778249164 conn(0x7f92080fd8b0 msgr2=0x7f92080ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 -- 172.21.15.19:0/1778249164 shutdown_connections 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.330+0000 7f920d3af640 1 -- 172.21.15.19:0/1778249164 wait complete. 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.331+0000 7f920d3af640 1 Processor -- start 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 -- start start 2024-09-12T03:20:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208102780 0x7f9208198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 0x7f920819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f9206ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9208111fb0 con 0x7f9208101dd0 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f9206ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43462/0 (socket says 172.21.15.19:43462) 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f9206ffd640 1 -- 172.21.15.19:0/4190202865 learned_addr learned my addr 172.21.15.19:0/4190202865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9208111e30 con 0x7f9208102780 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.332+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9208112130 con 0x7f9208106f00 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f92067fc640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208102780 0x7f9208198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f92077fe640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 0x7f920819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 -- 172.21.15.19:0/4190202865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 msgr2=0x7f920819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 0x7f920819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 -- 172.21.15.19:0/4190202865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208102780 msgr2=0x7f9208198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208102780 0x7f9208198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 -- 172.21.15.19:0/4190202865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920819d3d0 con 0x7f9208101dd0 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.333+0000 7f9206ffd640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f91f800d950 tx=0x7f91f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.334+0000 7f91effff640 1 -- 172.21.15.19:0/4190202865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8017070 con 0x7f9208101dd0 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.334+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920819d6c0 con 0x7f9208101dd0 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.334+0000 7f91effff640 1 -- 172.21.15.19:0/4190202865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91f8009c30 con 0x7f9208101dd0 2024-09-12T03:20:15.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.335+0000 7f91effff640 1 -- 172.21.15.19:0/4190202865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800f6b0 con 0x7f9208101dd0 2024-09-12T03:20:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.335+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92081a4fa0 con 0x7f9208101dd0 2024-09-12T03:20:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.337+0000 7f920f788740 1 Processor -- start 2024-09-12T03:20:15.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.337+0000 7f920f788740 1 -- start start 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 0x562fe14f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe1509110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562fe14592e0 con 0x562fe14f3880 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562fe1400ce0 con 0x562fe14f4a20 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f920f788740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562fe14552a0 con 0x562fe1506d20 2024-09-12T03:20:15.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33376/0 (socket says 172.21.15.19:33376) 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 -- 172.21.15.19:0/1187992740 learned_addr learned my addr 172.21.15.19:0/1187992740 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f91effff640 1 -- 172.21.15.19:0/4190202865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f91f80109d0 con 0x7f9208101dd0 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92067fc640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe1509110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f9206ffd640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 0x562fe14f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 -- 172.21.15.19:0/1187992740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 msgr2=0x562fe1509110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe1509110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.338+0000 7f92077fe640 1 -- 172.21.15.19:0/1187992740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 msgr2=0x562fe14f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f92077fe640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 0x562fe14f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f92077fe640 1 -- 172.21.15.19:0/1187992740 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe13e5440 con 0x562fe14f4a20 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f9206ffd640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 0x562fe14f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f92077fe640 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f80 tx=0x7f92000118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b49663202834cdad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f91ed7fa640 1 -- 172.21.15.19:0/1187992740 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200004d20 con 0x562fe14f4a20 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f91effff640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 0x7f91f00a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f91ed7fa640 1 -- 172.21.15.19:0/1187992740 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9200004050 con 0x562fe14f4a20 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f91ed7fa640 1 -- 172.21.15.19:0/1187992740 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200005240 con 0x562fe14f4a20 2024-09-12T03:20:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.339+0000 7f91effff640 1 -- 172.21.15.19:0/4190202865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91f80ce880 con 0x7f9208101dd0 2024-09-12T03:20:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f9206ffd640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 0x7f91f00a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 -- 172.21.15.19:0/1187992740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 msgr2=0x562fe14f3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f80 tx=0x7f92000118d0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 -- 172.21.15.19:0/1187992740 shutdown_connections 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe1509110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f3880 0x562fe14f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 --2- 172.21.15.19:0/1187992740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f4a20 0x562fe14f3340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 -- 172.21.15.19:0/1187992740 >> 172.21.15.19:0/1187992740 conn(0x562fe14efa30 msgr2=0x562fe14efe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f9206ffd640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 0x7f91f00a5140 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f91f80058c0 tx=0x7f91f8010d70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 -- 172.21.15.19:0/1187992740 shutdown_connections 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.340+0000 7f920f788740 1 -- 172.21.15.19:0/1187992740 wait complete. 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 -- start start 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 Processor -- start 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 0x562fe14f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe14f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562fe1509a50 con 0x562fe14f4a20 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562fe1414050 con 0x562fe14f3880 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562fe1509bd0 con 0x562fe1506d20 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.341+0000 7f920f788740 10 client.0 _ll_register_callbacks cb 0x562fe1454160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 0x562fe14f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92077fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92077fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe14f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92077fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33390/0 (socket says 172.21.15.19:33390) 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92077fe640 1 -- 172.21.15.19:0/633580114 learned_addr learned my addr 172.21.15.19:0/633580114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 -- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 msgr2=0x562fe14f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe14f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 -- 172.21.15.19:0/633580114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 msgr2=0x562fe14f9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.342+0000 7f92067fc640 1 -- 172.21.15.19:0/633580114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe11ca3c0 con 0x562fe14f4a20 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f92067fc640 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 0x562fe14f6e80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f91fc02aa00 tx=0x7f91fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a7f779b63462ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f91eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91fc035040 con 0x562fe14f4a20 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91fc0052c0 con 0x562fe14f4a20 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9200004050 con 0x562fe14f4a20 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f91fc0054e0 con 0x562fe14f4a20 2024-09-12T03:20:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f920f788740 10 client.15606 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.343+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fe1502950 con 0x562fe14f4a20 2024-09-12T03:20:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9200004050 con 0x562fe14f4a20 2024-09-12T03:20:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f91fc003a10 con 0x562fe14f4a20 2024-09-12T03:20:15.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 10 client.15606 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 20 client.15606 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe150c1a0 0x562fe150e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 4 client.15606 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fe1466850 con 0x562fe150c1a0 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.344+0000 7f92077fe640 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe150c1a0 0x562fe150e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.345+0000 7f91eaff5640 10 client.15606 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.345+0000 7f92077fe640 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe150c1a0 0x562fe150e5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54626bd6c1a7257 server_cookie=bc7344c5f58601ba in_seq=0 out_seq=0 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f91eaff5640 1 -- 172.21.15.19:0/633580114 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562fe1466850 con 0x562fe150c1a0 2024-09-12T03:20:15.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f91eaff5640 10 client.15606 handle_command_reply: tid=0 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 client.15606 shutdown 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 msgr2=0x562fe14f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 0x562fe14f6e80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f91fc02aa00 tx=0x7f91fc0045e0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 shutdown_connections 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe1506d20 0x562fe14f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe14f4a20 0x562fe14f6e80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe150c1a0 0x562fe150e5d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 --2- 172.21.15.19:0/633580114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 >> 172.21.15.19:0/633580114 conn(0x562fe14efa30 msgr2=0x562fe14f5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f91eaff5640 0 client.15606 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 shutdown_connections 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 1 -- 172.21.15.19:0/633580114 wait complete. 2024-09-12T03:20:15.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.346+0000 7f920f788740 20 client.15606 trim_cache size 0 max 16384 2024-09-12T03:20:15.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 Processor -- start 2024-09-12T03:20:15.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 -- start start 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ab400 con 0x562fe1506d20 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92081acf40 con 0x562fe14f3880 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.381+0000 7f920f788740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92081abad0 con 0x562fe14f4a20 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43496/0 (socket says 172.21.15.19:43496) 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 -- 172.21.15.19:0/3477383036 learned_addr learned my addr 172.21.15.19:0/3477383036 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:15.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 -- 172.21.15.19:0/3477383036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 msgr2=0x562fe14f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 -- 172.21.15.19:0/3477383036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 msgr2=0x562fe14f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 -- 172.21.15.19:0/3477383036 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fe14f4290 con 0x562fe1506d20 2024-09-12T03:20:15.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.382+0000 7f92077fe640 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f20 tx=0x7f9200005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0199f97442f717 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.383+0000 7f91ecff9640 1 -- 172.21.15.19:0/3477383036 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200004490 con 0x562fe1506d20 2024-09-12T03:20:15.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.383+0000 7f91ecff9640 1 -- 172.21.15.19:0/3477383036 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9200004db0 con 0x562fe1506d20 2024-09-12T03:20:15.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.383+0000 7f91ecff9640 1 -- 172.21.15.19:0/3477383036 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9200005890 con 0x562fe1506d20 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.383+0000 7f920f788740 1 -- 172.21.15.19:0/3477383036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 msgr2=0x562fe14f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.383+0000 7f920f788740 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f9200009f20 tx=0x7f9200005fd0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 -- 172.21.15.19:0/3477383036 shutdown_connections 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe14f6c80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 --2- 172.21.15.19:0/3477383036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 -- 172.21.15.19:0/3477383036 >> 172.21.15.19:0/3477383036 conn(0x562fe14efa30 msgr2=0x562fe1502d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 -- 172.21.15.19:0/3477383036 shutdown_connections 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.384+0000 7f920f788740 1 -- 172.21.15.19:0/3477383036 wait complete. 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 -- start start 2024-09-12T03:20:15.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 Processor -- start 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14ff5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ac870 con 0x562fe1506d20 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92081ac1a0 con 0x562fe14f3880 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92081a93e0 con 0x562fe14f4a20 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f920f788740 10 client.0 _ll_register_callbacks cb 0x7f91fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:15.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.385+0000 7f92077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43502/0 (socket says 172.21.15.19:43502) 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 -- 172.21.15.19:0/2616173552 learned_addr learned my addr 172.21.15.19:0/2616173552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92067fc640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 -- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 msgr2=0x562fe14ffae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14ffae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 -- 172.21.15.19:0/2616173552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 msgr2=0x562fe14ff5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14ff5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 -- 172.21.15.19:0/2616173552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fe1599440 con 0x562fe1506d20 2024-09-12T03:20:15.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f92077fe640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9200004fa0 tx=0x7f9200005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7f04be282a9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.386+0000 7f91eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920003f020 con 0x562fe1506d20 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9200004490 con 0x562fe1506d20 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9200004db0 con 0x562fe1506d20 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9200037040 con 0x562fe1506d20 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f920f788740 10 client.15612 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:15.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562fe159a750 con 0x562fe1506d20 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9200005040 con 0x562fe1506d20 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.387+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f920003a070 con 0x562fe1506d20 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 20 client.15612 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe159ac60 0x562fe159d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:15.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 4 client.15612 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9200004db0 con 0x562fe159ac60 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.388+0000 7f92067fc640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe159ac60 0x562fe159d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f91eaff5640 10 client.15612 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f92067fc640 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe159ac60 0x562fe159d080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93bdb066c61c15dd server_cookie=4fc5c369c22b1311 in_seq=0 out_seq=0 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f91eaff5640 1 -- 172.21.15.19:0/2616173552 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f9200004db0 con 0x562fe159ac60 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f91eaff5640 10 client.15612 handle_command_reply: tid=0 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 client.15612 shutdown 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 msgr2=0x562fe1500020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f9200004fa0 tx=0x7f9200005ad0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 shutdown_connections 2024-09-12T03:20:15.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562fe1506d20 0x562fe1500020 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562fe14f4a20 0x562fe14ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562fe159ac60 0x562fe159d080 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f91eaff5640 0 client.15612 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 --2- 172.21.15.19:0/2616173552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562fe14f3880 0x562fe14ff5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 >> 172.21.15.19:0/2616173552 conn(0x562fe14efa30 msgr2=0x562fe14f6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 shutdown_connections 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.389+0000 7f920f788740 1 -- 172.21.15.19:0/2616173552 wait complete. 2024-09-12T03:20:15.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.390+0000 7f920f788740 20 client.15612 trim_cache size 0 max 16384 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "777e1955-0afe-4589-95f3-3c7ed5ba5833": { 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "777e1955-0afe-4589-95f3-3c7ed5ba5833", 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:15.394 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.390+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 msgr2=0x7f91f00a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.390+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 0x7f91f00a5140 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f91f80058c0 tx=0x7f91f8010d70 comp rx=0 tx=0).stop 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.390+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 msgr2=0x7f9208198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.390+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f91f800d950 tx=0x7f91f800de10 comp rx=0 tx=0).stop 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 shutdown_connections 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9208106f00 0x7f920819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9208102780 0x7f9208198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91f00a2c80 0x7f91f00a5140 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 --2- 172.21.15.19:0/4190202865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9208101dd0 0x7f9208198420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 >> 172.21.15.19:0/4190202865 conn(0x7f92080fd8b0 msgr2=0x7f92080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 shutdown_connections 2024-09-12T03:20:15.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:15.391+0000 7f920d3af640 1 -- 172.21.15.19:0/4190202865 wait complete. 2024-09-12T03:20:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:15 smithi135 ceph-mon[35474]: pgmap v373: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:15 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:20:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:15 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:15 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:20:15.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:15 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:15 smithi121 ceph-mon[34963]: pgmap v373: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:15 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:20:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:15 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:15 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:20:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:15 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:15 smithi019 ceph-mon[30953]: pgmap v373: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-09-12T03:20:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:15 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:20:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:15 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:15 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:20:15.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:15 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:20:16.263 INFO:tasks.cephfs.filesystem:scrub status for tag:777e1955-0afe-4589-95f3-3c7ed5ba5833 - {'path': '/', 'tag': '777e1955-0afe-4589-95f3-3c7ed5ba5833', 'options': 'recursive,force'} 2024-09-12T03:20:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:17 smithi135 ceph-mon[35474]: pgmap v374: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:20:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:17 smithi121 ceph-mon[34963]: pgmap v374: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:20:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:17 smithi019 ceph-mon[30953]: pgmap v374: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T03:20:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:19 smithi135 ceph-mon[35474]: pgmap v375: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:20:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:19 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:19 smithi121 ceph-mon[34963]: pgmap v375: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:20:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:19 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:19 smithi019 ceph-mon[30953]: pgmap v375: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:20:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:19 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:20:20.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: pgmap v376: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:20.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:20.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:20.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: pgmap v376: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: pgmap v376: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:20:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:20:22.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:22 smithi121 ceph-mon[34963]: pgmap v377: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:22.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:22 smithi019 ceph-mon[30953]: pgmap v377: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:22.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:22 smithi135 ceph-mon[35474]: pgmap v377: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:23.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:24.183 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:24.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:24 smithi121 ceph-mon[34963]: pgmap v378: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:24.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:24 smithi019 ceph-mon[30953]: pgmap v378: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:24.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:24 smithi135 ceph-mon[35474]: pgmap v378: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 -- 172.21.15.19:0/4700822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec54106f20 msgr2=0x7fec5410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 --2- 172.21.15.19:0/4700822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec54106f20 0x7fec5410eaa0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fec4c009ca0 tx=0x7fec4c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 -- 172.21.15.19:0/4700822 shutdown_connections 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 --2- 172.21.15.19:0/4700822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec54106f20 0x7fec5410eaa0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 --2- 172.21.15.19:0/4700822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec541027a0 0x7fec541067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 --2- 172.21.15.19:0/4700822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54101df0 0x7fec541021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 -- 172.21.15.19:0/4700822 >> 172.21.15.19:0/4700822 conn(0x7fec540fd8f0 msgr2=0x7fec540ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 -- 172.21.15.19:0/4700822 shutdown_connections 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.799+0000 7fec58c06640 1 -- 172.21.15.19:0/4700822 wait complete. 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.801+0000 7fec58c06640 1 Processor -- start 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.801+0000 7fec58c06640 1 -- start start 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.801+0000 7fec58c06640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 0x7fec541983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.801+0000 7fec58c06640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec4bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46426/0 (socket says 172.21.15.19:46426) 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec58c06640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54106f20 0x7fec5419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec4bfff640 1 -- 172.21.15.19:0/1618994206 learned_addr learned my addr 172.21.15.19:0/1618994206 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111fd0 con 0x7fec541027a0 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fec54111e50 con 0x7fec54106f20 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fec54112150 con 0x7fec54101df0 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 0x7fec541983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec52d9d640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54106f20 0x7fec5419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 -- 172.21.15.19:0/1618994206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54106f20 msgr2=0x7fec5419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54106f20 0x7fec5419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 -- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 msgr2=0x7fec541988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec5259c640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5419d360 con 0x7fec54101df0 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.802+0000 7fec4bfff640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:24.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.803+0000 7fec5259c640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 0x7fec541983b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec4400bc10 tx=0x7fec44009610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.803+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4400bd00 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.803+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5419d650 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.803+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec54074420 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.803+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec44005500 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.804+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4401b680 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.805+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec34005290 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.806+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec440046c0 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.807+0000 7fec4b7fe640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 0x7fec400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.808+0000 7fec4bfff640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 0x7fec400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.808+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fec440d2920 con 0x7fec54101df0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.808+0000 7fec4bfff640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 0x7fec400a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec540fe1a0 tx=0x7fec3c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:24.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:24.812+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fec44016070 con 0x7fec54101df0 2024-09-12T03:20:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.106+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec34005580 con 0x7fec54101df0 2024-09-12T03:20:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.107+0000 7fec4b7fe640 1 -- 172.21.15.19:0/1618994206 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7fec4409f1e0 con 0x7fec54101df0 2024-09-12T03:20:25.109 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:20:25.109 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:20:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.110+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 msgr2=0x7fec400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.110+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 0x7fec400a50c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec540fe1a0 tx=0x7fec3c00a380 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 msgr2=0x7fec541983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 0x7fec541983b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec4400bc10 tx=0x7fec44009610 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 shutdown_connections 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fec54106f20 0x7fec5419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec400a2c00 0x7fec400a50c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fec541027a0 0x7fec541988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 --2- 172.21.15.19:0/1618994206 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fec54101df0 0x7fec541983b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 >> 172.21.15.19:0/1618994206 conn(0x7fec540fd8f0 msgr2=0x7fec540fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 shutdown_connections 2024-09-12T03:20:25.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:25.111+0000 7fec58c06640 1 -- 172.21.15.19:0/1618994206 wait complete. 2024-09-12T03:20:25.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:25 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1618994206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:25 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1618994206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:25 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1618994206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:25.898 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 1a0cc1b --release --await' 2024-09-12T03:20:25.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 1a0cc1b --release --await 2024-09-12T03:20:26.187 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:26.613 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:26 smithi019 ceph-mon[30953]: pgmap v379: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:26.636 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:26 smithi135 ceph-mon[35474]: pgmap v379: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:26.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:26 smithi121 ceph-mon[34963]: pgmap v379: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:20:26.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 -- 172.21.15.19:0/474321009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102570 msgr2=0x7feca81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 --2- 172.21.15.19:0/474321009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102570 0x7feca81029b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fec98009900 tx=0x7fec9802a480 comp rx=0 tx=0).stop 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 -- 172.21.15.19:0/474321009 shutdown_connections 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 --2- 172.21.15.19:0/474321009 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8102ef0 0x7feca810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 --2- 172.21.15.19:0/474321009 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102570 0x7feca81029b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 --2- 172.21.15.19:0/474321009 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feca8100e20 0x7feca8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 -- 172.21.15.19:0/474321009 >> 172.21.15.19:0/474321009 conn(0x7feca80fc7d0 msgr2=0x7feca80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 -- 172.21.15.19:0/474321009 shutdown_connections 2024-09-12T03:20:26.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.739+0000 7fecaf833640 1 -- 172.21.15.19:0/474321009 wait complete. 2024-09-12T03:20:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 Processor -- start 2024-09-12T03:20:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 -- start start 2024-09-12T03:20:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8100e20 0x7feca81a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feca8102570 0x7feca81a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feca8110fa0 con 0x7feca8102ef0 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feca8110e20 con 0x7feca8102570 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaf833640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca8111120 con 0x7feca8100e20 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46442/0 (socket says 172.21.15.19:46442) 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.740+0000 7fecaddd0640 1 -- 172.21.15.19:0/158297320 learned_addr learned my addr 172.21.15.19:0/158297320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecad5cf640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8100e20 0x7feca81a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 -- 172.21.15.19:0/158297320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8100e20 msgr2=0x7feca81a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8100e20 0x7feca81a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 -- 172.21.15.19:0/158297320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feca8102570 msgr2=0x7feca81a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feca8102570 0x7feca81a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 -- 172.21.15.19:0/158297320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca819f520 con 0x7feca8102ef0 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaddd0640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feca000eaa0 tx=0x7feca000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fec9e7fc640 1 -- 172.21.15.19:0/158297320 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca000cdb0 con 0x7feca8102ef0 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca819f810 con 0x7feca8102ef0 2024-09-12T03:20:26.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fec9e7fc640 1 -- 172.21.15.19:0/158297320 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca0005020 con 0x7feca8102ef0 2024-09-12T03:20:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.741+0000 7fec9e7fc640 1 -- 172.21.15.19:0/158297320 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca00116b0 con 0x7feca8102ef0 2024-09-12T03:20:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.742+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca81ab7b0 con 0x7feca8102ef0 2024-09-12T03:20:26.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.744+0000 7fec9e7fc640 1 -- 172.21.15.19:0/158297320 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feca00051d0 con 0x7feca8102ef0 2024-09-12T03:20:26.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.745+0000 7fec9e7fc640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 0x7fec940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.745+0000 7fecad5cf640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 0x7fec940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.745+0000 7fec9e7fc640 1 -- 172.21.15.19:0/158297320 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feca00cf2e0 con 0x7feca8102ef0 2024-09-12T03:20:26.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 Processor -- start 2024-09-12T03:20:26.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 -- start start 2024-09-12T03:20:26.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b46d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b47240 0x563253b49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563253aa1620 con 0x563253b3c980 2024-09-12T03:20:26.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563253a48e50 con 0x563253b47240 2024-09-12T03:20:26.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563253a9d5e0 con 0x563253b3c060 2024-09-12T03:20:26.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecad5cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecad5cf640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:38974/0 (socket says 172.21.15.19:38974) 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecad5cf640 1 -- 172.21.15.19:0/2066670596 learned_addr learned my addr 172.21.15.19:0/2066670596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecad5cf640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 0x7fec940a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feca400bbe0 tx=0x7feca400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.746+0000 7fecaddd0640 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b47240 0x563253b49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 -- 172.21.15.19:0/2066670596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b47240 msgr2=0x563253b49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b47240 0x563253b49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 -- 172.21.15.19:0/2066670596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 msgr2=0x563253b46d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b46d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 -- 172.21.15.19:0/2066670596 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563253a39d40 con 0x563253b3c060 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feca4010fc0 tx=0x7feca4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b297620918366b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fec937fe640 1 -- 172.21.15.19:0/2066670596 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca4005220 con 0x563253b3c060 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fec937fe640 1 -- 172.21.15.19:0/2066670596 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca4004c80 con 0x563253b3c060 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.747+0000 7fec937fe640 1 -- 172.21.15.19:0/2066670596 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca401eac0 con 0x563253b3c060 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2066670596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 msgr2=0x563253b3c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7feca4010fc0 tx=0x7feca4013000 comp rx=0 tx=0).stop 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2066670596 shutdown_connections 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b47240 0x563253b49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b46d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2066670596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c060 0x563253b3c440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2066670596 >> 172.21.15.19:0/2066670596 conn(0x563253b385f0 msgr2=0x563253b389d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2066670596 shutdown_connections 2024-09-12T03:20:26.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2066670596 wait complete. 2024-09-12T03:20:26.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.748+0000 7fecb1c0c740 1 -- start start 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.749+0000 7fecb1c0c740 1 Processor -- start 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.749+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b44150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.749+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.749+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.749+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563253b49e10 con 0x563253b3c980 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563253a29f10 con 0x563253b3c060 2024-09-12T03:20:26.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563253b49f90 con 0x563253b47240 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46456/0 (socket says 172.21.15.19:46456) 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 -- 172.21.15.19:0/2159355210 learned_addr learned my addr 172.21.15.19:0/2159355210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b44150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecb1c0c740 10 client.0 _ll_register_callbacks cb 0x563253a4ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 -- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 msgr2=0x563253b44bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b44bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 -- 172.21.15.19:0/2159355210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 msgr2=0x563253b44150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b44150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 -- 172.21.15.19:0/2159355210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632537b0580 con 0x563253b3c980 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.750+0000 7fecacdce640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec980314e0 tx=0x7fec98032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb863b7e6542c234 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fec90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9803e070 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fec98004460 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca4004c80 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fec9804a050 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fecb1c0c740 10 client.15627 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.751+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56325387a960 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca4004c80 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fec98004b10 con 0x563253b3c980 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 10 client.15627 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 10 client.15627 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 20 client.15627 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:26.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253b57180 0x563253b595c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 4 client.15627 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563253aaeb90 con 0x563253b57180 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.752+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253b57180 0x563253b595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.753+0000 7fec90ff9640 10 client.15627 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.753+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253b57180 0x563253b595c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2496ddbfc750fd8 server_cookie=6c6a019095731cb8 in_seq=0 out_seq=0 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.753+0000 7fec90ff9640 1 -- 172.21.15.19:0/2159355210 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563253aaeb90 con 0x563253b57180 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.753+0000 7fec90ff9640 10 client.15627 handle_command_reply: tid=0 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 client.15627 shutdown 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 msgr2=0x563253b44690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec980314e0 tx=0x7fec98032de0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 shutdown_connections 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b44bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b44690 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253b57180 0x563253b595c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2159355210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b44150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 >> 172.21.15.19:0/2159355210 conn(0x563253b385f0 msgr2=0x563253b3e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fec90ff9640 0 client.15627 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 shutdown_connections 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2159355210 wait complete. 2024-09-12T03:20:26.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.754+0000 7fecb1c0c740 20 client.15627 trim_cache size 0 max 16384 2024-09-12T03:20:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.788+0000 7fecb1c0c740 1 Processor -- start 2024-09-12T03:20:26.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 -- start start 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b3eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b3b60 con 0x563253b3c980 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b56a0 con 0x563253b3c060 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b4230 con 0x563253b47240 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecad5cf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b3eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:38994/0 (socket says 172.21.15.19:38994) 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 -- 172.21.15.19:0/4120500720 learned_addr learned my addr 172.21.15.19:0/4120500720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecad5cf640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b3eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:36316/0 (socket says 172.21.15.19:36316) 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecacdce640 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 -- 172.21.15.19:0/4120500720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 msgr2=0x563253b3eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b3eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 -- 172.21.15.19:0/4120500720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 msgr2=0x563253b37640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b37640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.789+0000 7fecaddd0640 1 -- 172.21.15.19:0/4120500720 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563253b396a0 con 0x563253b47240 2024-09-12T03:20:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.790+0000 7fecaddd0640 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feca0009270 tx=0x7feca009baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d567c38dc9139c70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.790+0000 7fec92ffd640 1 -- 172.21.15.19:0/4120500720 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca001f070 con 0x563253b47240 2024-09-12T03:20:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.790+0000 7fec92ffd640 1 -- 172.21.15.19:0/4120500720 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feca00a4760 con 0x563253b47240 2024-09-12T03:20:26.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.790+0000 7fec92ffd640 1 -- 172.21.15.19:0/4120500720 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca009b1a0 con 0x563253b47240 2024-09-12T03:20:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 -- 172.21.15.19:0/4120500720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 msgr2=0x563253b413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feca0009270 tx=0x7feca009baf0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 -- 172.21.15.19:0/4120500720 shutdown_connections 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b47240 0x563253b413b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b3c980 0x563253b37640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/4120500720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b3eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 -- 172.21.15.19:0/4120500720 >> 172.21.15.19:0/4120500720 conn(0x563253b385f0 msgr2=0x563253b36f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 -- 172.21.15.19:0/4120500720 shutdown_connections 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.791+0000 7fecb1c0c740 1 -- 172.21.15.19:0/4120500720 wait complete. 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 -- start start 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 Processor -- start 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b5d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c980 0x563253b5d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b4fd0 con 0x563253b47240 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b4900 con 0x563253b3c060 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecb1c0c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feca81b1b40 con 0x563253b3c980 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecaddd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46480/0 (socket says 172.21.15.19:46480) 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.792+0000 7fecaddd0640 1 -- 172.21.15.19:0/2400983586 learned_addr learned my addr 172.21.15.19:0/2400983586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b5d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 -- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c980 msgr2=0x563253b5d670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c980 0x563253b5d670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 -- 172.21.15.19:0/2400983586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 msgr2=0x563253b5d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b5d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 -- 172.21.15.19:0/2400983586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563253bc11c0 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecb1c0c740 10 client.0 _ll_register_callbacks cb 0x7fec98032a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feca00a4760 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fecaddd0640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feca00984c0 tx=0x7feca0099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c832ca885c558127 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fec90ff9640 10 client.15633 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fec90ff9640 1 -- 172.21.15.19:0/2400983586 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feca001f040 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fec90ff9640 1 -- 172.21.15.19:0/2400983586 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feca0098160 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.793+0000 7fec90ff9640 1 -- 172.21.15.19:0/2400983586 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feca0005520 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 10 client.15633 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=28}) -- 0x563253bc2310 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fec90ff9640 1 -- 172.21.15.19:0/2400983586 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feca000f040 con 0x563253b47240 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 10 client.15633 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:26.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 10 client.15633 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 20 client.15633 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253bc28d0 0x563253bc4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 4 client.15633 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1a0cc1b", "release": true, "await": true}] 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1a0cc1b", "release": true, "await": true}) -- 0x563253aa90a0 con 0x563253bc28d0 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.794+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253bc28d0 0x563253bc4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.795+0000 7fec90ff9640 10 client.15633 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.795+0000 7fecad5cf640 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253bc28d0 0x563253bc4d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf0db214bc190a3c server_cookie=b0ea2cb8f8965b23 in_seq=0 out_seq=0 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fec90ff9640 1 -- 172.21.15.19:0/2400983586 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x563253aa90a0 con 0x563253bc28d0 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fec90ff9640 10 client.15633 handle_command_reply: tid=0 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 client.15633 shutdown 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 msgr2=0x563253bc0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feca00984c0 tx=0x7feca0099df0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 shutdown_connections 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563253b47240 0x563253bc0c50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 28, 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 4, 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:20:26.804 INFO:teuthology.orchestra.run.smithi019.stdout: "1a0cc1b": { 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 4, 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 144.5, 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:20:26.805 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:20:26.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563253b3c980 0x563253b5d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563253bc28d0 0x563253bc4d50 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 --2- 172.21.15.19:0/2400983586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563253b3c060 0x563253b5d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 >> 172.21.15.19:0/2400983586 conn(0x563253b385f0 msgr2=0x563253b37750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fec90ff9640 0 client.15633 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 shutdown_connections 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.796+0000 7fecb1c0c740 1 -- 172.21.15.19:0/2400983586 wait complete. 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecb1c0c740 20 client.15633 trim_cache size 0 max 16384 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 msgr2=0x7fec940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 0x7fec940a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7feca400bbe0 tx=0x7feca400b390 comp rx=0 tx=0).stop 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 msgr2=0x7feca819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7feca000eaa0 tx=0x7feca000ef60 comp rx=0 tx=0).stop 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 shutdown_connections 2024-09-12T03:20:26.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fec940a2b80 0x7fec940a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feca8102ef0 0x7feca819ec60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feca8102570 0x7feca81a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 --2- 172.21.15.19:0/158297320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feca8100e20 0x7feca81a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 >> 172.21.15.19:0/158297320 conn(0x7feca80fc7d0 msgr2=0x7feca806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 shutdown_connections 2024-09-12T03:20:26.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:26.797+0000 7fecaf833640 1 -- 172.21.15.19:0/158297320 wait complete. 2024-09-12T03:20:27.651 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1a0cc1b', total seconds elapsed: 28.3 2024-09-12T03:20:28.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:28 smithi121 ceph-mon[34963]: pgmap v380: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:28.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:28 smithi019 ceph-mon[30953]: pgmap v380: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:28.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:28 smithi135 ceph-mon[35474]: pgmap v380: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:30 smithi121 ceph-mon[34963]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:20:30.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:30 smithi019 ceph-mon[30953]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:20:30.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:30 smithi135 ceph-mon[35474]: pgmap v381: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:20:32.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:32 smithi121 ceph-mon[34963]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:32.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:32 smithi019 ceph-mon[30953]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:32.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:32 smithi135 ceph-mon[35474]: pgmap v382: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:34.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:34 smithi121 ceph-mon[34963]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:20:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:34 smithi019 ceph-mon[30953]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:20:34.686 INFO:tasks.workunit.client.0.smithi019.stderr:Updating files: 93% (12002/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T03:20:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:34 smithi135 ceph-mon[35474]: pgmap v383: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T03:20:35.538 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: git switch -c 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr:Or undo this operation with: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: git switch - 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:35.539 INFO:tasks.workunit.client.0.smithi019.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T03:20:35.543 DEBUG:teuthology.orchestra.run.smithi019:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T03:20:35.567 INFO:tasks.workunit.client.0.smithi019.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T03:20:35.570 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:20:35.570 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T03:20:36.258 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T03:20:36.341 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T03:20:36.383 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T03:20:36.385 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:20:36.385 INFO:tasks.workunit.client.0.smithi019.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T03:20:36.426 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T03:20:36.429 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:20:36.429 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T03:20:36.486 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-09-12T03:20:36.488 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-09-12T03:20:36.488 DEBUG:teuthology.orchestra.run.smithi019:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-09-12T03:20:36.550 INFO:tasks.workunit.client.0.smithi019.stdout:getting blogbench 2024-09-12T03:20:36.551 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'getting blogbench' 2024-09-12T03:20:36.551 INFO:tasks.workunit.client.0.smithi019.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-12T03:20:36.582 INFO:tasks.workunit.client.0.smithi019.stderr:--2024-09-12 03:20:36-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-09-12T03:20:36.582 INFO:tasks.workunit.client.0.smithi019.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-12T03:20:36.604 INFO:tasks.workunit.client.0.smithi019.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-12T03:20:36.625 INFO:tasks.workunit.client.0.smithi019.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-12T03:20:36.625 INFO:tasks.workunit.client.0.smithi019.stderr:Length: 103355 (101K) [application/octet-stream] 2024-09-12T03:20:36.626 INFO:tasks.workunit.client.0.smithi019.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-09-12T03:20:36.629 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:36 smithi121 ceph-mon[34963]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:36.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:36 smithi019 ceph-mon[30953]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:36.667 INFO:tasks.workunit.client.0.smithi019.stderr: 0K .......... .......... .......... .......... .......... 49% 1.22M 0s 2024-09-12T03:20:36.687 INFO:tasks.workunit.client.0.smithi019.stderr: 50K .......... .......... .......... .......... .......... 99% 2.36M 0s 2024-09-12T03:20:36.688 INFO:tasks.workunit.client.0.smithi019.stderr: 100K 100% 1.74T=0.06s 2024-09-12T03:20:36.688 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:36.688 INFO:tasks.workunit.client.0.smithi019.stderr:2024-09-12 03:20:36 (1.62 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-09-12T03:20:36.688 INFO:tasks.workunit.client.0.smithi019.stderr: 2024-09-12T03:20:36.688 INFO:tasks.workunit.client.0.smithi019.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-09-12T03:20:36.715 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/ 2024-09-12T03:20:36.716 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/m4/ 2024-09-12T03:20:36.717 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-09-12T03:20:36.718 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/man/ 2024-09-12T03:20:36.719 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/man/blogbench.8 2024-09-12T03:20:36.720 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/man/Makefile.am 2024-09-12T03:20:36.721 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/man/Makefile.in 2024-09-12T03:20:36.723 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/ 2024-09-12T03:20:36.724 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/mysnprintf.c 2024-09-12T03:20:36.725 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/mysnprintf.h 2024-09-12T03:20:36.726 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/blogbench.c 2024-09-12T03:20:36.727 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/blogbench.h 2024-09-12T03:20:36.728 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/fakesnprintf.c 2024-09-12T03:20:36.729 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/fakesnprintf.h 2024-09-12T03:20:36.730 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/blogbench_p.h 2024-09-12T03:20:36.731 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-09-12T03:20:36.732 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-09-12T03:20:36.733 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/Makefile.am 2024-09-12T03:20:36.734 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/Makefile.in 2024-09-12T03:20:36.736 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/process.c 2024-09-12T03:20:36.736 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/globals.h 2024-09-12T03:20:36.737 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/commenter.c 2024-09-12T03:20:36.739 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/rewriter.c 2024-09-12T03:20:36.740 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/reader.c 2024-09-12T03:20:36.741 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/writer.c 2024-09-12T03:20:36.742 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/src/helpers.c 2024-09-12T03:20:36.742 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/NEWS 2024-09-12T03:20:36.743 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/compile 2024-09-12T03:20:36.744 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/depcomp 2024-09-12T03:20:36.745 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/aclocal.m4 2024-09-12T03:20:36.746 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/README 2024-09-12T03:20:36.747 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/configure 2024-09-12T03:20:36.751 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/configure.ac 2024-09-12T03:20:36.752 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/config.guess 2024-09-12T03:20:36.753 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/install-sh 2024-09-12T03:20:36.754 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/config.sub 2024-09-12T03:20:36.755 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/missing 2024-09-12T03:20:36.756 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/mkinstalldirs 2024-09-12T03:20:36.757 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/Makefile.am 2024-09-12T03:20:36.758 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/Makefile.in 2024-09-12T03:20:36.759 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/config.h.in 2024-09-12T03:20:36.760 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/AUTHORS 2024-09-12T03:20:36.761 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/INSTALL 2024-09-12T03:20:36.762 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/ChangeLog 2024-09-12T03:20:36.763 INFO:tasks.workunit.client.0.smithi019.stdout:blogbench-1.0/COPYING 2024-09-12T03:20:36.764 INFO:tasks.workunit.client.0.smithi019.stderr:+ cd blogbench-1.0/ 2024-09-12T03:20:36.765 INFO:tasks.workunit.client.0.smithi019.stdout:making blogbench 2024-09-12T03:20:36.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'making blogbench' 2024-09-12T03:20:36.765 INFO:tasks.workunit.client.0.smithi019.stderr:+ ./configure 2024-09-12T03:20:36.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:36 smithi135 ceph-mon[35474]: pgmap v384: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:20:36.865 INFO:tasks.workunit.client.0.smithi019.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T03:20:37.875 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether build environment is sane... yes 2024-09-12T03:20:37.885 INFO:tasks.workunit.client.0.smithi019.stdout:checking for gawk... gawk 2024-09-12T03:20:37.901 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether make sets $(MAKE)... yes 2024-09-12T03:20:37.921 INFO:tasks.workunit.client.0.smithi019.stdout:checking for gcc... gcc 2024-09-12T03:20:37.975 INFO:tasks.workunit.client.0.smithi019.stdout:checking for C compiler default output file name... a.out 2024-09-12T03:20:37.977 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether the C compiler works... yes 2024-09-12T03:20:37.979 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether we are cross compiling... no 2024-09-12T03:20:38.017 INFO:tasks.workunit.client.0.smithi019.stdout:checking for suffix of executables... 2024-09-12T03:20:38.060 INFO:tasks.workunit.client.0.smithi019.stdout:checking for suffix of object files... o 2024-09-12T03:20:38.092 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T03:20:38.126 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether gcc accepts -g... yes 2024-09-12T03:20:38.190 INFO:tasks.workunit.client.0.smithi019.stdout:checking for gcc option to accept ANSI C... none needed 2024-09-12T03:20:38.234 INFO:tasks.workunit.client.0.smithi019.stdout:checking for style of include used by make... GNU 2024-09-12T03:20:38.338 INFO:tasks.workunit.client.0.smithi019.stdout:checking dependency style of gcc... gcc3 2024-09-12T03:20:38.339 INFO:tasks.workunit.client.0.smithi019.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-12T03:20:38.391 INFO:tasks.workunit.client.0.smithi019.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T03:20:38.432 INFO:tasks.workunit.client.0.smithi019.stdout:checking for egrep... grep -E 2024-09-12T03:20:38.442 INFO:tasks.workunit.client.0.smithi019.stdout:checking for AIX... no 2024-09-12T03:20:38.507 INFO:tasks.workunit.client.0.smithi019.stdout:checking for library containing strerror... none required 2024-09-12T03:20:38.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:38 smithi121 ceph-mon[34963]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:38.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:38 smithi019 ceph-mon[30953]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:38.749 INFO:tasks.workunit.client.0.smithi019.stdout:checking for ANSI C header files... yes 2024-09-12T03:20:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:38 smithi135 ceph-mon[35474]: pgmap v385: 97 pgs: 97 active+clean; 659 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:20:38.796 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-12T03:20:38.843 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/types.h... yes 2024-09-12T03:20:38.892 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/stat.h... yes 2024-09-12T03:20:38.940 INFO:tasks.workunit.client.0.smithi019.stdout:checking for stdlib.h... yes 2024-09-12T03:20:38.989 INFO:tasks.workunit.client.0.smithi019.stdout:checking for string.h... yes 2024-09-12T03:20:39.042 INFO:tasks.workunit.client.0.smithi019.stdout:checking for memory.h... yes 2024-09-12T03:20:39.094 INFO:tasks.workunit.client.0.smithi019.stdout:checking for strings.h... yes 2024-09-12T03:20:39.145 INFO:tasks.workunit.client.0.smithi019.stdout:checking for inttypes.h... yes 2024-09-12T03:20:39.194 INFO:tasks.workunit.client.0.smithi019.stdout:checking for stdint.h... yes 2024-09-12T03:20:39.245 INFO:tasks.workunit.client.0.smithi019.stdout:checking for unistd.h... yes 2024-09-12T03:20:39.257 INFO:tasks.workunit.client.0.smithi019.stdout:checking for memory.h... (cached) yes 2024-09-12T03:20:39.262 INFO:tasks.workunit.client.0.smithi019.stdout:checking for strings.h... (cached) yes 2024-09-12T03:20:39.268 INFO:tasks.workunit.client.0.smithi019.stdout:checking for unistd.h... (cached) yes 2024-09-12T03:20:39.312 INFO:tasks.workunit.client.0.smithi019.stdout:checking locale.h usability... yes 2024-09-12T03:20:39.332 INFO:tasks.workunit.client.0.smithi019.stdout:checking locale.h presence... yes 2024-09-12T03:20:39.333 INFO:tasks.workunit.client.0.smithi019.stdout:checking for locale.h... yes 2024-09-12T03:20:39.348 INFO:tasks.workunit.client.0.smithi019.stdout:checking for stdlib.h... (cached) yes 2024-09-12T03:20:39.353 INFO:tasks.workunit.client.0.smithi019.stdout:checking for string.h... (cached) yes 2024-09-12T03:20:39.417 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/socket.h usability... yes 2024-09-12T03:20:39.439 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/socket.h presence... yes 2024-09-12T03:20:39.439 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/socket.h... yes 2024-09-12T03:20:39.493 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/time.h usability... yes 2024-09-12T03:20:39.513 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/time.h presence... yes 2024-09-12T03:20:39.514 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/time.h... yes 2024-09-12T03:20:39.572 INFO:tasks.workunit.client.0.smithi019.stdout:checking fcntl.h usability... yes 2024-09-12T03:20:39.595 INFO:tasks.workunit.client.0.smithi019.stdout:checking fcntl.h presence... yes 2024-09-12T03:20:39.595 INFO:tasks.workunit.client.0.smithi019.stdout:checking for fcntl.h... yes 2024-09-12T03:20:39.655 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/fcntl.h usability... yes 2024-09-12T03:20:39.678 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/fcntl.h presence... yes 2024-09-12T03:20:39.679 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/fcntl.h... yes 2024-09-12T03:20:39.764 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/param.h usability... yes 2024-09-12T03:20:39.787 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/param.h presence... yes 2024-09-12T03:20:39.788 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/param.h... yes 2024-09-12T03:20:39.835 INFO:tasks.workunit.client.0.smithi019.stdout:checking ioctl.h usability... no 2024-09-12T03:20:39.855 INFO:tasks.workunit.client.0.smithi019.stdout:checking ioctl.h presence... no 2024-09-12T03:20:39.856 INFO:tasks.workunit.client.0.smithi019.stdout:checking for ioctl.h... no 2024-09-12T03:20:39.911 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/ioctl.h usability... yes 2024-09-12T03:20:39.934 INFO:tasks.workunit.client.0.smithi019.stdout:checking sys/ioctl.h presence... yes 2024-09-12T03:20:39.935 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sys/ioctl.h... yes 2024-09-12T03:20:39.988 INFO:tasks.workunit.client.0.smithi019.stdout:checking return type of signal handlers... void 2024-09-12T03:20:40.038 INFO:tasks.workunit.client.0.smithi019.stdout:checking for size_t... yes 2024-09-12T03:20:40.075 INFO:tasks.workunit.client.0.smithi019.stdout:checking for dev_t... yes 2024-09-12T03:20:40.119 INFO:tasks.workunit.client.0.smithi019.stdout:checking for short... yes 2024-09-12T03:20:40.203 INFO:tasks.workunit.client.0.smithi019.stdout:checking size of short... 2 2024-09-12T03:20:40.252 INFO:tasks.workunit.client.0.smithi019.stdout:checking for int... yes 2024-09-12T03:20:40.321 INFO:tasks.workunit.client.0.smithi019.stdout:checking size of int... 4 2024-09-12T03:20:40.365 INFO:tasks.workunit.client.0.smithi019.stdout:checking for long... yes 2024-09-12T03:20:40.437 INFO:tasks.workunit.client.0.smithi019.stdout:checking size of long... 8 2024-09-12T03:20:40.485 INFO:tasks.workunit.client.0.smithi019.stdout:checking for long long... yes 2024-09-12T03:20:40.556 INFO:tasks.workunit.client.0.smithi019.stdout:checking size of long long... 8 2024-09-12T03:20:40.599 INFO:tasks.workunit.client.0.smithi019.stdout:checking for ssize_t... yes 2024-09-12T03:20:40.638 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sig_atomic_t... yes 2024-09-12T03:20:40.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:40 smithi121 ceph-mon[34963]: pgmap v386: 97 pgs: 97 active+clean; 660 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:40.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:40 smithi019 ceph-mon[30953]: pgmap v386: 97 pgs: 97 active+clean; 660 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:40.692 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether gcc needs -traditional... no 2024-09-12T03:20:40.725 INFO:tasks.workunit.client.0.smithi019.stdout:checking for an ANSI C-conforming const... yes 2024-09-12T03:20:40.749 INFO:tasks.workunit.client.0.smithi019.stdout:checking for inline... inline 2024-09-12T03:20:40.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:40 smithi135 ceph-mon[35474]: pgmap v386: 97 pgs: 97 active+clean; 660 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:20:40.809 INFO:tasks.workunit.client.0.smithi019.stdout:checking for working alloca.h... yes 2024-09-12T03:20:40.858 INFO:tasks.workunit.client.0.smithi019.stdout:checking for alloca... yes 2024-09-12T03:20:40.928 INFO:tasks.workunit.client.0.smithi019.stdout:checking for working memcmp... yes 2024-09-12T03:20:40.984 INFO:tasks.workunit.client.0.smithi019.stdout:checking for strftime... yes 2024-09-12T03:20:41.062 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-09-12T03:20:41.128 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether stat accepts an empty string... no 2024-09-12T03:20:41.183 INFO:tasks.workunit.client.0.smithi019.stdout:checking for vprintf... yes 2024-09-12T03:20:41.240 INFO:tasks.workunit.client.0.smithi019.stdout:checking for _doprnt... no 2024-09-12T03:20:41.262 INFO:tasks.workunit.client.0.smithi019.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-12T03:20:41.274 INFO:tasks.workunit.client.0.smithi019.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-12T03:20:41.332 INFO:tasks.workunit.client.0.smithi019.stdout:checking for the pthreads library -lpthreads... no 2024-09-12T03:20:41.386 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether pthreads work without any flags... yes 2024-09-12T03:20:41.431 INFO:tasks.workunit.client.0.smithi019.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-09-12T03:20:41.431 INFO:tasks.workunit.client.0.smithi019.stdout:checking if more special flags are required for pthreads... no 2024-09-12T03:20:41.431 INFO:tasks.workunit.client.0.smithi019.stdout:checking for cc_r... gcc 2024-09-12T03:20:41.513 INFO:tasks.workunit.client.0.smithi019.stdout:checking for setrlimit... yes 2024-09-12T03:20:41.574 INFO:tasks.workunit.client.0.smithi019.stdout:checking for setproctitle... no 2024-09-12T03:20:41.632 INFO:tasks.workunit.client.0.smithi019.stdout:checking for getopt_long... yes 2024-09-12T03:20:41.692 INFO:tasks.workunit.client.0.smithi019.stdout:checking for strtoull... yes 2024-09-12T03:20:41.759 INFO:tasks.workunit.client.0.smithi019.stdout:checking for strtoq... yes 2024-09-12T03:20:41.821 INFO:tasks.workunit.client.0.smithi019.stdout:checking for snprintf... yes 2024-09-12T03:20:41.882 INFO:tasks.workunit.client.0.smithi019.stdout:checking for vsnprintf... yes 2024-09-12T03:20:41.945 INFO:tasks.workunit.client.0.smithi019.stdout:checking for vfprintf... yes 2024-09-12T03:20:42.006 INFO:tasks.workunit.client.0.smithi019.stdout:checking for gethostname... yes 2024-09-12T03:20:42.100 INFO:tasks.workunit.client.0.smithi019.stdout:checking for setlocale... yes 2024-09-12T03:20:42.170 INFO:tasks.workunit.client.0.smithi019.stdout:checking for sync... yes 2024-09-12T03:20:42.235 INFO:tasks.workunit.client.0.smithi019.stdout:checking whether snprintf is C99 conformant... done 2024-09-12T03:20:42.245 INFO:tasks.workunit.client.0.smithi019.stdout:configure: creating ./config.status 2024-09-12T03:20:42.398 INFO:tasks.workunit.client.0.smithi019.stdout:config.status: creating Makefile 2024-09-12T03:20:42.415 INFO:tasks.workunit.client.0.smithi019.stdout:config.status: creating src/Makefile 2024-09-12T03:20:42.435 INFO:tasks.workunit.client.0.smithi019.stdout:config.status: creating man/Makefile 2024-09-12T03:20:42.447 INFO:tasks.workunit.client.0.smithi019.stdout:config.status: creating config.h 2024-09-12T03:20:42.496 INFO:tasks.workunit.client.0.smithi019.stdout:config.status: executing depfiles commands 2024-09-12T03:20:42.628 INFO:tasks.workunit.client.0.smithi019.stderr:+ make 2024-09-12T03:20:42.639 INFO:tasks.workunit.client.0.smithi019.stdout:make all-recursive 2024-09-12T03:20:42.641 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-12T03:20:42.644 INFO:tasks.workunit.client.0.smithi019.stdout:Making all in src 2024-09-12T03:20:42.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:42 smithi121 ceph-mon[34963]: pgmap v387: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 14 op/s 2024-09-12T03:20:42.650 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-12T03:20:42.650 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-09-12T03:20:42.650 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-09-12T03:20:42.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:42 smithi019 ceph-mon[30953]: pgmap v387: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 14 op/s 2024-09-12T03:20:42.754 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-09-12T03:20:42.754 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-09-12T03:20:42.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:42 smithi135 ceph-mon[35474]: pgmap v387: 97 pgs: 97 active+clean; 1.8 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 14 op/s 2024-09-12T03:20:42.822 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-09-12T03:20:42.822 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-09-12T03:20:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-09-12T03:20:42.878 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-09-12T03:20:42.931 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-09-12T03:20:42.931 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-09-12T03:20:42.983 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-09-12T03:20:42.983 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-09-12T03:20:43.037 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-09-12T03:20:43.037 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-09-12T03:20:43.125 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-09-12T03:20:43.125 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-09-12T03:20:43.161 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-09-12T03:20:43.161 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-09-12T03:20:43.196 INFO:tasks.workunit.client.0.smithi019.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-09-12T03:20:43.196 INFO:tasks.workunit.client.0.smithi019.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-09-12T03:20:43.237 INFO:tasks.workunit.client.0.smithi019.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-09-12T03:20:43.257 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-09-12T03:20:43.257 INFO:tasks.workunit.client.0.smithi019.stdout:Making all in man 2024-09-12T03:20:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-12T03:20:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Nothing to be done for 'all'. 2024-09-12T03:20:43.294 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-09-12T03:20:43.295 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-12T03:20:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-12T03:20:43.296 INFO:tasks.workunit.client.0.smithi019.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-09-12T03:20:43.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ cd src 2024-09-12T03:20:43.297 INFO:tasks.workunit.client.0.smithi019.stderr:+ mkdir blogtest_in 2024-09-12T03:20:43.299 INFO:tasks.workunit.client.0.smithi019.stdout:running blogbench 2024-09-12T03:20:43.299 INFO:tasks.workunit.client.0.smithi019.stderr:+ echo 'running blogbench' 2024-09-12T03:20:43.300 INFO:tasks.workunit.client.0.smithi019.stderr:+ ./blogbench -d blogtest_in 2024-09-12T03:20:43.300 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-09-12T03:20:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:Frequency = 10 secs 2024-09-12T03:20:43.300 INFO:tasks.workunit.client.0.smithi019.stdout:Scratch dir = [blogtest_in] 2024-09-12T03:20:44.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:44 smithi121 ceph-mon[34963]: pgmap v388: 97 pgs: 97 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 177 KiB/s wr, 24 op/s 2024-09-12T03:20:44.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:44 smithi019 ceph-mon[30953]: pgmap v388: 97 pgs: 97 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 177 KiB/s wr, 24 op/s 2024-09-12T03:20:44.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:44 smithi135 ceph-mon[35474]: pgmap v388: 97 pgs: 97 active+clean; 2.5 MiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 177 KiB/s wr, 24 op/s 2024-09-12T03:20:45.691 INFO:tasks.workunit.client.0.smithi019.stdout:Spawning 3 writers... 2024-09-12T03:20:45.691 INFO:tasks.workunit.client.0.smithi019.stdout:Spawning 1 rewriters... 2024-09-12T03:20:45.691 INFO:tasks.workunit.client.0.smithi019.stdout:Spawning 5 commenters... 2024-09-12T03:20:45.691 INFO:tasks.workunit.client.0.smithi019.stdout:Spawning 100 readers... 2024-09-12T03:20:45.694 INFO:tasks.workunit.client.0.smithi019.stdout:Benchmarking for 30 iterations. 2024-09-12T03:20:45.694 INFO:tasks.workunit.client.0.smithi019.stdout:The test will run during 5 minutes. 2024-09-12T03:20:45.694 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-09-12T03:20:45.694 INFO:tasks.workunit.client.0.smithi019.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-09-12T03:20:46.264 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:20:46.537 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:46.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:46 smithi121 ceph-mon[34963]: pgmap v389: 97 pgs: 97 active+clean; 4.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 354 KiB/s wr, 41 op/s 2024-09-12T03:20:46.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:46 smithi019 ceph-mon[30953]: pgmap v389: 97 pgs: 97 active+clean; 4.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 354 KiB/s wr, 41 op/s 2024-09-12T03:20:46.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:46 smithi135 ceph-mon[35474]: pgmap v389: 97 pgs: 97 active+clean; 4.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 354 KiB/s wr, 41 op/s 2024-09-12T03:20:47.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 -- 172.21.15.19:0/3819886496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 msgr2=0x7ffafc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 --2- 172.21.15.19:0/3819886496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc101200 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0009900 tx=0x7ffaf002a480 comp rx=0 tx=0).stop 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 -- 172.21.15.19:0/3819886496 shutdown_connections 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 --2- 172.21.15.19:0/3819886496 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 --2- 172.21.15.19:0/3819886496 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 --2- 172.21.15.19:0/3819886496 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc101200 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 -- 172.21.15.19:0/3819886496 >> 172.21.15.19:0/3819886496 conn(0x7ffafc0fc7d0 msgr2=0x7ffafc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 -- 172.21.15.19:0/3819886496 shutdown_connections 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.053+0000 7ffb0420a640 1 -- 172.21.15.19:0/3819886496 wait complete. 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 Processor -- start 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 -- start start 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc110f80 con 0x7ffafc100e20 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc110e00 con 0x7ffafc102570 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb0420a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc111100 con 0x7ffafc102ef0 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32964/0 (socket says 172.21.15.19:32964) 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1006850617 learned_addr learned my addr 172.21.15.19:0/1006850617 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb027a7640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.054+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1006850617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 msgr2=0x7ffafc1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1006850617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 msgr2=0x7ffafc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1006850617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc1a48f0 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb027a7640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0006010 tx=0x7ffaf0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffaeeffd640 1 -- 172.21.15.19:0/1006850617 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf003f070 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffaeeffd640 1 -- 172.21.15.19:0/1006850617 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf0031c30 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffaeeffd640 1 -- 172.21.15.19:0/1006850617 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0035030 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1a4ac0 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.055+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc1a4f10 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.057+0000 7ffaeeffd640 1 -- 172.21.15.19:0/1006850617 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffaf0042070 con 0x7ffafc100e20 2024-09-12T03:20:47.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 Processor -- start 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 -- start start 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaee380 0x55713aaee760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55713aa542e0 con 0x55713aaeeca0 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55713a9fbce0 con 0x55713aaee380 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb065e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55713aa502a0 con 0x55713aafda90 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffaeeffd640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 0x7ffae80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb017a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb027a7640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 0x7ffae80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb017a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55130/0 (socket says 172.21.15.19:55130) 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.058+0000 7ffb017a5640 1 -- 172.21.15.19:0/291804855 learned_addr learned my addr 172.21.15.19:0/291804855 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffaeeffd640 1 -- 172.21.15.19:0/1006850617 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffaf00ef740 con 0x7ffafc100e20 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 -- 172.21.15.19:0/291804855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 msgr2=0x55713aaffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 -- 172.21.15.19:0/291804855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaee380 msgr2=0x55713aaee760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaee380 0x55713aaee760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 -- 172.21.15.19:0/291804855 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55713a9e0440 con 0x55713aaeeca0 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb017a5640 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafd550 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bbf40 tx=0x7ffaf008f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2affff6d6de78f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffb027a7640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 0x7ffae80a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4005fd0 tx=0x7ffaf4009f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffae7fff640 1 -- 172.21.15.19:0/291804855 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0049070 con 0x55713aaeeca0 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffae7fff640 1 -- 172.21.15.19:0/291804855 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf00bc6b0 con 0x55713aaeeca0 2024-09-12T03:20:47.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.059+0000 7ffae7fff640 1 -- 172.21.15.19:0/291804855 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00c4c20 con 0x55713aaeeca0 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 -- 172.21.15.19:0/291804855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 msgr2=0x55713aafd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafd550 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bbf40 tx=0x7ffaf008f070 comp rx=0 tx=0).stop 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 -- 172.21.15.19:0/291804855 shutdown_connections 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713aaffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafd550 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 --2- 172.21.15.19:0/291804855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaee380 0x55713aaee760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 -- 172.21.15.19:0/291804855 >> 172.21.15.19:0/291804855 conn(0x55713aaeaa30 msgr2=0x55713aaeae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 -- 172.21.15.19:0/291804855 shutdown_connections 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.060+0000 7ffb065e3740 1 -- 172.21.15.19:0/291804855 wait complete. 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 -- start start 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 Processor -- start 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713aafab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55713ab00420 con 0x55713aaeeca0 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55713aa0f050 con 0x55713aafda90 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55713ab005a0 con 0x55713aaee380 2024-09-12T03:20:47.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb065e3740 10 client.0 _ll_register_callbacks cb 0x55713aa4f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb017a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55142/0 (socket says 172.21.15.19:55142) 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.061+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1411929196 learned_addr learned my addr 172.21.15.19:0/1411929196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb027a7640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713aafab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1411929196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 msgr2=0x55713aafab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713aafab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1411929196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 msgr2=0x55713aafa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1411929196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55713a7c53c0 con 0x55713aaee380 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:47.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00ed2a0 tx=0x7ffaf00c0360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c3f116a303d646 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0049040 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf00bc6b0 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffaf00b8620 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffaf00c4580 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb065e3740 10 client.25195 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55713ab00420 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.062+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf00bc6b0 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffaf00baa90 con 0x55713aaee380 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 10 client.25195 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 10 client.25195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 20 client.25195 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:47.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab0d0e0 0x55713ab0f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 4 client.25195 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55713aa61850 con 0x55713ab0d0e0 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab0d0e0 0x55713ab0f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab0d0e0 0x55713ab0f4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56539acbbca00dc server_cookie=ed57676ed53578ad in_seq=0 out_seq=0 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.063+0000 7ffae57fa640 10 client.25195 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.066+0000 7ffae57fa640 1 -- 172.21.15.19:0/1411929196 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55713aa61850 con 0x55713ab0d0e0 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.066+0000 7ffae57fa640 10 client.25195 handle_command_reply: tid=0 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.066+0000 7ffb065e3740 1 client.25195 shutdown 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 msgr2=0x55713aafd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00ed2a0 tx=0x7ffaf00c0360 comp rx=0 tx=0).stop 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 shutdown_connections 2024-09-12T03:20:47.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab0d0e0 0x55713ab0f4d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713aafab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaeeca0 0x55713aafa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffae57fa640 0 client.25195 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1411929196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaee380 0x55713aafd3a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 >> 172.21.15.19:0/1411929196 conn(0x55713aaeaa30 msgr2=0x55713aaf0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 shutdown_connections 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 1 -- 172.21.15.19:0/1411929196 wait complete. 2024-09-12T03:20:47.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.067+0000 7ffb065e3740 20 client.25195 trim_cache size 0 max 16384 2024-09-12T03:20:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.085+0000 7ffb065e3740 1 Processor -- start 2024-09-12T03:20:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.085+0000 7ffb065e3740 1 -- start start 2024-09-12T03:20:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaeeca0 0x55713ab14da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713ab152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1ab220 con 0x55713aaee380 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1acd60 con 0x55713aafda90 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb065e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1ab8f0 con 0x55713aaeeca0 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32986/0 (socket says 172.21.15.19:32986) 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1655531577 learned_addr learned my addr 172.21.15.19:0/1655531577 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb017a5640 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaeeca0 0x55713ab14da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1655531577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaeeca0 msgr2=0x55713ab14da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaeeca0 0x55713ab14da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1655531577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 msgr2=0x55713ab152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713ab152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 -- 172.21.15.19:0/1655531577 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55713aaea860 con 0x55713aaee380 2024-09-12T03:20:47.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.086+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bbcb0 tx=0x7ffaf00ba4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23135729095be28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffae77fe640 1 -- 172.21.15.19:0/1655531577 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00ea070 con 0x55713aaee380 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffae77fe640 1 -- 172.21.15.19:0/1655531577 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf00bc6b0 con 0x55713aaee380 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffae77fe640 1 -- 172.21.15.19:0/1655531577 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf0049040 con 0x55713aaee380 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 -- 172.21.15.19:0/1655531577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 msgr2=0x55713ab09680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bbcb0 tx=0x7ffaf00ba4c0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 -- 172.21.15.19:0/1655531577 shutdown_connections 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aafda90 0x55713ab152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aaeeca0 0x55713ab14da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.087+0000 7ffb065e3740 1 --2- 172.21.15.19:0/1655531577 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab09680 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.088+0000 7ffb065e3740 1 -- 172.21.15.19:0/1655531577 >> 172.21.15.19:0/1655531577 conn(0x55713aaeaa30 msgr2=0x55713aaf5d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.088+0000 7ffb065e3740 1 -- 172.21.15.19:0/1655531577 shutdown_connections 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.088+0000 7ffb065e3740 1 -- 172.21.15.19:0/1655531577 wait complete. 2024-09-12T03:20:47.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.088+0000 7ffb065e3740 1 -- start start 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.088+0000 7ffb065e3740 1 Processor -- start 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaeeca0 0x55713ab010d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713ab01610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1ac690 con 0x55713aaee380 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1abfc0 con 0x55713aaeeca0 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc1a9220 con 0x55713aafda90 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 10 client.0 _ll_register_callbacks cb 0x7ffaf00ba820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33000/0 (socket says 172.21.15.19:33000) 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 -- 172.21.15.19:0/276166820 learned_addr learned my addr 172.21.15.19:0/276166820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:47.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb027a7640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713ab01610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb017a5640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaeeca0 0x55713ab010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 -- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 msgr2=0x55713ab01610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713ab01610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 -- 172.21.15.19:0/276166820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaeeca0 msgr2=0x55713ab010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaeeca0 0x55713ab010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 -- 172.21.15.19:0/276166820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55713ab01c50 con 0x55713aaee380 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb01fa6640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bc170 tx=0x7ffaf00c4ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784320ca9d30dda3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf00ea070 con 0x55713aaee380 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffaf00baa90 con 0x55713aaee380 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.089+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaf00bc6b0 con 0x55713aaee380 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffaf0049040 con 0x55713aaee380 2024-09-12T03:20:47.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 10 client.15651 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55713ab01c50 con 0x55713aaee380 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaf00bc6b0 con 0x55713aaee380 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ffaf00b9bf0 con 0x55713aaee380 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 10 client.15651 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 20 client.15651 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab77ee0 0x55713ab7a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 4 client.15651 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55713aa5bd60 con 0x55713ab77ee0 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.090+0000 7ffb017a5640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab77ee0 0x55713ab7a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.091+0000 7ffb017a5640 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab77ee0 0x55713ab7a2d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc2cdd7a5a5189b7 server_cookie=803698ca7ffa1918 in_seq=0 out_seq=0 2024-09-12T03:20:47.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.091+0000 7ffae57fa640 10 client.15651 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.095+0000 7ffae57fa640 1 -- 172.21.15.19:0/276166820 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55713aa5bd60 con 0x55713ab77ee0 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.095+0000 7ffae57fa640 10 client.15651 handle_command_reply: tid=0 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.095+0000 7ffb065e3740 1 client.15651 shutdown 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.095+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 msgr2=0x55713ab05d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.095+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00bc170 tx=0x7ffaf00c4ca0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 shutdown_connections 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55713aafda90 0x55713ab01610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55713ab77ee0 0x55713ab7a2d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55713aaeeca0 0x55713ab010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 --2- 172.21.15.19:0/276166820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55713aaee380 0x55713ab05d50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 >> 172.21.15.19:0/276166820 conn(0x55713aaeaa30 msgr2=0x55713aae9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffae57fa640 0 client.15651 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 shutdown_connections 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 1 -- 172.21.15.19:0/276166820 wait complete. 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb065e3740 20 client.15651 trim_cache size 0 max 16384 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:20:47.097 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:20:47.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 msgr2=0x7ffae80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 0x7ffae80a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4005fd0 tx=0x7ffaf4009f50 comp rx=0 tx=0).stop 2024-09-12T03:20:47.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 msgr2=0x7ffafc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:47.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.096+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ffaf0006010 tx=0x7ffaf0003c10 comp rx=0 tx=0).stop 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 shutdown_connections 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffafc102ef0 0x7ffafc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffafc102570 0x7ffafc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffae80a2b80 0x7ffae80a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 --2- 172.21.15.19:0/1006850617 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffafc100e20 0x7ffafc199330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 >> 172.21.15.19:0/1006850617 conn(0x7ffafc0fc7d0 msgr2=0x7ffafc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 shutdown_connections 2024-09-12T03:20:47.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:47.097+0000 7ffb0420a640 1 -- 172.21.15.19:0/1006850617 wait complete. 2024-09-12T03:20:47.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:20:47.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:20:48.141 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:48.619 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:48 smithi019 ceph-mon[30953]: pgmap v390: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 413 KiB/s wr, 59 op/s 2024-09-12T03:20:48.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:48 smithi121 ceph-mon[34963]: pgmap v390: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 413 KiB/s wr, 59 op/s 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.704+0000 7f695cea7640 1 -- 172.21.15.19:0/616645883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69581036e0 msgr2=0x7f6958107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.704+0000 7f695cea7640 1 --2- 172.21.15.19:0/616645883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69581036e0 0x7f6958107b90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f694c009930 tx=0x7f694c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 -- 172.21.15.19:0/616645883 shutdown_connections 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 --2- 172.21.15.19:0/616645883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f695810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 --2- 172.21.15.19:0/616645883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f69581036e0 0x7f6958107b90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 --2- 172.21.15.19:0/616645883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6958102d30 0x7f6958103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 -- 172.21.15.19:0/616645883 >> 172.21.15.19:0/616645883 conn(0x7f69580fc7d0 msgr2=0x7f69580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 -- 172.21.15.19:0/616645883 shutdown_connections 2024-09-12T03:20:48.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.705+0000 7f695cea7640 1 -- 172.21.15.19:0/616645883 wait complete. 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 Processor -- start 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 -- start start 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 0x7f6958100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 0x7f6958100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f6958101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f69581120a0 con 0x7f6958102d30 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6958111f20 con 0x7f69581082a0 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.706+0000 7f695cea7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6958112220 con 0x7f69581036e0 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f695659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 0x7f6958100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f695659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 0x7f6958100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33006/0 (socket says 172.21.15.19:33006) 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f695659c640 1 -- 172.21.15.19:0/3183808514 learned_addr learned my addr 172.21.15.19:0/3183808514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:48.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 0x7f6958100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6956d9d640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f6958101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 -- 172.21.15.19:0/3183808514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 msgr2=0x7f6958101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f6958101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 -- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 msgr2=0x7f6958100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 0x7f6958100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 -- 172.21.15.19:0/3183808514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581a6a40 con 0x7f69581036e0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6956d9d640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f6958101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.707+0000 7f6955d9b640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 0x7f6958100ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f694c02a940 tx=0x7f694c0040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.708+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c03f070 con 0x7f69581036e0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.708+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f694c0044c0 con 0x7f69581036e0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.708+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694c035070 con 0x7f69581036e0 2024-09-12T03:20:48.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.709+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69581a6cd0 con 0x7f69581036e0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.709+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a7130 con 0x7f69581036e0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.710+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6958103110 con 0x7f69581036e0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.712+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f694c03b460 con 0x7f69581036e0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.713+0000 7f69477fe640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.713+0000 7f695659c640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.713+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f694c0efba0 con 0x7f69581036e0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.714+0000 7f695659c640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 0x7f69400a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6948009cb0 tx=0x7f6948005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:48.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.717+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f694c048070 con 0x7f69581036e0 2024-09-12T03:20:48.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:48 smithi135 ceph-mon[35474]: pgmap v390: 97 pgs: 97 active+clean; 4.5 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 244 KiB/s rd, 413 KiB/s wr, 59 op/s 2024-09-12T03:20:48.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.947+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f695810a2b0 con 0x7f69581036e0 2024-09-12T03:20:48.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.949+0000 7f69477fe640 1 -- 172.21.15.19:0/3183808514 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11974 (secure 0 0 0) 0x7f694c0bcb10 con 0x7f69581036e0 2024-09-12T03:20:48.949 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:20:48.950 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":28,"btime":"2024-09-12T03:19:13:839126+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:19:13.230767+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:20:48.950 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 28 2024-09-12T03:20:48.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 msgr2=0x7f69400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 0x7f69400a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6948009cb0 tx=0x7f6948005f90 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 msgr2=0x7f6958100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 0x7f6958100ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f694c02a940 tx=0x7f694c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 shutdown_connections 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f69581082a0 0x7f6958101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f69581036e0 0x7f6958100ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69400a2c00 0x7f69400a50c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 --2- 172.21.15.19:0/3183808514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6958102d30 0x7f6958100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 >> 172.21.15.19:0/3183808514 conn(0x7f69580fc7d0 msgr2=0x7f6958105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 shutdown_connections 2024-09-12T03:20:48.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:48.951+0000 7f695cea7640 1 -- 172.21.15.19:0/3183808514 wait complete. 2024-09-12T03:20:49.659 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:49 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3183808514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:49.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:20:49.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:49 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3183808514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:49.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:49 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3183808514' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:20:49.958 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 -- 172.21.15.19:0/4193077261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 msgr2=0x7f7f5c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 --2- 172.21.15.19:0/4193077261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c103110 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009930 tx=0x7f7f4c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 -- 172.21.15.19:0/4193077261 shutdown_connections 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 --2- 172.21.15.19:0/4193077261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 --2- 172.21.15.19:0/4193077261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 --2- 172.21.15.19:0/4193077261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c103110 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 -- 172.21.15.19:0/4193077261 >> 172.21.15.19:0/4193077261 conn(0x7f7f5c0fc7d0 msgr2=0x7f7f5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 -- 172.21.15.19:0/4193077261 shutdown_connections 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.477+0000 7f7f617ec640 1 -- 172.21.15.19:0/4193077261 wait complete. 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 Processor -- start 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 -- start start 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1120a0 con 0x7f7f5c102d30 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c111f20 con 0x7f7f5c1036e0 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f617ec640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c112220 con 0x7f7f5c1082a0 2024-09-12T03:20:50.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.478+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40866/0 (socket says 172.21.15.19:40866) 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 -- 172.21.15.19:0/3167353462 learned_addr learned my addr 172.21.15.19:0/3167353462 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 -- 172.21.15.19:0/3167353462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 msgr2=0x7f7f5c19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 -- 172.21.15.19:0/3167353462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 msgr2=0x7f7f5c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 -- 172.21.15.19:0/3167353462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c19d570 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f5affd640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009230 tx=0x7f7f4c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f53fff640 1 -- 172.21.15.19:0/3167353462 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c03f030 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c19d800 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f53fff640 1 -- 172.21.15.19:0/3167353462 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c003ad0 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f53fff640 1 -- 172.21.15.19:0/3167353462 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c035070 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.479+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c1a5140 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.481+0000 7f7f53fff640 1 -- 172.21.15.19:0/3167353462 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f4c042030 con 0x7f7f5c102d30 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 Processor -- start 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 -- start start 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574d5d0 0x556a3574da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a3575a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a356b32f0 con 0x556a3574eb40 2024-09-12T03:20:50.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a3565acf0 con 0x556a3574d5d0 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556a356af2b0 con 0x556a35758370 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40880/0 (socket says 172.21.15.19:40880) 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f5b7fe640 1 -- 172.21.15.19:0/3817014447 learned_addr learned my addr 172.21.15.19:0/3817014447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f53fff640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 0x7f7f480a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.482+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a3575a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5affd640 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574d5d0 0x556a3574da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 0x7f7f480a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 -- 172.21.15.19:0/3817014447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 msgr2=0x556a3575a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f53fff640 1 -- 172.21.15.19:0/3167353462 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c0ef650 con 0x7f7f5c102d30 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a3575a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 -- 172.21.15.19:0/3817014447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574d5d0 msgr2=0x556a3574da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574d5d0 0x556a3574da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 -- 172.21.15.19:0/3817014447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a3563f3a0 con 0x556a3574eb40 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009fd0 tx=0x7f7f5400f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b68799582a1b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f517fa640 1 -- 172.21.15.19:0/3817014447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54023020 con 0x556a3574eb40 2024-09-12T03:20:50.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f517fa640 1 -- 172.21.15.19:0/3817014447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f540138a0 con 0x556a3574eb40 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.483+0000 7f7f517fa640 1 -- 172.21.15.19:0/3817014447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5401e720 con 0x556a3574eb40 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- 172.21.15.19:0/3817014447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 msgr2=0x556a3574d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7f54009fd0 tx=0x7f7f5400f7d0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 0x7f7f480a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f54024000 tx=0x7f7f54022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- 172.21.15.19:0/3817014447 shutdown_connections 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a3575a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574d5d0 0x556a3574da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/3817014447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574eb40 0x556a3574d090 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- 172.21.15.19:0/3817014447 >> 172.21.15.19:0/3817014447 conn(0x556a35716310 msgr2=0x556a35749c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- 172.21.15.19:0/3817014447 shutdown_connections 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- 172.21.15.19:0/3817014447 wait complete. 2024-09-12T03:20:50.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 -- start start 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.484+0000 7f7f63bc5740 1 Processor -- start 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35755010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 0x556a35755600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556a3575f480 con 0x556a35758370 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556a3566e060 con 0x556a3574eb40 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556a3575f600 con 0x556a3574d5d0 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f63bc5740 10 client.0 _ll_register_callbacks cb 0x556a353c18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 0x556a35755600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60508/0 (socket says 172.21.15.19:60508) 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 -- 172.21.15.19:0/1582422796 learned_addr learned my addr 172.21.15.19:0/1582422796 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35755010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 -- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 msgr2=0x556a35755010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35755010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 -- 172.21.15.19:0/1582422796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 msgr2=0x556a35755600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 0x556a35755600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5affd640 1 -- 172.21.15.19:0/1582422796 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a354243c0 con 0x556a3574d5d0 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.485+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 0x556a35755600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f5affd640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bbe50 tx=0x7f7f4c08ae40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ceb9a63ec446f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c049070 con 0x556a3574d5d0 2024-09-12T03:20:50.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c03bd40 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556a356c0860 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c0bb820 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 10 client.25219 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a3575f480 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556a356c0860 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f4c0b8530 con 0x556a3574d5d0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 10 client.25219 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 10 client.25219 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 20 client.25219 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a3576c2a0 0x556a3576e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 4 client.25219 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.486+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a356bad70 con 0x556a3576c2a0 2024-09-12T03:20:50.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.487+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a3576c2a0 0x556a3576e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.487+0000 7f7f427fc640 10 client.25219 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.487+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a3576c2a0 0x556a3576e6f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d5da398a7ca5e11 server_cookie=aac43d76482fe276 in_seq=0 out_seq=0 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f427fc640 1 -- 172.21.15.19:0/1582422796 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556a356bad70 con 0x556a3576c2a0 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f427fc640 10 client.25219 handle_command_reply: tid=0 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 client.25219 shutdown 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 msgr2=0x556a35757d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bbe50 tx=0x7f7f4c08ae40 comp rx=0 tx=0).stop 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 shutdown_connections 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a35758370 0x556a35755600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35755010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a3576c2a0 0x556a3576e6f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/1582422796 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574d5d0 0x556a35757d60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f427fc640 0 client.25219 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 >> 172.21.15.19:0/1582422796 conn(0x556a35716310 msgr2=0x556a3574f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 shutdown_connections 2024-09-12T03:20:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 1 -- 172.21.15.19:0/1582422796 wait complete. 2024-09-12T03:20:50.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.488+0000 7f7f63bc5740 20 client.25219 trim_cache size 0 max 16384 2024-09-12T03:20:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.502+0000 7f7f63bc5740 1 Processor -- start 2024-09-12T03:20:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.502+0000 7f7f63bc5740 1 -- start start 2024-09-12T03:20:50.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.502+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.502+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35750120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a35750660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ab260 con 0x556a3574d5d0 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1acda0 con 0x556a3574eb40 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ab930 con 0x556a35758370 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40892/0 (socket says 172.21.15.19:40892) 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 -- 172.21.15.19:0/234364976 learned_addr learned my addr 172.21.15.19:0/234364976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a35750660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35750120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 -- 172.21.15.19:0/234364976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 msgr2=0x556a35750660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a35750660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 -- 172.21.15.19:0/234364976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 msgr2=0x556a35750120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35750120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 -- 172.21.15.19:0/234364976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a35757980 con 0x556a3574d5d0 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35750120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f5affd640 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0ed4d0 tx=0x7f7f4c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874545fe0cd98bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f50ff9640 1 -- 172.21.15.19:0/234364976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c049070 con 0x556a3574d5d0 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f50ff9640 1 -- 172.21.15.19:0/234364976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c03b460 con 0x556a3574d5d0 2024-09-12T03:20:50.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.503+0000 7f7f50ff9640 1 -- 172.21.15.19:0/234364976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0c46a0 con 0x556a3574d5d0 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 -- 172.21.15.19:0/234364976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 msgr2=0x556a357548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0ed4d0 tx=0x7f7f4c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 -- 172.21.15.19:0/234364976 shutdown_connections 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a35758370 0x556a35750660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a3574eb40 0x556a35750120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/234364976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a357548f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 -- 172.21.15.19:0/234364976 >> 172.21.15.19:0/234364976 conn(0x556a35716310 msgr2=0x556a35757090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 -- 172.21.15.19:0/234364976 shutdown_connections 2024-09-12T03:20:50.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.504+0000 7f7f63bc5740 1 -- 172.21.15.19:0/234364976 wait complete. 2024-09-12T03:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 -- start start 2024-09-12T03:20:50.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 Processor -- start 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574eb40 0x556a35760350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a35758370 0x556a357d4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ac6d0 con 0x556a3574d5d0 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1ac000 con 0x556a35758370 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c1a9240 con 0x556a3574eb40 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f63bc5740 10 client.0 _ll_register_callbacks cb 0x7f7f4c039f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f5affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40904/0 (socket says 172.21.15.19:40904) 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f5affd640 1 -- 172.21.15.19:0/861706454 learned_addr learned my addr 172.21.15.19:0/861706454 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:50.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.505+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574eb40 0x556a35760350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5b7fe640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a35758370 0x556a357d4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 -- 172.21.15.19:0/861706454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574eb40 msgr2=0x556a35760350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574eb40 0x556a35760350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 -- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a35758370 msgr2=0x556a357d4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a35758370 0x556a357d4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 -- 172.21.15.19:0/861706454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a357d54c0 con 0x556a3574d5d0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f5affd640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bbbc0 tx=0x7f7f4c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced3d0d0ad187269 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c049070 con 0x556a3574d5d0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7f4c0c46a0 con 0x556a3574d5d0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556a35637170 con 0x556a3574d5d0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c0a0050 con 0x556a3574d5d0 2024-09-12T03:20:50.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f63bc5740 10 client.15675 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556a357d54c0 con 0x556a3574d5d0 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x556a35637170 con 0x556a3574d5d0 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.506+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7f4c0bb310 con 0x556a3574d5d0 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 10 client.15675 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 10 client.15675 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 20 client.15675 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a357d6a80 0x556a357d8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 4 client.15675 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a356849b0 con 0x556a357d6a80 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a357d6a80 0x556a357d8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f5a7fc640 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a357d6a80 0x556a357d8e70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59dd89035727c84a server_cookie=4ce44dad0dbf507a in_seq=0 out_seq=0 2024-09-12T03:20:50.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f427fc640 10 client.15675 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f427fc640 1 -- 172.21.15.19:0/861706454 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a356849b0 con 0x556a357d6a80 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f427fc640 10 client.15675 handle_command_reply: tid=0 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 1 client.15675 shutdown 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 msgr2=0x556a3575fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.507+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0bbbc0 tx=0x7f7f4c009660 comp rx=0 tx=0).stop 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 shutdown_connections 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556a35758370 0x556a357d4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x556a357d6a80 0x556a357d8e70 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556a3574eb40 0x556a35760350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 --2- 172.21.15.19:0/861706454 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556a3574d5d0 0x556a3575fe10 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 >> 172.21.15.19:0/861706454 conn(0x556a35716310 msgr2=0x556a357484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f427fc640 0 client.15675 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:20:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 shutdown_connections 2024-09-12T03:20:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 1 -- 172.21.15.19:0/861706454 wait complete. 2024-09-12T03:20:50.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f63bc5740 20 client.15675 trim_cache size 0 max 16384 2024-09-12T03:20:50.515 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 msgr2=0x7f7f480a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 0x7f7f480a5040 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7f54024000 tx=0x7f7f54022000 comp rx=0 tx=0).stop 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 msgr2=0x7f7f5c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009230 tx=0x7f7f4c004270 comp rx=0 tx=0).stop 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 shutdown_connections 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7f5c1082a0 0x7f7f5c19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c1036e0 0x7f7f5c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7f480a2b80 0x7f7f480a5040 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 --2- 172.21.15.19:0/3167353462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7f5c102d30 0x7f7f5c1985c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.508+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 >> 172.21.15.19:0/3167353462 conn(0x7f7f5c0fc7d0 msgr2=0x7f7f5c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.509+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 shutdown_connections 2024-09-12T03:20:50.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:50.509+0000 7f7f617ec640 1 -- 172.21.15.19:0/3167353462 wait complete. 2024-09-12T03:20:50.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:50 smithi019 ceph-mon[30953]: pgmap v391: 97 pgs: 97 active+clean; 9.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 823 KiB/s wr, 64 op/s 2024-09-12T03:20:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:50 smithi135 ceph-mon[35474]: pgmap v391: 97 pgs: 97 active+clean; 9.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 823 KiB/s wr, 64 op/s 2024-09-12T03:20:50.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:50 smithi121 ceph-mon[34963]: pgmap v391: 97 pgs: 97 active+clean; 9.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 823 KiB/s wr, 64 op/s 2024-09-12T03:20:51.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:20:51.662 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:52.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 -- 172.21.15.19:0/1060712711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102ef0 msgr2=0x7f14d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 --2- 172.21.15.19:0/1060712711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102ef0 0x7f14d810da40 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f14d0009900 tx=0x7f14d002a480 comp rx=0 tx=0).stop 2024-09-12T03:20:52.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 -- 172.21.15.19:0/1060712711 shutdown_connections 2024-09-12T03:20:52.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 --2- 172.21.15.19:0/1060712711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102ef0 0x7f14d810da40 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 --2- 172.21.15.19:0/1060712711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8102570 0x7f14d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 --2- 172.21.15.19:0/1060712711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8100e20 0x7f14d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.183+0000 7f14dce45640 1 -- 172.21.15.19:0/1060712711 >> 172.21.15.19:0/1060712711 conn(0x7f14d80fc7d0 msgr2=0x7f14d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.184+0000 7f14dce45640 1 -- 172.21.15.19:0/1060712711 shutdown_connections 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.184+0000 7f14dce45640 1 -- 172.21.15.19:0/1060712711 wait complete. 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.184+0000 7f14dce45640 1 Processor -- start 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.184+0000 7f14dce45640 1 -- start start 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8100e20 0x7f14d8197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8102ef0 0x7f14d819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8110f80 con 0x7f14d8102570 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8110e00 con 0x7f14d8100e20 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14dce45640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14d8111100 con 0x7f14d8102ef0 2024-09-12T03:20:52.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40924/0 (socket says 172.21.15.19:40924) 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 -- 172.21.15.19:0/4238049433 learned_addr learned my addr 172.21.15.19:0/4238049433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8102ef0 0x7f14d819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d659c640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8100e20 0x7f14d8197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 -- 172.21.15.19:0/4238049433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8102ef0 msgr2=0x7f14d819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8102ef0 0x7f14d819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 -- 172.21.15.19:0/4238049433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8100e20 msgr2=0x7f14d8197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8100e20 0x7f14d8197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 -- 172.21.15.19:0/4238049433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d819c410 con 0x7f14d8102570 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.185+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00d950 tx=0x7f14cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.186+0000 7f14c77fe640 1 -- 172.21.15.19:0/4238049433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc017070 con 0x7f14d8102570 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.186+0000 7f14c77fe640 1 -- 172.21.15.19:0/4238049433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14cc009c30 con 0x7f14d8102570 2024-09-12T03:20:52.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.186+0000 7f14c77fe640 1 -- 172.21.15.19:0/4238049433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc00f6b0 con 0x7f14d8102570 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.186+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d819c700 con 0x7f14d8102570 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.186+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d81a3fe0 con 0x7f14d8102570 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14c77fe640 1 -- 172.21.15.19:0/4238049433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14cc0109d0 con 0x7f14d8102570 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 Processor -- start 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 -- start start 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 0x561107fb2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fc1970 0x561107fc3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561107f182f0 con 0x561107fb3a20 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561107ebfcf0 con 0x561107fc1970 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.189+0000 7f14df21e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561107f142b0 con 0x561107fb25d0 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40926/0 (socket says 172.21.15.19:40926) 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 0x561107fb2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d6d9d640 1 -- 172.21.15.19:0/3423201000 learned_addr learned my addr 172.21.15.19:0/3423201000 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14c77fe640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 0x7f14c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fc1970 0x561107fc3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 -- 172.21.15.19:0/3423201000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fc1970 msgr2=0x561107fc3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fc1970 0x561107fc3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 -- 172.21.15.19:0/3423201000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 msgr2=0x561107fb2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 -- 172.21.15.19:0/3423201000 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561107ea43a0 con 0x561107fb25d0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14c77fe640 1 -- 172.21.15.19:0/4238049433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f14cc0ceb10 con 0x7f14d8102570 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 0x7f14c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.190+0000 7f14d659c640 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 0x561107fb2a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f80 tx=0x7f14c80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f33ed5dd781263 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14c4ff9640 1 -- 172.21.15.19:0/3423201000 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004d20 con 0x561107fb25d0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14c4ff9640 1 -- 172.21.15.19:0/3423201000 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8004050 con 0x561107fb25d0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 0x7f14c00a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005c40 tx=0x7f14d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14c4ff9640 1 -- 172.21.15.19:0/3423201000 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c8005260 con 0x561107fb25d0 2024-09-12T03:20:52.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 -- 172.21.15.19:0/3423201000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 msgr2=0x561107fb2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 0x561107fb2a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009f80 tx=0x7f14c80118d0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 -- 172.21.15.19:0/3423201000 shutdown_connections 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fc1970 0x561107fc3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fb25d0 0x561107fb2a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 --2- 172.21.15.19:0/3423201000 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fb2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.191+0000 7f14df21e740 1 -- 172.21.15.19:0/3423201000 >> 172.21.15.19:0/3423201000 conn(0x561107f7b310 msgr2=0x561107faec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- 172.21.15.19:0/3423201000 shutdown_connections 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- 172.21.15.19:0/3423201000 wait complete. 2024-09-12T03:20:52.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- start start 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 Processor -- start 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fc1970 0x561107fbea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561107fc4540 con 0x561107fb3a20 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561107ed3060 con 0x561107fb25d0 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561107fc46c0 con 0x561107fc1970 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14df21e740 10 client.0 _ll_register_callbacks cb 0x561107c268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:52.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40942/0 (socket says 172.21.15.19:40942) 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.192+0000 7f14d5d9b640 1 -- 172.21.15.19:0/3169892184 learned_addr learned my addr 172.21.15.19:0/3169892184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fc1970 0x561107fbea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d659c640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 -- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fc1970 msgr2=0x561107fbea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fc1970 0x561107fbea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 -- 172.21.15.19:0/3169892184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 msgr2=0x561107fc1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fc1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 -- 172.21.15.19:0/3169892184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561107c893c0 con 0x561107fb3a20 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f14cc097890 tx=0x7f14cc010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9304bbbfeb73e7d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc020050 con 0x561107fb3a20 2024-09-12T03:20:52.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f14cc09f130 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8004050 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14cc09f730 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14df21e740 10 client.15687 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561107fcff00 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.193+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14c8004050 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14cc0b4140 con 0x561107fb3a20 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 10 client.15687 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 10 client.15687 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 20 client.15687 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fd13c0 0x561107fd37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 4 client.15687 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561107f25860 con 0x561107fd13c0 2024-09-12T03:20:52.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14d659c640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fd13c0 0x561107fd37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14d659c640 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fd13c0 0x561107fd37b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15d62587651b87b8 server_cookie=a9ea1072e41655b9 in_seq=0 out_seq=0 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.194+0000 7f14bdffb640 10 client.15687 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14bdffb640 1 -- 172.21.15.19:0/3169892184 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561107f25860 con 0x561107fd13c0 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14bdffb640 10 client.15687 handle_command_reply: tid=0 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 client.15687 shutdown 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 msgr2=0x561107fbe540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f14cc097890 tx=0x7f14cc010d70 comp rx=0 tx=0).stop 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 shutdown_connections 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fd13c0 0x561107fd37b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107fc1970 0x561107fbea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fbe540 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14bdffb640 0 client.15687 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 --2- 172.21.15.19:0/3169892184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fc1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 >> 172.21.15.19:0/3169892184 conn(0x561107f7b310 msgr2=0x561107fb49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 shutdown_connections 2024-09-12T03:20:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.195+0000 7f14df21e740 1 -- 172.21.15.19:0/3169892184 wait complete. 2024-09-12T03:20:52.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.196+0000 7f14df21e740 20 client.15687 trim_cache size 0 max 16384 2024-09-12T03:20:52.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 Processor -- start 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 -- start start 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fcd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ab300 con 0x561107fb3a20 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ace40 con 0x561107fb25d0 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14df21e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ab9d0 con 0x561107f7b310 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14d659c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.209+0000 7f14d6d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40958/0 (socket says 172.21.15.19:40958) 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 -- 172.21.15.19:0/2083048345 learned_addr learned my addr 172.21.15.19:0/2083048345 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:52.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fcd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 -- 172.21.15.19:0/2083048345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 msgr2=0x561107fbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fbc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 -- 172.21.15.19:0/2083048345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 msgr2=0x561107fcd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fcd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 -- 172.21.15.19:0/2083048345 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561107fae4f0 con 0x561107fb3a20 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fcd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f14d0033000 tx=0x7f14d0040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68069edb7535f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14bffff640 1 -- 172.21.15.19:0/2083048345 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0035040 con 0x561107fb3a20 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14bffff640 1 -- 172.21.15.19:0/2083048345 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d004e070 con 0x561107fb3a20 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.210+0000 7f14bffff640 1 -- 172.21.15.19:0/2083048345 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d0053a10 con 0x561107fb3a20 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- 172.21.15.19:0/2083048345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 msgr2=0x561107fc2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f14d0033000 tx=0x7f14d0040000 comp rx=0 tx=0).stop 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- 172.21.15.19:0/2083048345 shutdown_connections 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb3a20 0x561107fc2460 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb25d0 0x561107fcd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 --2- 172.21.15.19:0/2083048345 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- 172.21.15.19:0/2083048345 >> 172.21.15.19:0/2083048345 conn(0x561107fb3460 msgr2=0x561107fc1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- 172.21.15.19:0/2083048345 shutdown_connections 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- 172.21.15.19:0/2083048345 wait complete. 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 -- start start 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.211+0000 7f14df21e740 1 Processor -- start 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fd98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 0x561107fc9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ac770 con 0x561107fb25d0 2024-09-12T03:20:52.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81ac0a0 con 0x561107fb3a20 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f14d81a92e0 con 0x561107f7b310 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14df21e740 10 client.0 _ll_register_callbacks cb 0x7f14cc00df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40962/0 (socket says 172.21.15.19:40962) 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 -- 172.21.15.19:0/1298668762 learned_addr learned my addr 172.21.15.19:0/1298668762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d659c640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fd98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 0x561107fc9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 -- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 msgr2=0x561107fd98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fd98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 -- 172.21.15.19:0/1298668762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 msgr2=0x561107fc9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 0x561107fc9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 -- 172.21.15.19:0/1298668762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561107fb0520 con 0x561107fb25d0 2024-09-12T03:20:52.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d6d9d640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 0x561107fc9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14d5d9b640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f14cc09a900 tx=0x7f14cc010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffbeb8c885e61f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc020070 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.212+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f14cc0ccae0 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14d004e070 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f14cc09af70 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 10 client.15693 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56110803c2f0 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14d004e070 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f14cc09a2f0 con 0x561107fb25d0 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 10 client.15693 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 10 client.15693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:20:52.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 20 client.15693 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fc1970 0x56110803ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 4 client.15693 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561107f1fd70 con 0x561107fc1970 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.213+0000 7f14d659c640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fc1970 0x56110803ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14d659c640 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fc1970 0x56110803ea20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=277eddf3d6406eb2 server_cookie=2fec474dc2ca3b01 in_seq=0 out_seq=0 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14bdffb640 10 client.15693 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14bdffb640 1 -- 172.21.15.19:0/1298668762 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561107f1fd70 con 0x561107fc1970 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14bdffb640 10 client.15693 handle_command_reply: tid=0 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 client.15693 shutdown 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 msgr2=0x561107fc9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f14cc09a900 tx=0x7f14cc010cc0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 shutdown_connections 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x561107fc1970 0x56110803ea20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561107fb3a20 0x561107fc9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561107fb25d0 0x561107fc9910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 --2- 172.21.15.19:0/1298668762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561107f7b310 0x561107fd98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 >> 172.21.15.19:0/1298668762 conn(0x561107fb3460 msgr2=0x561107fc2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14bdffb640 0 client.15693 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 shutdown_connections 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 1 -- 172.21.15.19:0/1298668762 wait complete. 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.214+0000 7f14df21e740 20 client.15693 trim_cache size 0 max 16384 2024-09-12T03:20:52.229 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 msgr2=0x7f14c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 0x7f14c00a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f14d0005c40 tx=0x7f14d0009260 comp rx=0 tx=0).stop 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 msgr2=0x7f14d81979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f14cc00d950 tx=0x7f14cc00de10 comp rx=0 tx=0).stop 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 shutdown_connections 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f14d8102ef0 0x7f14d819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f14c00a2b80 0x7f14c00a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f14d8102570 0x7f14d81979a0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 --2- 172.21.15.19:0/4238049433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f14d8100e20 0x7f14d8197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 >> 172.21.15.19:0/4238049433 conn(0x7f14d80fc7d0 msgr2=0x7f14d81068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 shutdown_connections 2024-09-12T03:20:52.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:52.215+0000 7f14dce45640 1 -- 172.21.15.19:0/4238049433 wait complete. 2024-09-12T03:20:52.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:52 smithi019 ceph-mon[30953]: pgmap v392: 97 pgs: 97 active+clean; 17 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-12T03:20:52.702 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:52 smithi121 ceph-mon[34963]: pgmap v392: 97 pgs: 97 active+clean; 17 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-12T03:20:52.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:52 smithi135 ceph-mon[35474]: pgmap v392: 97 pgs: 97 active+clean; 17 MiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-09-12T03:20:53.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:20:53.291 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:53.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 -- 172.21.15.19:0/563334686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13100ffb60 msgr2=0x7f13100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 --2- 172.21.15.19:0/563334686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13100ffb60 0x7f13100fffc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1304009900 tx=0x7f130402a410 comp rx=0 tx=0).stop 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 -- 172.21.15.19:0/563334686 shutdown_connections 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 --2- 172.21.15.19:0/563334686 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310100500 0x7f131010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 --2- 172.21.15.19:0/563334686 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f13100ffb60 0x7f13100fffc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 --2- 172.21.15.19:0/563334686 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1310106b80 0x7f1310106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 -- 172.21.15.19:0/563334686 >> 172.21.15.19:0/563334686 conn(0x7f13100fba90 msgr2=0x7f13100fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 -- 172.21.15.19:0/563334686 shutdown_connections 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.805+0000 7f13152b8640 1 -- 172.21.15.19:0/563334686 wait complete. 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 Processor -- start 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 -- start start 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13100ffb60 0x7f13101015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310106b80 0x7f1310102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111f60 con 0x7f1310100500 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1310111de0 con 0x7f1310106b80 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f13152b8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13101120e0 con 0x7f13100ffb60 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40988/0 (socket says 172.21.15.19:40988) 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 -- 172.21.15.19:0/2449508473 learned_addr learned my addr 172.21.15.19:0/2449508473 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130effd640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13100ffb60 0x7f13101015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310106b80 0x7f1310102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 -- 172.21.15.19:0/2449508473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13100ffb60 msgr2=0x7f13101015d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13100ffb60 0x7f13101015d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 -- 172.21.15.19:0/2449508473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310106b80 msgr2=0x7f1310102050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310106b80 0x7f1310102050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.806+0000 7f130e7fc640 1 -- 172.21.15.19:0/2449508473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13101a6900 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f1304003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f12f7fff640 1 -- 172.21.15.19:0/2449508473 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130403f070 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f12f7fff640 1 -- 172.21.15.19:0/2449508473 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1304031c30 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13101a6b90 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f12f7fff640 1 -- 172.21.15.19:0/2449508473 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304035030 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.807+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13101a70c0 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.809+0000 7f12f7fff640 1 -- 172.21.15.19:0/2449508473 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1304042070 con 0x7f1310100500 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 Processor -- start 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 -- start start 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34e5d0 0x5611eb34ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb35b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611eb2b42f0 con 0x5611eb34fb40 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5611eb25bcf0 con 0x5611eb359370 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f1317691740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5611eb2b02b0 con 0x5611eb34e5d0 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f130f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40996/0 (socket says 172.21.15.19:40996) 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f130f7fe640 1 -- 172.21.15.19:0/2619628295 learned_addr learned my addr 172.21.15.19:0/2619628295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb35b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f130effd640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34e5d0 0x5611eb34ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.810+0000 7f12f7fff640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 0x7f12f80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 -- 172.21.15.19:0/2619628295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34e5d0 msgr2=0x5611eb34ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34e5d0 0x5611eb34ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 -- 172.21.15.19:0/2619628295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 msgr2=0x5611eb35b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb35b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 -- 172.21.15.19:0/2619628295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611eb2403a0 con 0x5611eb34fb40 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb35b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f12f7fff640 1 -- 172.21.15.19:0/2449508473 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13040ef690 con 0x7f1310100500 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 0x7f12f80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1308011590 tx=0x7f1308011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd2ebc4f19e2f352 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f12f57fa640 1 -- 172.21.15.19:0/2619628295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308017050 con 0x5611eb34fb40 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f12f57fa640 1 -- 172.21.15.19:0/2619628295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130801bb90 con 0x5611eb34fb40 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.811+0000 7f12f57fa640 1 -- 172.21.15.19:0/2619628295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130800e5b0 con 0x5611eb34fb40 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- 172.21.15.19:0/2619628295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 msgr2=0x5611eb34e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 0x7f12f80a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1308022000 tx=0x7f1308023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1308011590 tx=0x7f1308011a50 comp rx=0 tx=0).stop 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- 172.21.15.19:0/2619628295 shutdown_connections 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb35b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34e5d0 0x5611eb34ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 --2- 172.21.15.19:0/2619628295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb34e090 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- 172.21.15.19:0/2619628295 >> 172.21.15.19:0/2619628295 conn(0x5611eb317310 msgr2=0x5611eb34ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- 172.21.15.19:0/2619628295 shutdown_connections 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- 172.21.15.19:0/2619628295 wait complete. 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 -- start start 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.812+0000 7f1317691740 1 Processor -- start 2024-09-12T03:20:53.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34e5d0 0x5611eb358d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb356610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5611eb35c070 con 0x5611eb34fb40 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5611eb26f060 con 0x5611eb34e5d0 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5611eb35c1f0 con 0x5611eb359370 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f1317691740 10 client.0 _ll_register_callbacks cb 0x5611eafc28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130effd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34e5d0 0x5611eb358d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41000/0 (socket says 172.21.15.19:41000) 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 -- 172.21.15.19:0/2674460830 learned_addr learned my addr 172.21.15.19:0/2674460830 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130f7fe640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb356610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 -- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 msgr2=0x5611eb356610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb356610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 -- 172.21.15.19:0/2674460830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34e5d0 msgr2=0x5611eb358d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34e5d0 0x5611eb358d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 -- 172.21.15.19:0/2674460830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611eb0253c0 con 0x5611eb34fb40 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f130e7fc640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c91cca33df641e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.813+0000 7f12f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1304049070 con 0x5611eb34fb40 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13040ed250 con 0x5611eb34fb40 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f130801bb90 con 0x5611eb34fb40 2024-09-12T03:20:53.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f13040c0250 con 0x5611eb34fb40 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 10 client.15705 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611eb367b40 con 0x5611eb34fb40 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f130801bb90 con 0x5611eb34fb40 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f13040bc1b0 con 0x5611eb34fb40 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 10 client.15705 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 10 client.15705 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 20 client.15705 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb369030 0x5611eb36b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 4 client.15705 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611eb2c1860 con 0x5611eb369030 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.814+0000 7f130effd640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb369030 0x5611eb36b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.815+0000 7f130effd640 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb369030 0x5611eb36b440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2bea38b18cf21e6 server_cookie=97fef73ee83b1caa in_seq=0 out_seq=0 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.815+0000 7f12f2ff5640 10 client.15705 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f12f2ff5640 1 -- 172.21.15.19:0/2674460830 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611eb2c1860 con 0x5611eb369030 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f12f2ff5640 10 client.15705 handle_command_reply: tid=0 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 client.15705 shutdown 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 msgr2=0x5611eb356020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1304000c00 tx=0x7f1304009560 comp rx=0 tx=0).stop 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 shutdown_connections 2024-09-12T03:20:53.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb369030 0x5611eb36b440 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb356610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34fb40 0x5611eb356020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f12f2ff5640 0 client.15705 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 --2- 172.21.15.19:0/2674460830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34e5d0 0x5611eb358d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 >> 172.21.15.19:0/2674460830 conn(0x5611eb317310 msgr2=0x5611eb350930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 shutdown_connections 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 1 -- 172.21.15.19:0/2674460830 wait complete. 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.816+0000 7f1317691740 20 client.15705 trim_cache size 0 max 16384 2024-09-12T03:20:53.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.829+0000 7f1317691740 1 Processor -- start 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.829+0000 7f1317691740 1 -- start start 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 0x5611eb3558e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb370d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13101af6c0 con 0x5611eb34e5d0 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b1200 con 0x5611eb359370 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f1317691740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13101afd90 con 0x5611eb34fb40 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41006/0 (socket says 172.21.15.19:41006) 2024-09-12T03:20:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 -- 172.21.15.19:0/129838653 learned_addr learned my addr 172.21.15.19:0/129838653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130f7fe640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb370d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130e7fc640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 0x5611eb3558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 -- 172.21.15.19:0/129838653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 msgr2=0x5611eb3558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 0x5611eb3558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 -- 172.21.15.19:0/129838653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 msgr2=0x5611eb370d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb370d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 -- 172.21.15.19:0/129838653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611eb36bac0 con 0x5611eb34e5d0 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130e7fc640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 0x5611eb3558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f130effd640 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1300005f00 tx=0x7f130000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5141ae28191c2982 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f12f4ff9640 1 -- 172.21.15.19:0/129838653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300003ca0 con 0x5611eb34e5d0 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f12f4ff9640 1 -- 172.21.15.19:0/129838653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1300003e40 con 0x5611eb34e5d0 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.830+0000 7f12f4ff9640 1 -- 172.21.15.19:0/129838653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000116a0 con 0x5611eb34e5d0 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- 172.21.15.19:0/129838653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 msgr2=0x5611eb365340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1300005f00 tx=0x7f130000b390 comp rx=0 tx=0).stop 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- 172.21.15.19:0/129838653 shutdown_connections 2024-09-12T03:20:53.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb359370 0x5611eb370d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb34fb40 0x5611eb3558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 --2- 172.21.15.19:0/129838653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb365340 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- 172.21.15.19:0/129838653 >> 172.21.15.19:0/129838653 conn(0x5611eb317310 msgr2=0x5611eb3580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- 172.21.15.19:0/129838653 shutdown_connections 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- 172.21.15.19:0/129838653 wait complete. 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 -- start start 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.831+0000 7f1317691740 1 Processor -- start 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34fb40 0x5611eb35d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb35dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b0b30 con 0x5611eb34e5d0 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b0460 con 0x5611eb34fb40 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13101ad6a0 con 0x5611eb359370 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f1317691740 10 client.0 _ll_register_callbacks cb 0x7f1304005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130e7fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34fb40 0x5611eb35d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41010/0 (socket says 172.21.15.19:41010) 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130f7fe640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb35dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 -- 172.21.15.19:0/3650297410 learned_addr learned my addr 172.21.15.19:0/3650297410 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 -- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 msgr2=0x5611eb35dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb35dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 -- 172.21.15.19:0/3650297410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34fb40 msgr2=0x5611eb35d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34fb40 0x5611eb35d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 -- 172.21.15.19:0/3650297410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611eb35bf50 con 0x5611eb34e5d0 2024-09-12T03:20:53.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f130effd640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1300004300 tx=0x7f1300003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8668cdb27bb823b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.832+0000 7f12f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300004940 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1300003e40 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1300004ae0 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1300005020 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 10 client.15711 fetch_fsmap learned FSMap version 28 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5611eb3d3dd0 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1300003e40 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f1300011c20 con 0x5611eb34e5d0 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 10 client.15711 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:20:53.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 20 client.15711 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb3d44c0 0x5611eb3d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 4 client.15711 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.833+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611eb2bbd70 con 0x5611eb3d44c0 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.838+0000 7f130e7fc640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb3d44c0 0x5611eb3d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f130e7fc640 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb3d44c0 0x5611eb3d68e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1613f9829a76d01 server_cookie=8a31a9675c0fd84 in_seq=0 out_seq=0 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f12f2ff5640 10 client.15711 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:53.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f12f2ff5640 1 -- 172.21.15.19:0/3650297410 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611eb2bbd70 con 0x5611eb3d44c0 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f12f2ff5640 10 client.15711 handle_command_reply: tid=0 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f1317691740 1 client.15711 shutdown 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 msgr2=0x5611eb35d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.842+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1300004300 tx=0x7f1300003cd0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.844+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 shutdown_connections 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.844+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5611eb3d44c0 0x5611eb3d68e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5611eb359370 0x5611eb35dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.914 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5611eb34fb40 0x5611eb35d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 --2- 172.21.15.19:0/3650297410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5611eb34e5d0 0x5611eb35d1b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f12f2ff5640 0 client.15711 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 >> 172.21.15.19:0/3650297410 conn(0x5611eb317310 msgr2=0x5611eb364330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 shutdown_connections 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 1 -- 172.21.15.19:0/3650297410 wait complete. 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.845+0000 7f1317691740 20 client.15711 trim_cache size 0 max 16384 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 msgr2=0x7f12f80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 0x7f12f80a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1308022000 tx=0x7f1308023000 comp rx=0 tx=0).stop 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 msgr2=0x7f1310101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f1304003c10 comp rx=0 tx=0).stop 2024-09-12T03:20:53.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 shutdown_connections 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1310106b80 0x7f1310102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f80a2c00 0x7f12f80a50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1310100500 0x7f1310101b10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 --2- 172.21.15.19:0/2449508473 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f13100ffb60 0x7f13101015d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 >> 172.21.15.19:0/2449508473 conn(0x7f13100fba90 msgr2=0x7f13100fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 shutdown_connections 2024-09-12T03:20:53.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:53.846+0000 7f13152b8640 1 -- 172.21.15.19:0/2449508473 wait complete. 2024-09-12T03:20:54.583 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:54 smithi019 ceph-mon[30953]: pgmap v393: 97 pgs: 97 active+clean; 30 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-09-12T03:20:54.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:20:54.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:54 smithi135 ceph-mon[35474]: pgmap v393: 97 pgs: 97 active+clean; 30 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-09-12T03:20:54.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:54 smithi121 ceph-mon[34963]: pgmap v393: 97 pgs: 97 active+clean; 30 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 2.6 MiB/s wr, 165 op/s 2024-09-12T03:20:54.994 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 -- 172.21.15.19:0/1004688852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 msgr2=0x7fe0701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 --2- 172.21.15.19:0/1004688852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe0701029b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe060009900 tx=0x7fe06002a480 comp rx=0 tx=0).stop 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 -- 172.21.15.19:0/1004688852 shutdown_connections 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 --2- 172.21.15.19:0/1004688852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 --2- 172.21.15.19:0/1004688852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe0701029b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 --2- 172.21.15.19:0/1004688852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe070101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.498+0000 7fe0780c2640 1 -- 172.21.15.19:0/1004688852 >> 172.21.15.19:0/1004688852 conn(0x7fe0700fc7d0 msgr2=0x7fe0700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.499+0000 7fe0780c2640 1 -- 172.21.15.19:0/1004688852 shutdown_connections 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.499+0000 7fe0780c2640 1 -- 172.21.15.19:0/1004688852 wait complete. 2024-09-12T03:20:55.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.499+0000 7fe0780c2640 1 Processor -- start 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.499+0000 7fe0780c2640 1 -- start start 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe070110f80 con 0x7fe070102570 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe070110e00 con 0x7fe070100e20 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe0780c2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe070111100 con 0x7fe070102ef0 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07665f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:35256/0 (socket says 172.21.15.19:35256) 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07665f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60640/0 (socket says 172.21.15.19:60640) 2024-09-12T03:20:55.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41026/0 (socket says 172.21.15.19:41026) 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07665f640 1 -- 172.21.15.19:0/2502172433 learned_addr learned my addr 172.21.15.19:0/2502172433 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 -- 172.21.15.19:0/2502172433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 msgr2=0x7fe07019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 -- 172.21.15.19:0/2502172433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 msgr2=0x7fe0701973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07565d640 1 -- 172.21.15.19:0/2502172433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07019c390 con 0x7fe070102570 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe07665f640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.500+0000 7fe075e5e640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe07565d640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe060005940 tx=0x7fe060003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe066ffd640 1 -- 172.21.15.19:0/2502172433 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06003f070 con 0x7fe070102570 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe066ffd640 1 -- 172.21.15.19:0/2502172433 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe060031c30 con 0x7fe070102570 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe066ffd640 1 -- 172.21.15.19:0/2502172433 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060035030 con 0x7fe070102570 2024-09-12T03:20:55.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07019c620 con 0x7fe070102570 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.501+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0701a3f60 con 0x7fe070102570 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.503+0000 7fe066ffd640 1 -- 172.21.15.19:0/2502172433 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe060042070 con 0x7fe070102570 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 Processor -- start 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe066ffd640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 0x7fe05c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 -- start start 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe066ffd640 1 -- 172.21.15.19:0/2502172433 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe0600ef5b0 con 0x7fe070102570 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07665f640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 0x7fe05c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837a5d0 0x55c21837aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c218385370 0x55c218387760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c2182e02f0 con 0x55c218385370 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c218287cf0 con 0x55c21837a5d0 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07a49b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2182dc2b0 con 0x55c21837bb40 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60642/0 (socket says 172.21.15.19:60642) 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe075e5e640 1 -- 172.21.15.19:0/4136389469 learned_addr learned my addr 172.21.15.19:0/4136389469 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.504+0000 7fe07565d640 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837a5d0 0x55c21837aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe07665f640 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c218385370 0x55c218387760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 -- 172.21.15.19:0/4136389469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837a5d0 msgr2=0x55c21837aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837a5d0 0x55c21837aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe07665f640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 0x7fe05c0a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0680049d0 tx=0x7fe06800fc10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 -- 172.21.15.19:0/4136389469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c218385370 msgr2=0x55c218387760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c218385370 0x55c218387760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 -- 172.21.15.19:0/4136389469 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21826c3a0 con 0x55c21837bb40 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe075e5e640 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00f3c0 tx=0x7fe06c00f880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb0ca48e0a0a5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe05bfff640 1 -- 172.21.15.19:0/4136389469 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c01d070 con 0x55c21837bb40 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe05bfff640 1 -- 172.21.15.19:0/4136389469 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c010de0 con 0x55c21837bb40 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.505+0000 7fe05bfff640 1 -- 172.21.15.19:0/4136389469 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c016340 con 0x55c21837bb40 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- 172.21.15.19:0/4136389469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 msgr2=0x55c21837a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe06c00f3c0 tx=0x7fe06c00f880 comp rx=0 tx=0).stop 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- 172.21.15.19:0/4136389469 shutdown_connections 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c218385370 0x55c218387760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837a5d0 0x55c21837aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 --2- 172.21.15.19:0/4136389469 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c21837a090 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- 172.21.15.19:0/4136389469 >> 172.21.15.19:0/4136389469 conn(0x55c218343310 msgr2=0x55c218376c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- 172.21.15.19:0/4136389469 shutdown_connections 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- 172.21.15.19:0/4136389469 wait complete. 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 -- start start 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.506+0000 7fe07a49b740 1 Processor -- start 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c218384d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c218382020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c218382610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c218388070 con 0x55c21837a5d0 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c21829b060 con 0x55c218385370 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c2183881f0 con 0x55c21837bb40 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07a49b740 10 client.0 _ll_register_callbacks cb 0x55c217fee8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:55.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c218384d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c218382020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c218384d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41052/0 (socket says 172.21.15.19:41052) 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe075e5e640 1 -- 172.21.15.19:0/865532186 learned_addr learned my addr 172.21.15.19:0/865532186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07665f640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c218382610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 -- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 msgr2=0x55c218382610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c218382610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 -- 172.21.15.19:0/865532186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 msgr2=0x55c218384d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c218384d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.507+0000 7fe07565d640 1 -- 172.21.15.19:0/865532186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2180513c0 con 0x55c21837bb40 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07565d640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c218382020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0600ed110 tx=0x7fe0600bba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4591ec1ec8032a46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:55.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060049040 con 0x55c21837bb40 2024-09-12T03:20:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c010de0 con 0x55c21837bb40 2024-09-12T03:20:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe06003b460 con 0x55c21837bb40 2024-09-12T03:20:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe0600c01d0 con 0x55c21837bb40 2024-09-12T03:20:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 10 client.25273 fetch_fsmap learned FSMap version 29 2024-09-12T03:20:55.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c218388120 con 0x55c21837bb40 2024-09-12T03:20:55.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c010de0 con 0x55c21837bb40 2024-09-12T03:20:55.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe0600ef0e0 con 0x55c21837bb40 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 10 client.25273 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 10 client.25273 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 20 client.25273 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218394fe0 0x55c218397410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 4 client.25273 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.508+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2182ed860 con 0x55c218394fe0 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.509+0000 7fe075e5e640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218394fe0 0x55c218397410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.509+0000 7fe075e5e640 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218394fe0 0x55c218397410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=222a8e562723cd34 server_cookie=2bd51a793826423c in_seq=0 out_seq=0 2024-09-12T03:20:55.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.509+0000 7fe0597fa640 10 client.25273 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe0597fa640 1 -- 172.21.15.19:0/865532186 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2182ed860 con 0x55c218394fe0 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe0597fa640 10 client.25273 handle_command_reply: tid=0 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 client.25273 shutdown 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 msgr2=0x55c218382020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c218382020 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe0600ed110 tx=0x7fe0600bba10 comp rx=0 tx=0).stop 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 shutdown_connections 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218394fe0 0x55c218397410 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c218382610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837bb40 0x55c218382020 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe0597fa640 0 client.25273 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 --2- 172.21.15.19:0/865532186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c218384d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 >> 172.21.15.19:0/865532186 conn(0x55c218343310 msgr2=0x55c21837c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 shutdown_connections 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 1 -- 172.21.15.19:0/865532186 wait complete. 2024-09-12T03:20:55.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.510+0000 7fe07a49b740 20 client.25273 trim_cache size 0 max 16384 2024-09-12T03:20:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 Processor -- start 2024-09-12T03:20:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 -- start start 2024-09-12T03:20:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c21837d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ab240 con 0x55c21837bb40 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701acdb0 con 0x55c218385370 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07a49b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ab910 con 0x55c21837a5d0 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07565d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60660/0 (socket says 172.21.15.19:60660) 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe075e5e640 1 -- 172.21.15.19:0/641804349 learned_addr learned my addr 172.21.15.19:0/641804349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.525+0000 7fe07565d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41056/0 (socket says 172.21.15.19:41056) 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe07665f640 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c21837d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 -- 172.21.15.19:0/641804349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 msgr2=0x55c21837d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c21837d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 -- 172.21.15.19:0/641804349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 msgr2=0x55c218383f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 -- 172.21.15.19:0/641804349 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c218380590 con 0x55c21837a5d0 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe07565d640 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe075e5e640 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005c20 tx=0x7fe06c00ff90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3f03c1b5236698 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe05b7fe640 1 -- 172.21.15.19:0/641804349 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c01d070 con 0x55c21837a5d0 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe05b7fe640 1 -- 172.21.15.19:0/641804349 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c018a80 con 0x55c21837a5d0 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.526+0000 7fe05b7fe640 1 -- 172.21.15.19:0/641804349 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c010a10 con 0x55c21837a5d0 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- 172.21.15.19:0/641804349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 msgr2=0x55c2183818e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005c20 tx=0x7fe06c00ff90 comp rx=0 tx=0).stop 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- 172.21.15.19:0/641804349 shutdown_connections 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c218385370 0x55c21837d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837bb40 0x55c218383f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 --2- 172.21.15.19:0/641804349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c21837a5d0 0x55c2183818e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- 172.21.15.19:0/641804349 >> 172.21.15.19:0/641804349 conn(0x55c218343310 msgr2=0x55c218383880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- 172.21.15.19:0/641804349 shutdown_connections 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- 172.21.15.19:0/641804349 wait complete. 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.527+0000 7fe07a49b740 1 -- start start 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 Processor -- start 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c21838dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 0x55c21837ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701ac6b0 con 0x55c21837a5d0 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701abfe0 con 0x55c21837bb40 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe0701a9220 con 0x55c218385370 2024-09-12T03:20:55.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07a49b740 10 client.0 _ll_register_callbacks cb 0x7fe060009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c21838dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07665f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07665f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60674/0 (socket says 172.21.15.19:60674) 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe075e5e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c21838dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41066/0 (socket says 172.21.15.19:41066) 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07665f640 1 -- 172.21.15.19:0/3437391193 learned_addr learned my addr 172.21.15.19:0/3437391193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.528+0000 7fe07565d640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 0x55c21837ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 -- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 msgr2=0x55c21837ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 0x55c21837ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 -- 172.21.15.19:0/3437391193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 msgr2=0x55c21838dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c21838dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 -- 172.21.15.19:0/3437391193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21837fca0 con 0x55c218385370 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07565d640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 0x55c21837ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:55.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07665f640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe068011fd0 tx=0x7fe068016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc2c9a8fb83cfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe0597fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06802e060 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06c018a80 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe0680293f0 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe068017040 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07a49b740 10 client.25279 fetch_fsmap learned FSMap version 29 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.529+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2183ffe50 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c018a80 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7fe068028070 con 0x55c218385370 2024-09-12T03:20:55.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 10 client.25279 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 10 client.25279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 20 client.25279 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218400600 0x55c218402a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 4 client.25279 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2182e7d70 con 0x55c218400600 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.530+0000 7fe075e5e640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218400600 0x55c218402a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe0597fa640 10 client.25279 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe075e5e640 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218400600 0x55c218402a20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b3878d8842a7820 server_cookie=63c2231f2e18b288 in_seq=0 out_seq=0 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe0597fa640 1 -- 172.21.15.19:0/3437391193 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2182e7d70 con 0x55c218400600 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe0597fa640 10 client.25279 handle_command_reply: tid=0 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 client.25279 shutdown 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 msgr2=0x55c21837f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe068011fd0 tx=0x7fe068016000 comp rx=0 tx=0).stop 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 shutdown_connections 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55c218400600 0x55c218402a20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c218385370 0x55c21837f4a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c21837bb40 0x55c21837ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 --2- 172.21.15.19:0/3437391193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c21837a5d0 0x55c21838dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe0597fa640 0 client.25279 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 >> 172.21.15.19:0/3437391193 conn(0x55c218343310 msgr2=0x55c218375d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 shutdown_connections 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 1 -- 172.21.15.19:0/3437391193 wait complete. 2024-09-12T03:20:55.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.531+0000 7fe07a49b740 20 client.25279 trim_cache size 0 max 16384 2024-09-12T03:20:55.545 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 msgr2=0x7fe05c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 0x7fe05c0a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe0680049d0 tx=0x7fe06800fc10 comp rx=0 tx=0).stop 2024-09-12T03:20:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 msgr2=0x7fe070197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:55.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe060005940 tx=0x7fe060003c10 comp rx=0 tx=0).stop 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 shutdown_connections 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe070102ef0 0x7fe07019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe05c0a2c00 0x7fe05c0a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe070102570 0x7fe070197920 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 --2- 172.21.15.19:0/2502172433 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe070100e20 0x7fe0701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 >> 172.21.15.19:0/2502172433 conn(0x7fe0700fc7d0 msgr2=0x7fe0701068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 shutdown_connections 2024-09-12T03:20:55.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:55.532+0000 7fe0780c2640 1 -- 172.21.15.19:0/2502172433 wait complete. 2024-09-12T03:20:55.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:55 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:20:55.694 INFO:tasks.workunit.client.0.smithi019.stdout: 16 289232 768 204844 882 132688 1418 2024-09-12T03:20:55.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:55 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:20:55.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:55 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:20:56.269 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:20:56.542 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:56.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:56 smithi019 ceph-mon[30953]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2024-09-12T03:20:56.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:56 smithi135 ceph-mon[35474]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2024-09-12T03:20:56.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:56 smithi121 ceph-mon[34963]: pgmap v394: 97 pgs: 97 active+clean; 50 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.3 MiB/s wr, 221 op/s 2024-09-12T03:20:57.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- 172.21.15.19:0/1355827439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41090f0 msgr2=0x7f67a4110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 --2- 172.21.15.19:0/1355827439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41090f0 0x7f67a4110c70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f679c009900 tx=0x7f679c02a410 comp rx=0 tx=0).stop 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- 172.21.15.19:0/1355827439 shutdown_connections 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 --2- 172.21.15.19:0/1355827439 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a41090f0 0x7f67a4110c70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 --2- 172.21.15.19:0/1355827439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a4104970 0x7f67a41089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 --2- 172.21.15.19:0/1355827439 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4103fc0 0x7f67a41043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- 172.21.15.19:0/1355827439 >> 172.21.15.19:0/1355827439 conn(0x7f67a40ffa40 msgr2=0x7f67a4101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- 172.21.15.19:0/1355827439 shutdown_connections 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- 172.21.15.19:0/1355827439 wait complete. 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 Processor -- start 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.076+0000 7f67ac114640 1 -- start start 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 0x7f67a419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 0x7f67a419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4114120 con 0x7f67a4103fc0 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67a4113fa0 con 0x7f67a41090f0 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67ac114640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41142a0 con 0x7f67a4104970 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41092/0 (socket says 172.21.15.19:41092) 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a96af640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 0x7f67a419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3430013508 learned_addr learned my addr 172.21.15.19:0/3430013508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 0x7f67a419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3430013508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 msgr2=0x7f67a419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 0x7f67a419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3430013508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 msgr2=0x7f67a419eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 0x7f67a419eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3430013508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a419f5b0 con 0x7f67a4103fc0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 0x7f67a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a96af640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 0x7f67a419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f679400eb90 tx=0x7f679400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.077+0000 7f67a2ffd640 1 -- 172.21.15.19:0/3430013508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794004d20 con 0x7f67a4103fc0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.078+0000 7f67a2ffd640 1 -- 172.21.15.19:0/3430013508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679400ee90 con 0x7f67a4103fc0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.078+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a419f8a0 con 0x7f67a4103fc0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.078+0000 7f67a2ffd640 1 -- 172.21.15.19:0/3430013508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6794010830 con 0x7f67a4103fc0 2024-09-12T03:20:57.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.078+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a41a7180 con 0x7f67a4103fc0 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.081+0000 7f67a2ffd640 1 -- 172.21.15.19:0/3430013508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6794005020 con 0x7f67a4103fc0 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.081+0000 7f67ae4ed740 1 Processor -- start 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.081+0000 7f67ae4ed740 1 -- start start 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.081+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077d9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.081+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 0x5641077e6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56410773f2f0 con 0x5641077d95d0 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641076e6cf0 con 0x5641077dab40 2024-09-12T03:20:57.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56410773b2b0 con 0x5641077e4370 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41094/0 (socket says 172.21.15.19:41094) 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 -- 172.21.15.19:0/934515393 learned_addr learned my addr 172.21.15.19:0/934515393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a96af640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 0x5641077e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077d9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 -- 172.21.15.19:0/934515393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 msgr2=0x5641077e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 0x5641077e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a2ffd640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 0x7f678c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 -- 172.21.15.19:0/934515393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 msgr2=0x5641077d9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077d9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 -- 172.21.15.19:0/934515393 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641076cb3a0 con 0x5641077d95d0 2024-09-12T03:20:57.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a96af640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 0x5641077e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 0x7f678c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a2ffd640 1 -- 172.21.15.19:0/3430013508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67940cf6e0 con 0x7f67a4103fc0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.082+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f679409a220 tx=0x7f6794098250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36953d63258e22e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.083+0000 7f678bfff640 1 -- 172.21.15.19:0/934515393 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679409ba30 con 0x5641077d95d0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.083+0000 7f678bfff640 1 -- 172.21.15.19:0/934515393 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67940d5e30 con 0x5641077d95d0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.083+0000 7f678bfff640 1 -- 172.21.15.19:0/934515393 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679409b850 con 0x5641077d95d0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- 172.21.15.19:0/934515393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 msgr2=0x5641077d9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 0x7f678c0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f679c005940 tx=0x7f679c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f679409a220 tx=0x7f6794098250 comp rx=0 tx=0).stop 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- 172.21.15.19:0/934515393 shutdown_connections 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077e4370 0x5641077e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077d95d0 0x5641077d9a10 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/934515393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077d9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- 172.21.15.19:0/934515393 >> 172.21.15.19:0/934515393 conn(0x5641077a2310 msgr2=0x5641077d5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- 172.21.15.19:0/934515393 shutdown_connections 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- 172.21.15.19:0/934515393 wait complete. 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 -- start start 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.084+0000 7f67ae4ed740 1 Processor -- start 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077e1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 0x5641077e17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5641077e7070 con 0x5641077e4370 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5641076fa060 con 0x5641077dab40 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5641077e71f0 con 0x5641077d95d0 2024-09-12T03:20:57.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67ae4ed740 10 client.0 _ll_register_callbacks cb 0x56410744d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67aa6b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 0x5641077e17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67aa6b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 0x5641077e17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41100/0 (socket says 172.21.15.19:41100) 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67aa6b1640 1 -- 172.21.15.19:0/3964161387 learned_addr learned my addr 172.21.15.19:0/3964161387 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a96af640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077e1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3964161387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 msgr2=0x5641077e1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077e1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3964161387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 msgr2=0x5641077e17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 0x5641077e17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 -- 172.21.15.19:0/3964161387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641074b03c0 con 0x5641077d95d0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e3db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6794009270 tx=0x7f6794009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=672aafd52a5284a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.085+0000 7f67897fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:57.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679409b850 con 0x5641077d95d0 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67940d5e30 con 0x5641077d95d0 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f679409a1f0 con 0x5641077d95d0 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f679409ba30 con 0x5641077d95d0 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 10 client.25291 fetch_fsmap learned FSMap version 29 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5641077e7070 con 0x5641077d95d0 2024-09-12T03:20:57.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67940d5e30 con 0x5641077d95d0 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f679409c150 con 0x5641077d95d0 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 10 client.25291 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 10 client.25291 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 20 client.25291 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5641077f3ce0 0x5641077f60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 4 client.25291 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:57.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.086+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56410774c860 con 0x5641077f3ce0 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.087+0000 7f67a96af640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5641077f3ce0 0x5641077f60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.087+0000 7f67a96af640 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5641077f3ce0 0x5641077f60d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=805b4b7ed207dd45 server_cookie=1eb1a87fc8949623 in_seq=0 out_seq=0 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.087+0000 7f67897fa640 10 client.25291 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67897fa640 1 -- 172.21.15.19:0/3964161387 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56410774c860 con 0x5641077f3ce0 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67897fa640 10 client.25291 handle_command_reply: tid=0 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 client.25291 shutdown 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 msgr2=0x5641077e3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e3db0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6794009270 tx=0x7f6794009200 comp rx=0 tx=0).stop 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 shutdown_connections 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077e4370 0x5641077e17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5641077f3ce0 0x5641077f60d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077dab40 0x5641077e1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3964161387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e3db0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 >> 172.21.15.19:0/3964161387 conn(0x5641077a2310 msgr2=0x5641077db930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67897fa640 0 client.25291 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.090+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 shutdown_connections 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.091+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3964161387 wait complete. 2024-09-12T03:20:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.091+0000 7f67ae4ed740 20 client.25291 trim_cache size 0 max 16384 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 Processor -- start 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 -- start start 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41af7a0 con 0x5641077dab40 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b1310 con 0x5641077e4370 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41afe70 con 0x5641077d95d0 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41102/0 (socket says 172.21.15.19:41102) 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 -- 172.21.15.19:0/2077818523 learned_addr learned my addr 172.21.15.19:0/2077818523 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 -- 172.21.15.19:0/2077818523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 msgr2=0x5641077e0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 -- 172.21.15.19:0/2077818523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 msgr2=0x5641077fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.104+0000 7f67a96af640 1 -- 172.21.15.19:0/2077818523 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641077d5dc0 con 0x5641077dab40 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67a96af640 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5641077e08f0 tx=0x7f679800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec689b97de1311f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f678b7fe640 1 -- 172.21.15.19:0/2077818523 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798004220 con 0x5641077dab40 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f678b7fe640 1 -- 172.21.15.19:0/2077818523 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67980043c0 con 0x5641077dab40 2024-09-12T03:20:57.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f678b7fe640 1 -- 172.21.15.19:0/2077818523 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798003e10 con 0x5641077dab40 2024-09-12T03:20:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 -- 172.21.15.19:0/2077818523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 msgr2=0x5641077fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x5641077e08f0 tx=0x7f679800b390 comp rx=0 tx=0).stop 2024-09-12T03:20:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 -- 172.21.15.19:0/2077818523 shutdown_connections 2024-09-12T03:20:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077fb700 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.105+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/2077818523 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- 172.21.15.19:0/2077818523 >> 172.21.15.19:0/2077818523 conn(0x5641077a2310 msgr2=0x5641077e0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- 172.21.15.19:0/2077818523 shutdown_connections 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- 172.21.15.19:0/2077818523 wait complete. 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- start start 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 Processor -- start 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077e8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b0c10 con 0x5641077dab40 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41b0540 con 0x5641077e4370 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f67a41ad780 con 0x5641077d95d0 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.106+0000 7f67ae4ed740 10 client.0 _ll_register_callbacks cb 0x7f67940093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41114/0 (socket says 172.21.15.19:41114) 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 -- 172.21.15.19:0/3745731347 learned_addr learned my addr 172.21.15.19:0/3745731347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077e8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 -- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 msgr2=0x5641077e8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 -- 172.21.15.19:0/3745731347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 msgr2=0x5641077e8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077e8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 -- 172.21.15.19:0/3745731347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56410785d750 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67aa6b1640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67a96af640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6798025000 tx=0x7f6798011770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1178e2a5e8ecc98f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67897fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679802b070 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6798003c90 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67980043c0 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6798003770 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67ae4ed740 10 client.15747 fetch_fsmap learned FSMap version 29 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.107+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56410785ea30 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67980043c0 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f67980268a0 con 0x5641077dab40 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:20:57.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 20 client.15747 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56410785f100 0x564107861520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 4 client.15747 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564107746d70 con 0x56410785f100 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.108+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56410785f100 0x564107861520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.109+0000 7f67897fa640 10 client.15747 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:57.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.109+0000 7f67a9eb0640 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56410785f100 0x564107861520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b17c52a1ec5323c server_cookie=5cdaf4bbe2885830 in_seq=0 out_seq=0 2024-09-12T03:20:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.115+0000 7f67897fa640 1 -- 172.21.15.19:0/3745731347 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564107746d70 con 0x56410785f100 2024-09-12T03:20:57.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.115+0000 7f67897fa640 10 client.15747 handle_command_reply: tid=0 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 client.15747 shutdown 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 msgr2=0x5641077e87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6798025000 tx=0x7f6798011770 comp rx=0 tx=0).stop 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 shutdown_connections 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56410785f100 0x564107861520 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5641077e4370 0x5641077e8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5641077dab40 0x5641077e87a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67897fa640 0 client.15747 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 --2- 172.21.15.19:0/3745731347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5641077d95d0 0x5641077e8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 >> 172.21.15.19:0/3745731347 conn(0x5641077a2310 msgr2=0x5641077d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 shutdown_connections 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 1 -- 172.21.15.19:0/3745731347 wait complete. 2024-09-12T03:20:57.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.116+0000 7f67ae4ed740 20 client.15747 trim_cache size 0 max 16384 2024-09-12T03:20:57.118 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:20:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 msgr2=0x7f678c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 0x7f678c0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f679c005940 tx=0x7f679c009260 comp rx=0 tx=0).stop 2024-09-12T03:20:57.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 msgr2=0x7f67a419a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f679400eb90 tx=0x7f679400c6a0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 shutdown_connections 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f67a41090f0 0x7f67a419eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f67a4104970 0x7f67a419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f678c0a2c00 0x7f678c0a50c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 --2- 172.21.15.19:0/3430013508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f67a4103fc0 0x7f67a419a600 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 >> 172.21.15.19:0/3430013508 conn(0x7f67a40ffa40 msgr2=0x7f67a4100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 shutdown_connections 2024-09-12T03:20:57.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:57.117+0000 7f67ac114640 1 -- 172.21.15.19:0/3430013508 wait complete. 2024-09-12T03:20:58.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:20:58 smithi135 ceph-mon[35474]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-09-12T03:20:58.875 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:20:58.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:20:58.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:20:58 smithi121 ceph-mon[34963]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-09-12T03:20:58.903 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:20:58 smithi019 ceph-mon[30953]: pgmap v395: 97 pgs: 97 active+clean; 50 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 238 op/s 2024-09-12T03:20:59.151 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:20:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 -- 172.21.15.19:0/346703229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c1019e0 msgr2=0x7f427c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 --2- 172.21.15.19:0/346703229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c1019e0 0x7f427c101e40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f426c009930 tx=0x7f426c02a420 comp rx=0 tx=0).stop 2024-09-12T03:20:59.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 -- 172.21.15.19:0/346703229 shutdown_connections 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 --2- 172.21.15.19:0/346703229 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 0x7f427c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 --2- 172.21.15.19:0/346703229 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c1019e0 0x7f427c101e40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 --2- 172.21.15.19:0/346703229 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c107af0 0x7f427c107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 -- 172.21.15.19:0/346703229 >> 172.21.15.19:0/346703229 conn(0x7f427c0fd700 msgr2=0x7f427c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 -- 172.21.15.19:0/346703229 shutdown_connections 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 -- 172.21.15.19:0/346703229 wait complete. 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.891+0000 7f428339f640 1 Processor -- start 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 -- start start 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 0x7f427c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 0x7f427c198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f427c111f40 con 0x7f427c107af0 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c111dc0 con 0x7f427c102380 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428339f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1120c0 con 0x7f427c1019e0 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428239d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 0x7f427c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41126/0 (socket says 172.21.15.19:41126) 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 -- 172.21.15.19:0/1166927035 learned_addr learned my addr 172.21.15.19:0/1166927035 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4281b9c640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 0x7f427c198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 -- 172.21.15.19:0/1166927035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 msgr2=0x7f427c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 0x7f427c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 -- 172.21.15.19:0/1166927035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 msgr2=0x7f427c198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 0x7f427c198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f4282b9e640 1 -- 172.21.15.19:0/1166927035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427c19d390 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.892+0000 7f428239d640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 0x7f427c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f4282b9e640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4274009fd0 tx=0x7f427400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f42737fe640 1 -- 172.21.15.19:0/1166927035 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274017070 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f42737fe640 1 -- 172.21.15.19:0/1166927035 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4274003e50 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f427c19d680 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f42737fe640 1 -- 172.21.15.19:0/1166927035 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42740106c0 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.893+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f427c1a4f60 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.895+0000 7f42737fe640 1 -- 172.21.15.19:0/1166927035 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4274003970 con 0x7f427c107af0 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 Processor -- start 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 -- start start 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42737fe640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 0x7f42680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e46340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4d9e280 con 0x55a1f4e38d70 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4d45b10 con 0x55a1f4e43f50 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42869da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4d9a240 con 0x55a1f4e39690 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f428239d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f4281b9c640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 0x7f42680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f428239d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41138/0 (socket says 172.21.15.19:41138) 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f428239d640 1 -- 172.21.15.19:0/1697377832 learned_addr learned my addr 172.21.15.19:0/1697377832 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.896+0000 7f42737fe640 1 -- 172.21.15.19:0/1166927035 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f42740cedd0 con 0x7f427c107af0 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4281b9c640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4282b9e640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e46340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 -- 172.21.15.19:0/1697377832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 msgr2=0x55a1f4e43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 -- 172.21.15.19:0/1697377832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 msgr2=0x55a1f4e46340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e46340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 -- 172.21.15.19:0/1697377832 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f4d521f0 con 0x55a1f4e38d70 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4282b9e640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f428239d640 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42780114c0 tx=0x7f4278011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d62ce33ce15ffe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4281b9c640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 0x7f42680a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f426c006010 tx=0x7f426c0095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4270ff9640 1 -- 172.21.15.19:0/1697377832 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278004020 con 0x55a1f4e38d70 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4270ff9640 1 -- 172.21.15.19:0/1697377832 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780041c0 con 0x55a1f4e38d70 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.897+0000 7f4270ff9640 1 -- 172.21.15.19:0/1697377832 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800f9d0 con 0x55a1f4e38d70 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- 172.21.15.19:0/1697377832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 msgr2=0x55a1f4e39150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42780114c0 tx=0x7f4278011980 comp rx=0 tx=0).stop 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- 172.21.15.19:0/1697377832 shutdown_connections 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e46340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 --2- 172.21.15.19:0/1697377832 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e39150 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- 172.21.15.19:0/1697377832 >> 172.21.15.19:0/1697377832 conn(0x55a1f4e35300 msgr2=0x55a1f4e356e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- 172.21.15.19:0/1697377832 shutdown_connections 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- 172.21.15.19:0/1697377832 wait complete. 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 -- start start 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.898+0000 7f42869da740 1 Processor -- start 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e41390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4e46b20 con 0x55a1f4e39690 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4d58f10 con 0x55a1f4e38d70 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a1f4e46ca0 con 0x55a1f4e43f50 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f42869da740 10 client.0 _ll_register_callbacks cb 0x55a1f4d21a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4281b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e41390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60752/0 (socket says 172.21.15.19:60752) 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4281b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e41390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41148/0 (socket says 172.21.15.19:41148) 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 -- 172.21.15.19:0/694514342 learned_addr learned my addr 172.21.15.19:0/694514342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f428239d640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 -- 172.21.15.19:0/694514342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 msgr2=0x55a1f4e40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e40e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 -- 172.21.15.19:0/694514342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 msgr2=0x55a1f4e41390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e41390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.899+0000 7f4282b9e640 1 -- 172.21.15.19:0/694514342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f4b0f6a0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4282b9e640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4274005c40 tx=0x7f427409afa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea588d2a8dd9504d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4274020040 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f42740040b0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42780041c0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f427409f9f0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 10 client.25309 fetch_fsmap learned FSMap version 30 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1f4e526c0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780041c0 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f427408db50 con 0x55a1f4e43f50 2024-09-12T03:20:59.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 10 client.25309 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 20 client.25309 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4e53c20 0x55a1f4e56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 4 client.25309 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.900+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1f4dab7f0 con 0x55a1f4e53c20 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.901+0000 7f428239d640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4e53c20 0x55a1f4e56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.901+0000 7f4265ffb640 10 client.25309 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.901+0000 7f428239d640 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4e53c20 0x55a1f4e56080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f86d7a05d523a5b server_cookie=c049309d090ecdf in_seq=0 out_seq=0 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f4265ffb640 1 -- 172.21.15.19:0/694514342 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1f4dab7f0 con 0x55a1f4e53c20 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f4265ffb640 10 client.25309 handle_command_reply: tid=0 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 client.25309 shutdown 2024-09-12T03:20:59.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 msgr2=0x55a1f4e418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4274005c40 tx=0x7f427409afa0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 shutdown_connections 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e418d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e41390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.904+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4e53c20 0x55a1f4e56080 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f42869da740 1 --2- 172.21.15.19:0/694514342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 >> 172.21.15.19:0/694514342 conn(0x55a1f4e35300 msgr2=0x55a1f4e3afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f4265ffb640 0 client.25309 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 shutdown_connections 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f42869da740 1 -- 172.21.15.19:0/694514342 wait complete. 2024-09-12T03:20:59.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.905+0000 7f42869da740 20 client.25309 trim_cache size 0 max 16384 2024-09-12T03:20:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 Processor -- start 2024-09-12T03:20:59.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 -- start start 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e42cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1ad330 con 0x55a1f4e39690 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1aef30 con 0x55a1f4e38d70 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f42869da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1ada00 con 0x55a1f4e43f50 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f4281b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f4281b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41156/0 (socket says 172.21.15.19:41156) 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f4281b9c640 1 -- 172.21.15.19:0/34835914 learned_addr learned my addr 172.21.15.19:0/34835914 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f4282b9e640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.923+0000 7f428239d640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e42cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 -- 172.21.15.19:0/34835914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 msgr2=0x55a1f4e5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 -- 172.21.15.19:0/34835914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 msgr2=0x55a1f4e42cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e42cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 -- 172.21.15.19:0/34835914 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1f4e3f3a0 con 0x55a1f4e39690 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4282b9e640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4281b9c640 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f426c009930 tx=0x7f426c039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be23b8ffbe014a3a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4267fff640 1 -- 172.21.15.19:0/34835914 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c043040 con 0x55a1f4e39690 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4267fff640 1 -- 172.21.15.19:0/34835914 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c0325c0 con 0x55a1f4e39690 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.924+0000 7f4267fff640 1 -- 172.21.15.19:0/34835914 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c035070 con 0x55a1f4e39690 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- 172.21.15.19:0/34835914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 msgr2=0x55a1f4e5b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f426c009930 tx=0x7f426c039000 comp rx=0 tx=0).stop 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- 172.21.15.19:0/34835914 shutdown_connections 2024-09-12T03:20:59.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e43f50 0x55a1f4e5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e39690 0x55a1f4e5b950 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 --2- 172.21.15.19:0/34835914 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e38d70 0x55a1f4e42cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- 172.21.15.19:0/34835914 >> 172.21.15.19:0/34835914 conn(0x55a1f4e35300 msgr2=0x55a1f4e425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- 172.21.15.19:0/34835914 shutdown_connections 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- 172.21.15.19:0/34835914 wait complete. 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.925+0000 7f42869da740 1 -- start start 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 Processor -- start 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e48400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e48940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1ae7a0 con 0x55a1f4e38d70 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1ae0d0 con 0x55a1f4e43f50 2024-09-12T03:20:59.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f427c1ab310 con 0x55a1f4e39690 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f42869da740 10 client.0 _ll_register_callbacks cb 0x7f4274097220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41172/0 (socket says 172.21.15.19:41172) 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 -- 172.21.15.19:0/3244241837 learned_addr learned my addr 172.21.15.19:0/3244241837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f4281b9c640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e48400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f4282b9e640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e48940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 -- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 msgr2=0x55a1f4e48400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e48400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 -- 172.21.15.19:0/3244241837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 msgr2=0x55a1f4e48940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e48940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 -- 172.21.15.19:0/3244241837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1f4ebdb70 con 0x55a1f4e38d70 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f428239d640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f427800f5c0 tx=0x7f42780112c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d5c15209ba3f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:20:59.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.926+0000 7f4265ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278033060 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4278005210 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426c0325c0 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f427801d040 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 10 client.15765 fetch_fsmap learned FSMap version 30 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1f4ebeeb0 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f426c0325c0 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f42780035f0 con 0x55a1f4e38d70 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 10 client.15765 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 20 client.15765 populate_metadata read hostname 'smithi019' 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4ebf520 0x55a1f4ec1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:20:59.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 4 client.15765 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:20:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.927+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a1f4da5d00 con 0x55a1f4ebf520 2024-09-12T03:20:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.928+0000 7f4281b9c640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4ebf520 0x55a1f4ec1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:20:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.928+0000 7f4281b9c640 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4ebf520 0x55a1f4ec1940 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13eb905eccf8ae8c server_cookie=d841deec3820e42e in_seq=0 out_seq=0 2024-09-12T03:20:59.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:20:59.928+0000 7f4265ffb640 10 client.15765 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.052+0000 7f4265ffb640 1 -- 172.21.15.19:0/3244241837 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a1f4da5d00 con 0x55a1f4ebf520 2024-09-12T03:21:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.052+0000 7f4265ffb640 10 client.15765 handle_command_reply: tid=0 2024-09-12T03:21:00.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.052+0000 7f42869da740 1 client.15765 shutdown 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 msgr2=0x55a1f4e47ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f427800f5c0 tx=0x7f42780112c0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 shutdown_connections 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a1f4ebf520 0x55a1f4ec1940 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a1f4e43f50 0x55a1f4e48940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a1f4e39690 0x55a1f4e48400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f4265ffb640 0 client.15765 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 --2- 172.21.15.19:0/3244241837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a1f4e38d70 0x55a1f4e47ec0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 >> 172.21.15.19:0/3244241837 conn(0x55a1f4e35300 msgr2=0x55a1f4e3afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 shutdown_connections 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 1 -- 172.21.15.19:0/3244241837 wait complete. 2024-09-12T03:21:00.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.053+0000 7f42869da740 20 client.15765 trim_cache size 0 max 16384 2024-09-12T03:21:00.057 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:00.057 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:21:00.057 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "20e6ad3c-db7b-4021-9086-47e1e7ba5c0c", 2024-09-12T03:21:00.057 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:21:00.057 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 msgr2=0x7f42680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 0x7f42680a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f426c006010 tx=0x7f426c0095f0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 msgr2=0x7f427c19ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4274009fd0 tx=0x7f427400b260 comp rx=0 tx=0).stop 2024-09-12T03:21:00.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 shutdown_connections 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f42680a2b80 0x7f42680a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f427c107af0 0x7f427c19ccb0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f427c102380 0x7f427c198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 --2- 172.21.15.19:0/1166927035 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f427c1019e0 0x7f427c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 >> 172.21.15.19:0/1166927035 conn(0x7f427c0fd700 msgr2=0x7f427c0fdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:00.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 shutdown_connections 2024-09-12T03:21:00.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:00.054+0000 7f428339f640 1 -- 172.21.15.19:0/1166927035 wait complete. 2024-09-12T03:21:00.737 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:00 smithi019 ceph-mon[30953]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T03:21:00.738 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:00 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:21:00.766 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:00 smithi135 ceph-mon[35474]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T03:21:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:00 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:21:00.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:00 smithi121 ceph-mon[34963]: pgmap v396: 97 pgs: 97 active+clean; 54 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.6 MiB/s wr, 230 op/s 2024-09-12T03:21:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:00 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:21:01.084 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:01.475 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.476 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.477 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:01.478 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:01 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 -- 172.21.15.19:0/684112476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e010e440 msgr2=0x7f27e0110830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 --2- 172.21.15.19:0/684112476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e010e440 0x7f27e0110830 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f27d8009ca0 tx=0x7f27d802a950 comp rx=0 tx=0).stop 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 -- 172.21.15.19:0/684112476 shutdown_connections 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 --2- 172.21.15.19:0/684112476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e010e440 0x7f27e0110830 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 --2- 172.21.15.19:0/684112476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e0074e30 0x7f27e010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 --2- 172.21.15.19:0/684112476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074510 0x7f27e00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 -- 172.21.15.19:0/684112476 >> 172.21.15.19:0/684112476 conn(0x7f27e00fd8c0 msgr2=0x7f27e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 -- 172.21.15.19:0/684112476 shutdown_connections 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.569+0000 7f27e5e96640 1 -- 172.21.15.19:0/684112476 wait complete. 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.570+0000 7f27e5e96640 1 Processor -- start 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.570+0000 7f27e5e96640 1 -- start start 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074e30 0x7f27e019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e010e440 0x7f27e019eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0114160 con 0x7f27e0074510 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27e0113fe0 con 0x7f27e010e440 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01142e0 con 0x7f27e0074e30 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43456/0 (socket says 172.21.15.19:43456) 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 -- 172.21.15.19:0/1548583741 learned_addr learned my addr 172.21.15.19:0/1548583741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27deffd640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074e30 0x7f27e019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27dffff640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e010e440 0x7f27e019eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 -- 172.21.15.19:0/1548583741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074e30 msgr2=0x7f27e019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074e30 0x7f27e019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 -- 172.21.15.19:0/1548583741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e010e440 msgr2=0x7f27e019eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e010e440 0x7f27e019eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 -- 172.21.15.19:0/1548583741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27e019f5a0 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27df7fe640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e9f0 tx=0x7f27d000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27dcff9640 1 -- 172.21.15.19:0/1548583741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d000ccc0 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27e019f890 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27dcff9640 1 -- 172.21.15.19:0/1548583741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d000ce60 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27dcff9640 1 -- 172.21.15.19:0/1548583741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d0010690 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.571+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27e01a7170 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27dcff9640 1 -- 172.21.15.19:0/1548583741 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27d00046c0 con 0x7f27e0074510 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27e826f740 1 Processor -- start 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27e826f740 1 -- start start 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27e826f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27e826f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c50140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.574+0000 7f27e826f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c52a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27e826f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563936bab2e0 con 0x563936c50680 2024-09-12T03:21:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27e826f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563936b52ce0 con 0x563936c454a0 2024-09-12T03:21:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27e826f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563936ba72a0 con 0x563936c45dc0 2024-09-12T03:21:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c52a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27df7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c50140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39996/0 (socket says 172.21.15.19:39996) 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27deffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c52a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43470/0 (socket says 172.21.15.19:43470) 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 -- 172.21.15.19:0/2784868720 learned_addr learned my addr 172.21.15.19:0/2784868720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 -- 172.21.15.19:0/2784868720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 msgr2=0x563936c50140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c50140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 -- 172.21.15.19:0/2784868720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 msgr2=0x563936c52a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c52a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 -- 172.21.15.19:0/2784868720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563936b37440 con 0x563936c454a0 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27df7fe640 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dcff9640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 0x7f27c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dffff640 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e00754f0 tx=0x7f27d8009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c7f804d7bb560f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27df7fe640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 0x7f27c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.575+0000 7f27dcff9640 1 -- 172.21.15.19:0/1548583741 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27d00cf450 con 0x7f27e0074510 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27c5ffb640 1 -- 172.21.15.19:0/2784868720 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8048070 con 0x563936c454a0 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27c5ffb640 1 -- 172.21.15.19:0/2784868720 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d802f940 con 0x563936c454a0 2024-09-12T03:21:01.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27c5ffb640 1 -- 172.21.15.19:0/2784868720 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d8034040 con 0x563936c454a0 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 -- 172.21.15.19:0/2784868720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 msgr2=0x563936c45880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27df7fe640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 0x7f27c80a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd2d0 tx=0x7f27d00d2f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27e00754f0 tx=0x7f27d8009820 comp rx=0 tx=0).stop 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 -- 172.21.15.19:0/2784868720 shutdown_connections 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c52a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c50140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 --2- 172.21.15.19:0/2784868720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c454a0 0x563936c45880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 -- 172.21.15.19:0/2784868720 >> 172.21.15.19:0/2784868720 conn(0x563936c41a30 msgr2=0x563936c41e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.576+0000 7f27e826f740 1 -- 172.21.15.19:0/2784868720 shutdown_connections 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 -- 172.21.15.19:0/2784868720 wait complete. 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 -- start start 2024-09-12T03:21:01.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 Processor -- start 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c50020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c4d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c4d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563936c53250 con 0x563936c50680 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563936b66050 con 0x563936c45dc0 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563936c533d0 con 0x563936c454a0 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.577+0000 7f27e826f740 10 client.0 _ll_register_callbacks cb 0x563936ba6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c4d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c4d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43482/0 (socket says 172.21.15.19:43482) 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27dffff640 1 -- 172.21.15.19:0/2533287275 learned_addr learned my addr 172.21.15.19:0/2533287275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c4d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27dffff640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c50020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 -- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 msgr2=0x563936c50020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c50020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 -- 172.21.15.19:0/2533287275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 msgr2=0x563936c4d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c4d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 -- 172.21.15.19:0/2533287275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56393691c3c0 con 0x563936c45dc0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27deffd640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c4d2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27e010f7d0 tx=0x7f27d4010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84af8425ab5db6fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27beffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d400fb10 con 0x563936c45dc0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27d400fcb0 con 0x563936c45dc0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.578+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d802f940 con 0x563936c45dc0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27d4005020 con 0x563936c45dc0 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 10 client.25310 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:01.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563936c5ecc0 con 0x563936c45dc0 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d802f940 con 0x563936c45dc0 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f27d4019410 con 0x563936c45dc0 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 10 client.25310 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 10 client.25310 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 20 client.25310 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936c60240 0x563936c62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 4 client.25310 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563936bb8850 con 0x563936c60240 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.579+0000 7f27dffff640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936c60240 0x563936c62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.580+0000 7f27beffd640 10 client.25310 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.580+0000 7f27dffff640 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936c60240 0x563936c62660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9386accd0f5d3d20 server_cookie=63aa761c89b86837 in_seq=0 out_seq=0 2024-09-12T03:21:01.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27beffd640 1 -- 172.21.15.19:0/2533287275 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563936bb8850 con 0x563936c60240 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27beffd640 10 client.25310 handle_command_reply: tid=0 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 client.25310 shutdown 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 msgr2=0x563936c4d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c4d2d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f27e010f7d0 tx=0x7f27d4010c80 comp rx=0 tx=0).stop 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 shutdown_connections 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c4d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936c60240 0x563936c62660 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c4d2d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 --2- 172.21.15.19:0/2533287275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c50020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 >> 172.21.15.19:0/2533287275 conn(0x563936c41a30 msgr2=0x563936c40b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27beffd640 0 client.25310 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 shutdown_connections 2024-09-12T03:21:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 1 -- 172.21.15.19:0/2533287275 wait complete. 2024-09-12T03:21:01.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.582+0000 7f27e826f740 20 client.25310 trim_cache size 0 max 16384 2024-09-12T03:21:01.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 Processor -- start 2024-09-12T03:21:01.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 -- start start 2024-09-12T03:21:01.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c5c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c67f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01af670 con 0x563936c50680 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01b1270 con 0x563936c45dc0 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27e826f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01afd40 con 0x563936c454a0 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c67f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27deffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c67f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43484/0 (socket says 172.21.15.19:43484) 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.601+0000 7f27dffff640 1 -- 172.21.15.19:0/175328421 learned_addr learned my addr 172.21.15.19:0/175328421 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27dffff640 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c5c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 -- 172.21.15.19:0/175328421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 msgr2=0x563936c5c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c5c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 -- 172.21.15.19:0/175328421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 msgr2=0x563936c67f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c67f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 -- 172.21.15.19:0/175328421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563936c4d290 con 0x563936c45dc0 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27deffd640 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c480c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27d4019900 tx=0x7f27d4010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51ada9f2fa2bd39f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27c57fa640 1 -- 172.21.15.19:0/175328421 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4028040 con 0x563936c45dc0 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27c57fa640 1 -- 172.21.15.19:0/175328421 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d400a440 con 0x563936c45dc0 2024-09-12T03:21:01.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.602+0000 7f27c57fa640 1 -- 172.21.15.19:0/175328421 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d4005540 con 0x563936c45dc0 2024-09-12T03:21:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 -- 172.21.15.19:0/175328421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 msgr2=0x563936c480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c480c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f27d4019900 tx=0x7f27d4010430 comp rx=0 tx=0).stop 2024-09-12T03:21:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 -- 172.21.15.19:0/175328421 shutdown_connections 2024-09-12T03:21:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c50680 0x563936c67f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c45dc0 0x563936c480c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 --2- 172.21.15.19:0/175328421 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c454a0 0x563936c5c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 -- 172.21.15.19:0/175328421 >> 172.21.15.19:0/175328421 conn(0x563936c41a30 msgr2=0x563936c4f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 -- 172.21.15.19:0/175328421 shutdown_connections 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.603+0000 7f27e826f740 1 -- 172.21.15.19:0/175328421 wait complete. 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 -- start start 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 Processor -- start 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c454a0 0x563936c58620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c50680 0x563936c555f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01b0ae0 con 0x563936c454a0 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01b0410 con 0x563936c50680 2024-09-12T03:21:01.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f27e01ad650 con 0x563936c45dc0 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27e826f740 10 client.0 _ll_register_callbacks cb 0x7f27d400a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27dffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c454a0 0x563936c58620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27deffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:36206/0 (socket says 172.21.15.19:36206) 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.604+0000 7f27deffd640 1 -- 172.21.15.19:0/1086668605 learned_addr learned my addr 172.21.15.19:0/1086668605 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27dffff640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c50680 0x563936c555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 -- 172.21.15.19:0/1086668605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c50680 msgr2=0x563936c555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c50680 0x563936c555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 -- 172.21.15.19:0/1086668605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c454a0 msgr2=0x563936c58620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c454a0 0x563936c58620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 -- 172.21.15.19:0/1086668605 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563936c591d0 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27deffd640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27d40199a0 tx=0x7f27d4005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c07c65074bcdff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27beffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27d403a060 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f27d40195a0 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.605+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27d400a440 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f27d4028040 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 10 client.25333 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563936ccb490 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27d400a440 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f27d4004890 con 0x563936c45dc0 2024-09-12T03:21:01.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 10 client.25333 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 20 client.25333 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936ccbc60 0x563936cce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 4 client.25333 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563936c48f70 con 0x563936ccbc60 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.606+0000 7f27dffff640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936ccbc60 0x563936cce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.607+0000 7f27beffd640 10 client.25333 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:01.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.607+0000 7f27dffff640 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936ccbc60 0x563936cce080 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f3bc27efbd36cf server_cookie=de52d122a4f8d1a8 in_seq=0 out_seq=0 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27beffd640 1 -- 172.21.15.19:0/1086668605 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x563936c48f70 con 0x563936ccbc60 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27beffd640 10 client.25333 handle_command_reply: tid=0 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 client.25333 shutdown 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 msgr2=0x563936c58b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f27d40199a0 tx=0x7f27d4005d60 comp rx=0 tx=0).stop 2024-09-12T03:21:01.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 shutdown_connections 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563936ccbc60 0x563936cce080 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563936c50680 0x563936c555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563936c45dc0 0x563936c58b60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 --2- 172.21.15.19:0/1086668605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563936c454a0 0x563936c58620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 >> 172.21.15.19:0/1086668605 conn(0x563936c41a30 msgr2=0x563936c4bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.609+0000 7f27beffd640 0 client.25333 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 shutdown_connections 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e826f740 1 -- 172.21.15.19:0/1086668605 wait complete. 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e826f740 20 client.25333 trim_cache size 0 max 16384 2024-09-12T03:21:01.613 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (1128 inodes in the stack)", 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "20e6ad3c-db7b-4021-9086-47e1e7ba5c0c": { 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "20e6ad3c-db7b-4021-9086-47e1e7ba5c0c", 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:01.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 msgr2=0x7f27c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 0x7f27c80a5040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f27d00cd2d0 tx=0x7f27d00d2f30 comp rx=0 tx=0).stop 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 msgr2=0x7f27e019a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f27d000e9f0 tx=0x7f27d000edf0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 shutdown_connections 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f27e010e440 0x7f27e019eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f27e0074e30 0x7f27e019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f27c80a2b80 0x7f27c80a5040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 --2- 172.21.15.19:0/1548583741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f27e0074510 0x7f27e019a5f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:01.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.610+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 >> 172.21.15.19:0/1548583741 conn(0x7f27e00fd8c0 msgr2=0x7f27e00fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:01.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.611+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 shutdown_connections 2024-09-12T03:21:01.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:01.611+0000 7f27e5e96640 1 -- 172.21.15.19:0/1548583741 wait complete. 2024-09-12T03:21:01.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:01.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:01 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:01.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:01.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:01 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.345 INFO:tasks.cephfs.filesystem:scrub status for tag:20e6ad3c-db7b-4021-9086-47e1e7ba5c0c - {'path': '/', 'tag': '20e6ad3c-db7b-4021-9086-47e1e7ba5c0c', 'options': 'recursive,force'} 2024-09-12T03:21:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.669 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.671 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.672 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.673 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.674 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.675 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.676 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.677 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.678 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.679 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.680 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.681 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.682 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: pgmap v397: 97 pgs: 97 active+clean; 57 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 235 op/s 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.683 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.684 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.685 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.686 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.687 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.688 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.689 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.690 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.691 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.692 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.693 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.694 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.695 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.696 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.697 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.698 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.699 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.700 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.701 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.702 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.703 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.704 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.705 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.706 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.707 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.708 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.709 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.710 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.711 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.712 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.713 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.714 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.715 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.716 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.718 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.719 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:02 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: pgmap v397: 97 pgs: 97 active+clean; 57 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 235 op/s 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.847 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.848 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:02 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:02.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: pgmap v397: 97 pgs: 97 active+clean; 57 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 235 op/s 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.952 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.953 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.954 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.955 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.956 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.957 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.958 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.959 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:02.959 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:02.959 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:02 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.798 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.800 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.801 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.802 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.803 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.804 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.805 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.806 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.807 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.808 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.809 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.810 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.811 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.812 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.813 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.814 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.815 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.816 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.817 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.818 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.819 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.820 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.821 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.822 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.823 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.824 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.825 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.826 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.827 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.828 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.829 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.830 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.831 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.832 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.833 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.834 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.835 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.836 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.837 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.838 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.840 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.841 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.842 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.843 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.844 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.845 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.846 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.903 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.904 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.905 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.906 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.907 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.908 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.909 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.910 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.911 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.912 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.913 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.914 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.915 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.916 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.917 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.918 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.919 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.920 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.921 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.922 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.923 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.924 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.925 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.926 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.927 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.928 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.929 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.930 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.931 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.932 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.933 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.934 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.935 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.936 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.937 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.938 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.939 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.940 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.941 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.942 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.943 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.944 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.945 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.946 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.947 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.948 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.949 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.950 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.953 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.954 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.955 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.956 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.957 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.958 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.959 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.960 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.961 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.962 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.963 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.964 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.965 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.966 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.967 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.968 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.969 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.970 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.971 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.972 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.973 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.974 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.975 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.976 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.977 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.978 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.979 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.980 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.981 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.982 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.983 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.984 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.985 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.986 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.987 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.988 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.989 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.990 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.991 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.992 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.993 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.994 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.995 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.996 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.997 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.998 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:03.999 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.000 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:04.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.001 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:04 smithi135 ceph-mon[35474]: pgmap v398: 97 pgs: 97 active+clean; 59 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2024-09-12T03:21:04.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:04 smithi121 ceph-mon[34963]: pgmap v398: 97 pgs: 97 active+clean; 59 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2024-09-12T03:21:04.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:04 smithi019 ceph-mon[30953]: pgmap v398: 97 pgs: 97 active+clean; 59 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 214 op/s 2024-09-12T03:21:05.694 INFO:tasks.workunit.client.0.smithi019.stdout: 23 87250 413 64748 339 49114 1015 2024-09-12T03:21:06.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:06 smithi135 ceph-mon[35474]: pgmap v399: 97 pgs: 97 active+clean; 62 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-09-12T03:21:06.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:06 smithi121 ceph-mon[34963]: pgmap v399: 97 pgs: 97 active+clean; 62 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-09-12T03:21:06.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:06 smithi019 ceph-mon[30953]: pgmap v399: 97 pgs: 97 active+clean; 62 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 166 op/s 2024-09-12T03:21:08.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:08 smithi135 ceph-mon[35474]: pgmap v400: 97 pgs: 97 active+clean; 65 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-09-12T03:21:08.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:08 smithi121 ceph-mon[34963]: pgmap v400: 97 pgs: 97 active+clean; 65 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-09-12T03:21:08.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:08 smithi019 ceph-mon[30953]: pgmap v400: 97 pgs: 97 active+clean; 65 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 106 op/s 2024-09-12T03:21:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:09 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:09 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:09 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:09.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:09 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:09.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:09 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:09.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:09 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:09.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:09 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:09.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:09 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:09.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:09 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:09.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:09 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:09 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:09 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:09 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:09 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:09 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:10 smithi135 ceph-mon[35474]: pgmap v401: 97 pgs: 97 active+clean; 66 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-09-12T03:21:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:10 smithi121 ceph-mon[34963]: pgmap v401: 97 pgs: 97 active+clean; 66 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-09-12T03:21:10.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:10 smithi019 ceph-mon[30953]: pgmap v401: 97 pgs: 97 active+clean; 66 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-09-12T03:21:12.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:12 smithi135 ceph-mon[35474]: pgmap v402: 97 pgs: 97 active+clean; 70 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-12T03:21:12.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:12 smithi121 ceph-mon[34963]: pgmap v402: 97 pgs: 97 active+clean; 70 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-12T03:21:12.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:12 smithi019 ceph-mon[30953]: pgmap v402: 97 pgs: 97 active+clean; 70 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-12T03:21:13.152 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T03:21:13.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:13.756 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 -- 172.21.15.19:0/2475711823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 msgr2=0x7f9eec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 --2- 172.21.15.19:0/2475711823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec107b90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009860 tx=0x7f9edc02a340 comp rx=0 tx=0).stop 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 -- 172.21.15.19:0/2475711823 shutdown_connections 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 --2- 172.21.15.19:0/2475711823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 0x7f9eec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 --2- 172.21.15.19:0/2475711823 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec107b90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 --2- 172.21.15.19:0/2475711823 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 -- 172.21.15.19:0/2475711823 >> 172.21.15.19:0/2475711823 conn(0x7f9eec0fc7d0 msgr2=0x7f9eec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 -- 172.21.15.19:0/2475711823 shutdown_connections 2024-09-12T03:21:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.285+0000 7f9ef3767640 1 -- 172.21.15.19:0/2475711823 wait complete. 2024-09-12T03:21:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.286+0000 7f9ef3767640 1 Processor -- start 2024-09-12T03:21:14.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.286+0000 7f9ef3767640 1 -- start start 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 0x7f9eec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec112060 con 0x7f9eec102d30 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec111ee0 con 0x7f9eec1082a0 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef3767640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec1121e0 con 0x7f9eec1036e0 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef0d02640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44448/0 (socket says 172.21.15.19:44448) 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 -- 172.21.15.19:0/742789339 learned_addr learned my addr 172.21.15.19:0/742789339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1d04640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 0x7f9eec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 -- 172.21.15.19:0/742789339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 msgr2=0x7f9eec100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 -- 172.21.15.19:0/742789339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 msgr2=0x7f9eec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 0x7f9eec101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.287+0000 7f9ef1503640 1 -- 172.21.15.19:0/742789339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec1a6a60 con 0x7f9eec102d30 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ef0d02640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ef1503640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e9c0 tx=0x7f9ee800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800cde0 con 0x7f9eec102d30 2024-09-12T03:21:14.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ee80035f0 con 0x7f9eec102d30 2024-09-12T03:21:14.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec1a6d50 con 0x7f9eec102d30 2024-09-12T03:21:14.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8010690 con 0x7f9eec102d30 2024-09-12T03:21:14.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.288+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec1a7250 con 0x7f9eec102d30 2024-09-12T03:21:14.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.289+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec103110 con 0x7f9eec102d30 2024-09-12T03:21:14.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.292+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ee80037a0 con 0x7f9eec102d30 2024-09-12T03:21:14.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.293+0000 7f9ee27fc640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 0x7f9ed80a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.293+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9ee80cf3d0 con 0x7f9eec102d30 2024-09-12T03:21:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.293+0000 7f9ef0d02640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 0x7f9ed80a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:14.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.294+0000 7f9ef0d02640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 0x7f9ed80a5140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005940 tx=0x7f9edc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:14.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.297+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ee809c370 con 0x7f9eec102d30 2024-09-12T03:21:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.555+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eec101ec0 con 0x7f9eec102d30 2024-09-12T03:21:14.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.556+0000 7f9ee27fc640 1 -- 172.21.15.19:0/742789339 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7f9eec101ec0 con 0x7f9eec102d30 2024-09-12T03:21:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:21:14.557 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:21:14.557 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:21:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 msgr2=0x7f9ed80a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 0x7f9ed80a5140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9edc005940 tx=0x7f9edc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 msgr2=0x7f9eec100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e9c0 tx=0x7f9ee800ee80 comp rx=0 tx=0).stop 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 shutdown_connections 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9eec1082a0 0x7f9eec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9eec1036e0 0x7f9eec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9ed80a2c80 0x7f9ed80a5140 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 --2- 172.21.15.19:0/742789339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9eec102d30 0x7f9eec100990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 >> 172.21.15.19:0/742789339 conn(0x7f9eec0fc7d0 msgr2=0x7f9eec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 shutdown_connections 2024-09-12T03:21:14.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:14.558+0000 7f9ef3767640 1 -- 172.21.15.19:0/742789339 wait complete. 2024-09-12T03:21:14.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:14 smithi135 ceph-mon[35474]: pgmap v403: 97 pgs: 97 active+clean; 75 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-09-12T03:21:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:14 smithi121 ceph-mon[34963]: pgmap v403: 97 pgs: 97 active+clean; 75 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-09-12T03:21:14.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:14 smithi019 ceph-mon[30953]: pgmap v403: 97 pgs: 97 active+clean; 75 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 93 op/s 2024-09-12T03:21:15.574 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T03:21:15.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T03:21:15.694 INFO:tasks.workunit.client.0.smithi019.stdout: 54 547979 1724 362665 1554 208434 236 2024-09-12T03:21:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/742789339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:15.861 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:15.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/742789339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:15.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/742789339' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:16.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.468+0000 7f7bbd700640 1 -- 172.21.15.19:0/1727420451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 msgr2=0x7f7bb8110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.468+0000 7f7bb5ffb640 1 -- 172.21.15.19:0/1727420451 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac004230 con 0x7f7bb8109450 2024-09-12T03:21:16.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.468+0000 7f7bbd700640 1 --2- 172.21.15.19:0/1727420451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 0x7f7bb8110250 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009900 tx=0x7f7bac02a420 comp rx=0 tx=0).stop 2024-09-12T03:21:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 -- 172.21.15.19:0/1727420451 shutdown_connections 2024-09-12T03:21:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 --2- 172.21.15.19:0/1727420451 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 --2- 172.21.15.19:0/1727420451 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 0x7f7bb8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 --2- 172.21.15.19:0/1727420451 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 0x7f7bb8110250 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 -- 172.21.15.19:0/1727420451 >> 172.21.15.19:0/1727420451 conn(0x7f7bb806e140 msgr2=0x7f7bb806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 -- 172.21.15.19:0/1727420451 shutdown_connections 2024-09-12T03:21:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.469+0000 7f7bbd700640 1 -- 172.21.15.19:0/1727420451 wait complete. 2024-09-12T03:21:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.470+0000 7f7bbd700640 1 Processor -- start 2024-09-12T03:21:16.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.470+0000 7f7bbd700640 1 -- start start 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 0x7f7bb81a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 0x7f7bb81a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8120df0 con 0x7f7bb8109450 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8120c70 con 0x7f7bb8109830 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.471+0000 7f7bbd700640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb8120f70 con 0x7f7bb811b140 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.472+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.472+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:36676/0 (socket says 172.21.15.19:36676) 2024-09-12T03:21:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.472+0000 7f7bb77fe640 1 -- 172.21.15.19:0/3577961155 learned_addr learned my addr 172.21.15.19:0/3577961155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 0x7f7bb81a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb77fe640 1 -- 172.21.15.19:0/3577961155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 msgr2=0x7f7bb81a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 0x7f7bb81a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb77fe640 1 -- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 msgr2=0x7f7bb81a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 0x7f7bb81a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.473+0000 7f7bb77fe640 1 -- 172.21.15.19:0/3577961155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb81ac220 con 0x7f7bb811b140 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.474+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800d720 tx=0x7f7ba800dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.474+0000 7f7b9ffff640 1 -- 172.21.15.19:0/3577961155 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8017030 con 0x7f7bb811b140 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.474+0000 7f7b9ffff640 1 -- 172.21.15.19:0/3577961155 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba800de30 con 0x7f7bb811b140 2024-09-12T03:21:16.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.474+0000 7f7b9ffff640 1 -- 172.21.15.19:0/3577961155 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8004ba0 con 0x7f7bb811b140 2024-09-12T03:21:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.475+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb81ac510 con 0x7f7bb811b140 2024-09-12T03:21:16.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.476+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb81b3df0 con 0x7f7bb811b140 2024-09-12T03:21:16.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.481+0000 7f7bbfad9740 1 Processor -- start 2024-09-12T03:21:16.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.482+0000 7f7bbfad9740 1 -- start start 2024-09-12T03:21:16.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.485+0000 7f7b9ffff640 1 -- 172.21.15.19:0/3577961155 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ba801a070 con 0x7f7bb811b140 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7b9ffff640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 0x7f7ba00a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x560643373b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7b9ffff640 1 -- 172.21.15.19:0/3577961155 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7ba80cdec0 con 0x7f7bb811b140 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5606432d92a0 con 0x560643374080 2024-09-12T03:21:16.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.486+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560643280b10 con 0x56064337e940 2024-09-12T03:21:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.487+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5606432d5260 con 0x560643373760 2024-09-12T03:21:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.487+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.487+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37694/0 (socket says 172.21.15.19:37694) 2024-09-12T03:21:16.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.487+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2878306689 learned_addr learned my addr 172.21.15.19:0/2878306689 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2878306689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 msgr2=0x560643373b40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:21:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 0x7f7ba00a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x560643373b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2878306689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 msgr2=0x56064337e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2878306689 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064304ab90 con 0x56064337e940 2024-09-12T03:21:16.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.489+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:16.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.490+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a2e10 tx=0x7f7ba80a1050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb329e6683b30a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.490+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 0x7f7ba00a5100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x560643370b50 tx=0x7f7bb0009440 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.491+0000 7f7b9d7fa640 1 -- 172.21.15.19:0/2878306689 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba809a760 con 0x56064337e940 2024-09-12T03:21:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.491+0000 7f7b9d7fa640 1 -- 172.21.15.19:0/2878306689 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80b3070 con 0x56064337e940 2024-09-12T03:21:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.491+0000 7f7b9d7fa640 1 -- 172.21.15.19:0/2878306689 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80a2eb0 con 0x56064337e940 2024-09-12T03:21:16.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.492+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2878306689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 msgr2=0x560643380d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.492+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80a2e10 tx=0x7f7ba80a1050 comp rx=0 tx=0).stop 2024-09-12T03:21:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.493+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2878306689 shutdown_connections 2024-09-12T03:21:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.493+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643380d30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.493+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.493+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2878306689 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x560643373b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.494+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2878306689 >> 172.21.15.19:0/2878306689 conn(0x56064336fcf0 msgr2=0x5606433700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.496+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2878306689 shutdown_connections 2024-09-12T03:21:16.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.498+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2878306689 wait complete. 2024-09-12T03:21:16.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.499+0000 7f7bbfad9740 1 -- start start 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.499+0000 7f7bbfad9740 1 Processor -- start 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x56064337c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5606433815b0 con 0x560643374080 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5606432616e0 con 0x56064337e940 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.500+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x56064337c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.501+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x56064337c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37708/0 (socket says 172.21.15.19:37708) 2024-09-12T03:21:16.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.501+0000 7f7bb77fe640 1 -- 172.21.15.19:0/283444568 learned_addr learned my addr 172.21.15.19:0/283444568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.501+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560643381730 con 0x560643373760 2024-09-12T03:21:16.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.501+0000 7f7bbfad9740 10 client.0 _ll_register_callbacks cb 0x560643285a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.501+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 -- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 msgr2=0x56064337b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 -- 172.21.15.19:0/283444568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 msgr2=0x56064337c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x56064337c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.502+0000 7f7bb67fc640 1 -- 172.21.15.19:0/283444568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064338aac0 con 0x560643374080 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.503+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337bd30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7bb811d8f0 tx=0x7f7bb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed2a9643594cc6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.503+0000 7f7b9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.503+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb001cd60 con 0x560643374080 2024-09-12T03:21:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.504+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7bb001cb60 con 0x560643374080 2024-09-12T03:21:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.504+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba80b3070 con 0x560643374080 2024-09-12T03:21:16.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.504+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bb001cd60 con 0x560643374080 2024-09-12T03:21:16.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.507+0000 7f7bbfad9740 10 client.15795 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:16.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.507+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56064338d400 con 0x560643374080 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.507+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80b3070 con 0x560643374080 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.508+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7bb000f050 con 0x560643374080 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 10 client.15795 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 10 client.15795 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 20 client.15795 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x560643390c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 4 client.15795 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:16.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.510+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606432e69d0 con 0x56064338e860 2024-09-12T03:21:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.514+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x560643390c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.514+0000 7f7b9aff5640 10 client.15795 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:16.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.515+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x560643390c90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ed5be120a2b6f2 server_cookie=44f4e66540b218e in_seq=0 out_seq=0 2024-09-12T03:21:16.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.527+0000 7f7b9aff5640 1 -- 172.21.15.19:0/283444568 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606432e69d0 con 0x56064338e860 2024-09-12T03:21:16.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.527+0000 7f7b9aff5640 10 client.15795 handle_command_reply: tid=0 2024-09-12T03:21:16.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.529+0000 7f7bbfad9740 1 client.15795 shutdown 2024-09-12T03:21:16.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.534+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 msgr2=0x56064337bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.534+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337bd30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7bb811d8f0 tx=0x7f7bb0004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 shutdown_connections 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x560643390c90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7b9aff5640 0 client.15795 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x56064337c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064337bd30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/283444568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.536+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 >> 172.21.15.19:0/283444568 conn(0x56064336fcf0 msgr2=0x560643375a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.539+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 shutdown_connections 2024-09-12T03:21:16.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.540+0000 7f7bbfad9740 1 -- 172.21.15.19:0/283444568 wait complete. 2024-09-12T03:21:16.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.541+0000 7f7bbfad9740 20 client.15795 trim_cache size 0 max 16384 2024-09-12T03:21:16.542 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:16 smithi019 ceph-mon[30953]: pgmap v404: 97 pgs: 97 active+clean; 75 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.603+0000 7f7bbfad9740 1 Processor -- start 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 -- start start 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064336ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81bc770 con 0x560643374080 2024-09-12T03:21:16.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81be370 con 0x56064337e940 2024-09-12T03:21:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.604+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81bce40 con 0x560643373760 2024-09-12T03:21:16.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.606+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.607+0000 7f7bb77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37712/0 (socket says 172.21.15.19:37712) 2024-09-12T03:21:16.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.607+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2802901718 learned_addr learned my addr 172.21.15.19:0/2802901718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.609+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064336ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.610+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.611+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2802901718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 msgr2=0x56064337b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.611+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.611+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2802901718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 msgr2=0x56064336ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.611+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064336ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.611+0000 7f7bb77fe640 1 -- 172.21.15.19:0/2802901718 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064336b260 con 0x56064337e940 2024-09-12T03:21:16.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.613+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ba809a6a0 tx=0x7f7ba800b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b8eb7bd3f47d5ab server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.613+0000 7f7b9cff9640 1 -- 172.21.15.19:0/2802901718 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8098a60 con 0x56064337e940 2024-09-12T03:21:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.613+0000 7f7b9cff9640 1 -- 172.21.15.19:0/2802901718 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80b3040 con 0x56064337e940 2024-09-12T03:21:16.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.613+0000 7f7b9cff9640 1 -- 172.21.15.19:0/2802901718 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80a3110 con 0x56064337e940 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.614+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2802901718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 msgr2=0x560643396560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ba809a6a0 tx=0x7f7ba800b400 comp rx=0 tx=0).stop 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2802901718 shutdown_connections 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x560643396560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643374080 0x56064336ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2802901718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643373760 0x56064337b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2802901718 >> 172.21.15.19:0/2802901718 conn(0x56064336fcf0 msgr2=0x56064336e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.615+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2802901718 shutdown_connections 2024-09-12T03:21:16.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.616+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2802901718 wait complete. 2024-09-12T03:21:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.616+0000 7f7bbfad9740 1 -- start start 2024-09-12T03:21:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.617+0000 7f7bbfad9740 1 Processor -- start 2024-09-12T03:21:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.617+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643373760 0x560643383050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.617+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bbfad9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x5606433f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81bdbe0 con 0x560643373760 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81bd510 con 0x56064337e940 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bbfad9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb81ba6f0 con 0x560643374080 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bb67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bb67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:36708/0 (socket says 172.21.15.19:36708) 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bb67fc640 1 -- 172.21.15.19:0/2430796138 learned_addr learned my addr 172.21.15.19:0/2430796138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:16.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.618+0000 7f7bb77fe640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x5606433f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.619+0000 7f7bbfad9740 10 client.0 _ll_register_callbacks cb 0x7f7bb00127f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643373760 0x560643383050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb67fc640 1 -- 172.21.15.19:0/2430796138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 msgr2=0x5606433f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x5606433f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb67fc640 1 -- 172.21.15.19:0/2430796138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643373760 msgr2=0x560643383050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643373760 0x560643383050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.620+0000 7f7bb67fc640 1 -- 172.21.15.19:0/2430796138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064336fb20 con 0x560643374080 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.621+0000 7f7bb67fc640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0012fd0 tx=0x7f7bb00094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bffca11bc5966a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:16.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.621+0000 7f7b9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:21:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.623+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb001cd60 con 0x560643374080 2024-09-12T03:21:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.623+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ba80b3040 con 0x560643374080 2024-09-12T03:21:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.623+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7bb001cb60 con 0x560643374080 2024-09-12T03:21:16.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.623+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7bb002f380 con 0x560643374080 2024-09-12T03:21:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.629+0000 7f7bbfad9740 10 client.25354 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:16.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.629+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5606433f9d40 con 0x560643374080 2024-09-12T03:21:16.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba80b3040 con 0x560643374080 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f7bb000f050 con 0x560643374080 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 10 client.25354 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 10 client.25354 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 20 client.25354 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x5606433fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 4 client.25354 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x5606432e0d20 con 0x56064338e860 2024-09-12T03:21:16.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.634+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x5606433fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:16.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.638+0000 7f7b9aff5640 10 client.25354 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:16.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:16.639+0000 7f7bb6ffd640 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x5606433fc5a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c8a307816c12244 server_cookie=5f3b63ba6b34abb1 in_seq=0 out_seq=0 2024-09-12T03:21:16.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:16 smithi135 ceph-mon[35474]: pgmap v404: 97 pgs: 97 active+clean; 75 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-12T03:21:16.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:16 smithi121 ceph-mon[34963]: pgmap v404: 97 pgs: 97 active+clean; 75 MiB data, 454 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 85 op/s 2024-09-12T03:21:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:18 smithi135 ceph-mon[35474]: pgmap v405: 97 pgs: 97 active+clean; 108 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-09-12T03:21:18.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:18 smithi121 ceph-mon[34963]: pgmap v405: 97 pgs: 97 active+clean; 108 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-09-12T03:21:18.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:18 smithi019 ceph-mon[30953]: pgmap v405: 97 pgs: 97 active+clean; 108 MiB data, 465 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 198 op/s 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: pgmap v406: 97 pgs: 97 active+clean; 140 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.6 MiB/s wr, 347 op/s 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: pgmap v406: 97 pgs: 97 active+clean; 140 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.6 MiB/s wr, 347 op/s 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: pgmap v406: 97 pgs: 97 active+clean; 140 MiB data, 506 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.6 MiB/s wr, 347 op/s 2024-09-12T03:21:20.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:21:20.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:20.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:21.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:21.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:21.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:21.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:21.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:21.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:21.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:21:21.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:21:21.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:21:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:22 smithi135 ceph-mon[35474]: pgmap v407: 97 pgs: 97 active+clean; 213 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 659 op/s 2024-09-12T03:21:23.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:22 smithi121 ceph-mon[34963]: pgmap v407: 97 pgs: 97 active+clean; 213 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 659 op/s 2024-09-12T03:21:23.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:22 smithi019 ceph-mon[30953]: pgmap v407: 97 pgs: 97 active+clean; 213 MiB data, 595 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 13 MiB/s wr, 659 op/s 2024-09-12T03:21:24.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7b9aff5640 1 -- 172.21.15.19:0/2430796138 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5606432e0d20 con 0x56064338e860 2024-09-12T03:21:24.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7b9aff5640 10 client.25354 handle_command_reply: tid=0 2024-09-12T03:21:24.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 client.25354 shutdown 2024-09-12T03:21:24.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 msgr2=0x560643383590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0012fd0 tx=0x7f7bb00094b0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 shutdown_connections 2024-09-12T03:21:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56064338e860 0x5606433fc5a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56064337e940 0x5606433f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560643374080 0x560643383590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 --2- 172.21.15.19:0/2430796138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560643373760 0x560643383050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 >> 172.21.15.19:0/2430796138 conn(0x56064336fcf0 msgr2=0x56064336ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7b9aff5640 0 client.25354 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 shutdown_connections 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 1 -- 172.21.15.19:0/2430796138 wait complete. 2024-09-12T03:21:24.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.191+0000 7f7bbfad9740 20 client.25354 trim_cache size 0 max 16384 2024-09-12T03:21:24.198 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-09-12T03:21:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:21:24.198 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 6, 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "ab77af49": { 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 6, 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:21:24.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 133.4, 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:24.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 msgr2=0x7f7ba00a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 0x7f7ba00a5100 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x560643370b50 tx=0x7f7bb0009440 comp rx=0 tx=0).stop 2024-09-12T03:21:24.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 msgr2=0x7f7bb81abb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800d720 tx=0x7f7ba800dbe0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 shutdown_connections 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7bb811b140 0x7f7bb81abb40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7bb8109830 0x7f7bb81a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7ba00a2c40 0x7f7ba00a5100 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 --2- 172.21.15.19:0/3577961155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7bb8109450 0x7f7bb81a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 >> 172.21.15.19:0/3577961155 conn(0x7f7bb806e140 msgr2=0x7f7bb811bd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 shutdown_connections 2024-09-12T03:21:24.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:24.192+0000 7f7bbd700640 1 -- 172.21.15.19:0/3577961155 wait complete. 2024-09-12T03:21:24.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:24 smithi019 ceph-mon[30953]: pgmap v408: 97 pgs: 97 active+clean; 229 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 788 op/s 2024-09-12T03:21:25.036 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ab77af49', quiesce took 11.9 seconds. Will release after: 1.5 2024-09-12T03:21:25.043 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:24 smithi135 ceph-mon[35474]: pgmap v408: 97 pgs: 97 active+clean; 229 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 788 op/s 2024-09-12T03:21:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:24 smithi121 ceph-mon[34963]: pgmap v408: 97 pgs: 97 active+clean; 229 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 14 MiB/s wr, 788 op/s 2024-09-12T03:21:25.695 INFO:tasks.workunit.client.0.smithi019.stdout: 54 45513 3 31932 108 12759 1 2024-09-12T03:21:26.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:26.812 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:26 smithi121 ceph-mon[34963]: pgmap v409: 97 pgs: 97 active+clean; 229 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-09-12T03:21:27.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:26 smithi019 ceph-mon[30953]: pgmap v409: 97 pgs: 97 active+clean; 229 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-09-12T03:21:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:26 smithi135 ceph-mon[35474]: pgmap v409: 97 pgs: 97 active+clean; 229 MiB data, 639 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-09-12T03:21:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 -- 172.21.15.19:0/917636696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102570 msgr2=0x7f81401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 --2- 172.21.15.19:0/917636696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102570 0x7f81401029b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8134009900 tx=0x7f813402a480 comp rx=0 tx=0).stop 2024-09-12T03:21:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 -- 172.21.15.19:0/917636696 shutdown_connections 2024-09-12T03:21:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 --2- 172.21.15.19:0/917636696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102ef0 0x7f814010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 --2- 172.21.15.19:0/917636696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102570 0x7f81401029b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 --2- 172.21.15.19:0/917636696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 -- 172.21.15.19:0/917636696 >> 172.21.15.19:0/917636696 conn(0x7f81400fc7d0 msgr2=0x7f81400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 -- 172.21.15.19:0/917636696 shutdown_connections 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.346+0000 7f8145c0b640 1 -- 172.21.15.19:0/917636696 wait complete. 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 Processor -- start 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 -- start start 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102570 0x7f8140110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 0x7f8140110fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f814006aaa0 con 0x7f8140102ef0 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f814006a920 con 0x7f8140102570 2024-09-12T03:21:27.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f8145c0b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f814006ac20 con 0x7f8140100e20 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f813f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.347+0000 7f813ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 0x7f8140110fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:46622/0 (socket says 172.21.15.19:46622) 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813f7fe640 1 -- 172.21.15.19:0/1108523697 learned_addr learned my addr 172.21.15.19:0/1108523697 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813effd640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102570 0x7f8140110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 -- 172.21.15.19:0/1108523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 msgr2=0x7f8140110520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 -- 172.21.15.19:0/1108523697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102570 msgr2=0x7f8140110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102570 0x7f8140110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 -- 172.21.15.19:0/1108523697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81401a6a20 con 0x7f8140102ef0 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813f7fe640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813ffff640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 0x7f8140110fa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f813800ca30 tx=0x7f813800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:27.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138007d90 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401a6d10 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8138005020 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401a7200 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.348+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813801d060 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.349+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8140101200 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.351+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81380051d0 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.351+0000 7f813cff9640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 0x7f81280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.351+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f81380cf610 con 0x7f8140102ef0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.351+0000 7f813f7fe640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 0x7f81280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.352+0000 7f813f7fe640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 0x7f81280a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8130011500 tx=0x7f8130005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:27.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.355+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8138014070 con 0x7f8140102ef0 2024-09-12T03:21:27.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.581+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8140109090 con 0x7f8140102ef0 2024-09-12T03:21:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.582+0000 7f813cff9640 1 -- 172.21.15.19:0/1108523697 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7f813809c580 con 0x7f8140102ef0 2024-09-12T03:21:27.582 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:21:27.582 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:21:27.582 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:21:27.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 msgr2=0x7f81280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:27.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 0x7f81280a5040 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8130011500 tx=0x7f8130005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 msgr2=0x7f8140110fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 0x7f8140110fa0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f813800ca30 tx=0x7f813800cef0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 shutdown_connections 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81280a2b80 0x7f81280a5040 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8140102ef0 0x7f8140110fa0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8140102570 0x7f8140110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 --2- 172.21.15.19:0/1108523697 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8140100e20 0x7f8140110520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 >> 172.21.15.19:0/1108523697 conn(0x7f81400fc7d0 msgr2=0x7f81400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 shutdown_connections 2024-09-12T03:21:27.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:27.584+0000 7f8145c0b640 1 -- 172.21.15.19:0/1108523697 wait complete. 2024-09-12T03:21:28.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1108523697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:28.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1108523697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:28.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1108523697' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:28.420 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id ab77af49 --release --await' 2024-09-12T03:21:28.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id ab77af49 --release --await 2024-09-12T03:21:28.698 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:29.127 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:28 smithi019 ceph-mon[30953]: pgmap v410: 97 pgs: 97 active+clean; 234 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-09-12T03:21:29.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:28 smithi121 ceph-mon[34963]: pgmap v410: 97 pgs: 97 active+clean; 234 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-09-12T03:21:29.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 -- 172.21.15.19:0/3906927268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 msgr2=0x7f6d9010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 --2- 172.21.15.19:0/3906927268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9010ea90 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009900 tx=0x7f6d8802a410 comp rx=0 tx=0).stop 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 -- 172.21.15.19:0/3906927268 shutdown_connections 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 --2- 172.21.15.19:0/3906927268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9010ea90 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 --2- 172.21.15.19:0/3906927268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90102790 0x7f6d901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 --2- 172.21.15.19:0/3906927268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90101de0 0x7f6d901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 -- 172.21.15.19:0/3906927268 >> 172.21.15.19:0/3906927268 conn(0x7f6d900fd8c0 msgr2=0x7f6d900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 -- 172.21.15.19:0/3906927268 shutdown_connections 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.220+0000 7f6d9768e640 1 -- 172.21.15.19:0/3906927268 wait complete. 2024-09-12T03:21:29.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 Processor -- start 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 -- start start 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90101de0 0x7f6d9019a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90102790 0x7f6d9019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90111f70 con 0x7f6d90106f10 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d90111df0 con 0x7f6d90101de0 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d9768e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901120f0 con 0x7f6d90102790 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d95c2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d95c2b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34200/0 (socket says 172.21.15.19:34200) 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.221+0000 7f6d95c2b640 1 -- 172.21.15.19:0/2329715572 learned_addr learned my addr 172.21.15.19:0/2329715572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d94c29640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90102790 0x7f6d9019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d9542a640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90101de0 0x7f6d9019a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 -- 172.21.15.19:0/2329715572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90102790 msgr2=0x7f6d9019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90102790 0x7f6d9019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 -- 172.21.15.19:0/2329715572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90101de0 msgr2=0x7f6d9019a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90101de0 0x7f6d9019a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 -- 172.21.15.19:0/2329715572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d9019f4f0 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005ab0 tx=0x7f6d88004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d8e7fc640 1 -- 172.21.15.19:0/2329715572 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8803f030 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d8e7fc640 1 -- 172.21.15.19:0/2329715572 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d88003ad0 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d9019f780 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d8e7fc640 1 -- 172.21.15.19:0/2329715572 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d88035070 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.222+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d901a70c0 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.225+0000 7f6d8e7fc640 1 -- 172.21.15.19:0/2329715572 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d88042030 con 0x7f6d90106f10 2024-09-12T03:21:29.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.225+0000 7f6d99a67740 1 Processor -- start 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.225+0000 7f6d99a67740 1 -- start start 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.225+0000 7f6d8e7fc640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 0x7f6d780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 0x5592c56afd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56b0240 0x5592c56b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592c560a620 con 0x5592c56a5060 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5592c55b1e50 con 0x5592c56a5980 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d99a67740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592c56065e0 con 0x5592c56b0240 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d8e7fc640 1 -- 172.21.15.19:0/2329715572 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d880ef910 con 0x7f6d90106f10 2024-09-12T03:21:29.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d94c29640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 0x7f6d780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34206/0 (socket says 172.21.15.19:34206) 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 -- 172.21.15.19:0/1038262993 learned_addr learned my addr 172.21.15.19:0/1038262993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56b0240 0x5592c56b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d94c29640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 0x5592c56afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 -- 172.21.15.19:0/1038262993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56b0240 msgr2=0x5592c56b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56b0240 0x5592c56b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 -- 172.21.15.19:0/1038262993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 msgr2=0x5592c56afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 0x5592c56afd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 -- 172.21.15.19:0/1038262993 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c55a2d40 con 0x5592c56a5060 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d94c29640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 0x5592c56afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.226+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d80010400 tx=0x7f6d800108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7925c2d1579f56d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d94c29640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 0x7f6d780a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5592c56a2450 tx=0x7f6d8400aad0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d777fe640 1 -- 172.21.15.19:0/1038262993 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003b70 con 0x5592c56a5060 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d777fe640 1 -- 172.21.15.19:0/1038262993 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80003d10 con 0x5592c56a5060 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d777fe640 1 -- 172.21.15.19:0/1038262993 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80011b40 con 0x5592c56a5060 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d99a67740 1 -- 172.21.15.19:0/1038262993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 msgr2=0x5592c56a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.227+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6d80010400 tx=0x7f6d800108c0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 -- 172.21.15.19:0/1038262993 shutdown_connections 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56b0240 0x5592c56b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56a5980 0x5592c56afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1038262993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56a5440 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 -- 172.21.15.19:0/1038262993 >> 172.21.15.19:0/1038262993 conn(0x5592c56a15f0 msgr2=0x5592c56a19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 -- 172.21.15.19:0/1038262993 shutdown_connections 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 -- 172.21.15.19:0/1038262993 wait complete. 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 -- start start 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.228+0000 7f6d99a67740 1 Processor -- start 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592c56b2e10 con 0x5592c56a5980 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5592c5592f10 con 0x5592c56b0240 2024-09-12T03:21:29.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592c56b2f90 con 0x5592c56a5060 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d99a67740 10 client.0 _ll_register_callbacks cb 0x5592c55b6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d94c29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d94c29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34220/0 (socket says 172.21.15.19:34220) 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:46680/0 (socket says 172.21.15.19:46680) 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d94c29640 1 -- 172.21.15.19:0/3277221838 learned_addr learned my addr 172.21.15.19:0/3277221838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 -- 172.21.15.19:0/3277221838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 msgr2=0x5592c56adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 -- 172.21.15.19:0/3277221838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 msgr2=0x5592c56ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 -- 172.21.15.19:0/3277221838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c5319580 con 0x5592c56a5060 2024-09-12T03:21:29.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d94c29640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d9542a640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d80000f80 tx=0x7f6d80004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb4c70fca1eda1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.229+0000 7f6d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80009c20 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d80011840 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d80003d10 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d80011a40 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d99a67740 10 client.25372 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592c53e3960 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80003d10 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6d800052b0 con 0x5592c56a5060 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.230+0000 7f6d99a67740 10 client.25372 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d99a67740 10 client.25372 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:29.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d99a67740 20 client.25372 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c56bfd50 0x5592c56c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d99a67740 4 client.25372 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c5617b90 con 0x5592c56bfd50 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d94c29640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c56bfd50 0x5592c56c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d94c29640 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c56bfd50 0x5592c56c2140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ab0f86446439795 server_cookie=640ee3150a8311f3 in_seq=0 out_seq=0 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.231+0000 7f6d74ff9640 10 client.25372 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d74ff9640 1 -- 172.21.15.19:0/3277221838 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592c5617b90 con 0x5592c56bfd50 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d74ff9640 10 client.25372 handle_command_reply: tid=0 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 client.25372 shutdown 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 msgr2=0x5592c56ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6d80000f80 tx=0x7f6d80004c70 comp rx=0 tx=0).stop 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 shutdown_connections 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c56bfd50 0x5592c56c2140 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5980 0x5592c56ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 --2- 172.21.15.19:0/3277221838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5060 0x5592c56ad150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d74ff9640 0 client.25372 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 >> 172.21.15.19:0/3277221838 conn(0x5592c56a15f0 msgr2=0x5592c56a0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 shutdown_connections 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 1 -- 172.21.15.19:0/3277221838 wait complete. 2024-09-12T03:21:29.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.233+0000 7f6d99a67740 20 client.25372 trim_cache size 0 max 16384 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 Processor -- start 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 -- start start 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901af6d0 con 0x5592c56a5060 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901b12d0 con 0x5592c56b0240 2024-09-12T03:21:29.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.256+0000 7f6d99a67740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901afda0 con 0x5592c56a5980 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d94c29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34228/0 (socket says 172.21.15.19:34228) 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 -- 172.21.15.19:0/2781803814 learned_addr learned my addr 172.21.15.19:0/2781803814 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 -- 172.21.15.19:0/2781803814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 msgr2=0x5592c56c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 -- 172.21.15.19:0/2781803814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 msgr2=0x5592c56c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 -- 172.21.15.19:0/2781803814 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c56ac100 con 0x5592c56a5060 2024-09-12T03:21:29.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d9542a640 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006010 tx=0x7f6d80010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d85500a807f44f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d76ffd640 1 -- 172.21.15.19:0/2781803814 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80009610 con 0x5592c56a5060 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d76ffd640 1 -- 172.21.15.19:0/2781803814 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80009bb0 con 0x5592c56a5060 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.257+0000 7f6d76ffd640 1 -- 172.21.15.19:0/2781803814 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8001b780 con 0x5592c56a5060 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 -- 172.21.15.19:0/2781803814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 msgr2=0x5592c56af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6d80006010 tx=0x7f6d80010e40 comp rx=0 tx=0).stop 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 -- 172.21.15.19:0/2781803814 shutdown_connections 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 --2- 172.21.15.19:0/2781803814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56af0c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 -- 172.21.15.19:0/2781803814 >> 172.21.15.19:0/2781803814 conn(0x5592c56a15f0 msgr2=0x5592c56ae9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 -- 172.21.15.19:0/2781803814 shutdown_connections 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.258+0000 7f6d99a67740 1 -- 172.21.15.19:0/2781803814 wait complete. 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 -- start start 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 Processor -- start 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56b4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901b0b40 con 0x5592c56a5060 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901b0470 con 0x5592c56b0240 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d901ad6b0 con 0x5592c56a5980 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d99a67740 10 client.0 _ll_register_callbacks cb 0x7f6d80005bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:29.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d9542a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34230/0 (socket says 172.21.15.19:34230) 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d94c29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.259+0000 7f6d9542a640 1 -- 172.21.15.19:0/1346923972 learned_addr learned my addr 172.21.15.19:0/1346923972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d95c2b640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 -- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 msgr2=0x5592c56b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56b3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 -- 172.21.15.19:0/1346923972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 msgr2=0x5592c56b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56b4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 -- 172.21.15.19:0/1346923972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c56b4770 con 0x5592c56a5060 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d9542a640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d80004db0 tx=0x7f6d80010f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dc8e54dc9890e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:29.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8003e030 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d80022040 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d80009bb0 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6d80009590 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d99a67740 10 client.15825 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.260+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5592c572a6f0 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d80009bb0 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f6d8001b7c0 con 0x5592c56a5060 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 10 client.15825 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 10 client.15825 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 20 client.15825 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:29.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c572acf0 0x5592c572d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 4 client.15825 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ab77af49", "release": true, "await": true}] 2024-09-12T03:21:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ab77af49", "release": true, "await": true}) -- 0x5592c56120a0 con 0x5592c572acf0 2024-09-12T03:21:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d94c29640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c572acf0 0x5592c572d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d94c29640 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c572acf0 0x5592c572d110 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b19fdbd6278883 server_cookie=c299879dd0ca2420 in_seq=0 out_seq=0 2024-09-12T03:21:29.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.261+0000 7f6d74ff9640 10 client.15825 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:28 smithi135 ceph-mon[35474]: pgmap v410: 97 pgs: 97 active+clean; 234 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-09-12T03:21:29.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.514+0000 7f6d74ff9640 1 -- 172.21.15.19:0/1346923972 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5592c56120a0 con 0x5592c572acf0 2024-09-12T03:21:29.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.514+0000 7f6d74ff9640 10 client.15825 handle_command_reply: tid=0 2024-09-12T03:21:29.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.514+0000 7f6d99a67740 1 client.15825 shutdown 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.514+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 msgr2=0x5592c56b8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.514+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6d80004db0 tx=0x7f6d80010f70 comp rx=0 tx=0).stop 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 shutdown_connections 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592c572acf0 0x5592c572d110 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592c56b0240 0x5592c56b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d74ff9640 0 client.15825 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592c56a5980 0x5592c56b3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 --2- 172.21.15.19:0/1346923972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592c56a5060 0x5592c56b8810 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 >> 172.21.15.19:0/1346923972 conn(0x5592c56a15f0 msgr2=0x5592c56a0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 shutdown_connections 2024-09-12T03:21:29.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 1 -- 172.21.15.19:0/1346923972 wait complete. 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.515+0000 7f6d99a67740 20 client.15825 trim_cache size 0 max 16384 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 8, 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "ab77af49": { 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 8, 2024-09-12T03:21:29.517 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 133.4, 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:21:29.518 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:29.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:29.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:29.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:29.520 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:29.520 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:29.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 msgr2=0x7f6d780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 0x7f6d780a50c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5592c56a2450 tx=0x7f6d8400aad0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 msgr2=0x7f6d9019ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f6d88005ab0 tx=0x7f6d88004270 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 shutdown_connections 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d780a2c00 0x7f6d780a50c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d90106f10 0x7f6d9019ee10 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d90102790 0x7f6d9019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 --2- 172.21.15.19:0/2329715572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d90101de0 0x7f6d9019a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 >> 172.21.15.19:0/2329715572 conn(0x7f6d900fd8c0 msgr2=0x7f6d900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 shutdown_connections 2024-09-12T03:21:29.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:29.516+0000 7f6d9768e640 1 -- 172.21.15.19:0/2329715572 wait complete. 2024-09-12T03:21:30.665 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ab77af49', total seconds elapsed: 17.5 2024-09-12T03:21:30.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:30 smithi019 ceph-mon[30953]: pgmap v411: 97 pgs: 97 active+clean; 238 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-12T03:21:31.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:30 smithi121 ceph-mon[34963]: pgmap v411: 97 pgs: 97 active+clean; 238 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-12T03:21:31.293 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:30 smithi135 ceph-mon[35474]: pgmap v411: 97 pgs: 97 active+clean; 238 MiB data, 649 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 689 op/s 2024-09-12T03:21:32.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:32.769 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:32 smithi121 ceph-mon[34963]: pgmap v412: 97 pgs: 97 active+clean; 238 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.8 MiB/s wr, 576 op/s 2024-09-12T03:21:33.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:32 smithi019 ceph-mon[30953]: pgmap v412: 97 pgs: 97 active+clean; 238 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.8 MiB/s wr, 576 op/s 2024-09-12T03:21:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:32 smithi135 ceph-mon[35474]: pgmap v412: 97 pgs: 97 active+clean; 238 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.8 MiB/s wr, 576 op/s 2024-09-12T03:21:33.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.348+0000 7f5d06273640 1 -- 172.21.15.19:0/3062486975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109830 msgr2=0x7f5d00109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.348+0000 7f5d06273640 1 --2- 172.21.15.19:0/3062486975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109830 0x7f5d00109c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009900 tx=0x7f5ce802a420 comp rx=0 tx=0).stop 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.352+0000 7f5d06273640 1 -- 172.21.15.19:0/3062486975 shutdown_connections 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.352+0000 7f5d06273640 1 --2- 172.21.15.19:0/3062486975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d0011b140 0x7f5d0011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.352+0000 7f5d06273640 1 --2- 172.21.15.19:0/3062486975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109830 0x7f5d00109c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.352+0000 7f5d06273640 1 --2- 172.21.15.19:0/3062486975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d00109450 0x7f5d00110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.352+0000 7f5d06273640 1 -- 172.21.15.19:0/3062486975 >> 172.21.15.19:0/3062486975 conn(0x7f5d0006e140 msgr2=0x7f5d0006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.356+0000 7f5d06273640 1 -- 172.21.15.19:0/3062486975 shutdown_connections 2024-09-12T03:21:33.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.359+0000 7f5d06273640 1 -- 172.21.15.19:0/3062486975 wait complete. 2024-09-12T03:21:33.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.360+0000 7f5d06273640 1 Processor -- start 2024-09-12T03:21:33.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 -- start start 2024-09-12T03:21:33.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d00109830 0x7f5d00113760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0011b140 0x7f5d00113ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00120e30 con 0x7f5d00109830 2024-09-12T03:21:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00120cb0 con 0x7f5d0011b140 2024-09-12T03:21:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.362+0000 7f5d06273640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00120fb0 con 0x7f5d00109450 2024-09-12T03:21:33.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40658/0 (socket says 172.21.15.19:40658) 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 -- 172.21.15.19:0/1947583956 learned_addr learned my addr 172.21.15.19:0/1947583956 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 -- 172.21.15.19:0/1947583956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0011b140 msgr2=0x7f5d00113ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0011b140 0x7f5d00113ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 -- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d00109830 msgr2=0x7f5d00113760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d00109830 0x7f5d00113760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.363+0000 7f5cfffff640 1 -- 172.21.15.19:0/1947583956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d00114210 con 0x7f5d00109450 2024-09-12T03:21:33.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.364+0000 7f5cfffff640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c890 tx=0x7f5cf000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.368+0000 7f5cfd7fa640 1 -- 172.21.15.19:0/1947583956 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0004220 con 0x7f5d00109450 2024-09-12T03:21:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.368+0000 7f5cfd7fa640 1 -- 172.21.15.19:0/1947583956 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf00043c0 con 0x7f5d00109450 2024-09-12T03:21:33.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.368+0000 7f5cfd7fa640 1 -- 172.21.15.19:0/1947583956 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00049c0 con 0x7f5d00109450 2024-09-12T03:21:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.370+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d00114500 con 0x7f5d00109450 2024-09-12T03:21:33.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.370+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d00118490 con 0x7f5d00109450 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 Processor -- start 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 -- start start 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8ea2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dee8dfd2e0 con 0x55dee8e974a0 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dee8da4ce0 con 0x55dee8e97dc0 2024-09-12T03:21:33.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.375+0000 7f5d0864c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dee8df92a0 con 0x55dee8ea2680 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38250/0 (socket says 172.21.15.19:38250) 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 -- 172.21.15.19:0/4216920196 learned_addr learned my addr 172.21.15.19:0/4216920196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 -- 172.21.15.19:0/4216920196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 msgr2=0x55dee8ea4a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea4a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 -- 172.21.15.19:0/4216920196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 msgr2=0x55dee8ea2140 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8ea2140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.376+0000 7f5d04810640 1 -- 172.21.15.19:0/4216920196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee8d89440 con 0x55dee8e974a0 2024-09-12T03:21:33.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.377+0000 7f5d04810640 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c50 tx=0x7f5cf800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c706c06dfc77fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.379+0000 7f5cf67fc640 1 -- 172.21.15.19:0/4216920196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8009e60 con 0x55dee8e974a0 2024-09-12T03:21:33.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.380+0000 7f5cf67fc640 1 -- 172.21.15.19:0/4216920196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8017070 con 0x55dee8e974a0 2024-09-12T03:21:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.381+0000 7f5d0864c740 1 -- 172.21.15.19:0/4216920196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 msgr2=0x55dee8e97880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.381+0000 7f5d0864c740 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009c50 tx=0x7f5cf800b420 comp rx=0 tx=0).stop 2024-09-12T03:21:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.383+0000 7f5d0864c740 1 -- 172.21.15.19:0/4216920196 shutdown_connections 2024-09-12T03:21:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.383+0000 7f5d0864c740 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.383+0000 7f5d0864c740 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8ea2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.383+0000 7f5d0864c740 1 --2- 172.21.15.19:0/4216920196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8e974a0 0x55dee8e97880 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.383+0000 7f5d0864c740 1 -- 172.21.15.19:0/4216920196 >> 172.21.15.19:0/4216920196 conn(0x55dee8e93a30 msgr2=0x55dee8e93e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.384+0000 7f5d0864c740 1 -- 172.21.15.19:0/4216920196 shutdown_connections 2024-09-12T03:21:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.384+0000 7f5d0864c740 1 -- 172.21.15.19:0/4216920196 wait complete. 2024-09-12T03:21:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.385+0000 7f5d0864c740 1 -- start start 2024-09-12T03:21:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.385+0000 7f5cfd7fa640 1 -- 172.21.15.19:0/1947583956 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cf0005120 con 0x7f5d00109450 2024-09-12T03:21:33.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.385+0000 7f5d0864c740 1 Processor -- start 2024-09-12T03:21:33.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8e9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dee8ea52f0 con 0x55dee8ea2680 2024-09-12T03:21:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dee8db8050 con 0x55dee8e97dc0 2024-09-12T03:21:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dee8ea5470 con 0x55dee8e974a0 2024-09-12T03:21:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.387+0000 7f5d0864c740 10 client.0 _ll_register_callbacks cb 0x55dee8df8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:33.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.389+0000 7f5cfd7fa640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 0x7f5cec0a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.389+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.389+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:46934/0 (socket says 172.21.15.19:46934) 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.389+0000 7f5cfffff640 1 -- 172.21.15.19:0/2223167071 learned_addr learned my addr 172.21.15.19:0/2223167071 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfd7fa640 1 -- 172.21.15.19:0/1947583956 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf00ceed0 con 0x7f5d00109450 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8e9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfffff640 1 -- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 msgr2=0x55dee8ea1ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfffff640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea1ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfffff640 1 -- 172.21.15.19:0/2223167071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 msgr2=0x55dee8e9f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfffff640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8e9f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.390+0000 7f5cfffff640 1 -- 172.21.15.19:0/2223167071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee8b6e3c0 con 0x55dee8e97dc0 2024-09-12T03:21:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.391+0000 7f5d04810640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 0x7f5cec0a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.391+0000 7f5cfffff640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007590 tx=0x7f5cf0005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc56b7d2d13bfec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.391+0000 7f5cdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.392+0000 7f5d04810640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 0x7f5cec0a5110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8012d20 tx=0x7f5cf8012d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.395+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b4060 con 0x55dee8e97dc0 2024-09-12T03:21:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.395+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf009ac30 con 0x55dee8e97dc0 2024-09-12T03:21:33.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.395+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf8017070 con 0x55dee8e97dc0 2024-09-12T03:21:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.403+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cf00b9050 con 0x55dee8e97dc0 2024-09-12T03:21:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.405+0000 7f5d0864c740 10 client.25355 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:33.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.405+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dee8eb0cf0 con 0x55dee8e97dc0 2024-09-12T03:21:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8017070 con 0x55dee8e97dc0 2024-09-12T03:21:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5cf00976d0 con 0x55dee8e97dc0 2024-09-12T03:21:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 10 client.25355 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:33.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 20 client.25355 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8eb21b0 0x55dee8eb45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 4 client.25355 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:33.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.406+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee8e0a850 con 0x55dee8eb21b0 2024-09-12T03:21:33.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.412+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8eb21b0 0x55dee8eb45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.416+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8eb21b0 0x55dee8eb45a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=379c8a942e2ac8c5 server_cookie=93790abd117e3472 in_seq=0 out_seq=0 2024-09-12T03:21:33.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.416+0000 7f5cdf7fe640 10 client.25355 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.419+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/2223167071 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dee8e0a850 con 0x55dee8eb21b0 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.419+0000 7f5cdf7fe640 10 client.25355 handle_command_reply: tid=0 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.419+0000 7f5d0864c740 1 client.25355 shutdown 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 msgr2=0x55dee8e9f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007590 tx=0x7f5cf0005de0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 shutdown_connections 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8e9f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8eb21b0 0x55dee8eb45a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e97dc0 0x55dee8e9f1e0 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007590 tx=0x7f5cf0005de0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 --2- 172.21.15.19:0/2223167071 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 >> 172.21.15.19:0/2223167071 conn(0x55dee8e93a30 msgr2=0x55dee8e99760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.420+0000 7f5cdf7fe640 0 client.25355 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:33.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.421+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 shutdown_connections 2024-09-12T03:21:33.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.422+0000 7f5d0864c740 1 -- 172.21.15.19:0/2223167071 wait complete. 2024-09-12T03:21:33.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.423+0000 7f5d0864c740 20 client.25355 trim_cache size 0 max 16384 2024-09-12T03:21:33.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.457+0000 7f5d0864c740 1 Processor -- start 2024-09-12T03:21:33.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.457+0000 7f5d0864c740 1 -- start start 2024-09-12T03:21:33.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c4d20 con 0x55dee8ea2680 2024-09-12T03:21:33.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c6920 con 0x55dee8eba360 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d0864c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c53f0 con 0x55dee8e974a0 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d04810640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38268/0 (socket says 172.21.15.19:38268) 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d04810640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:46938/0 (socket says 172.21.15.19:46938) 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 -- 172.21.15.19:0/3526733972 learned_addr learned my addr 172.21.15.19:0/3526733972 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:33.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cfffff640 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 -- 172.21.15.19:0/3526733972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 msgr2=0x55dee8ea0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 -- 172.21.15.19:0/3526733972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 msgr2=0x55dee8e9c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5cff7fe640 1 -- 172.21.15.19:0/3526733972 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee8ea18f0 con 0x55dee8ea2680 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.458+0000 7f5d04810640 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.459+0000 7f5cff7fe640 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802a990 tx=0x7f5ce802a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc1bef1f2b8c2d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.459+0000 7f5cf5ffb640 1 -- 172.21.15.19:0/3526733972 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8035040 con 0x55dee8ea2680 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.459+0000 7f5cf5ffb640 1 -- 172.21.15.19:0/3526733972 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8048070 con 0x55dee8ea2680 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.459+0000 7f5cf5ffb640 1 -- 172.21.15.19:0/3526733972 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004b80 con 0x55dee8ea2680 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 -- 172.21.15.19:0/3526733972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 msgr2=0x55dee8eb9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5ce802a990 tx=0x7f5ce802a920 comp rx=0 tx=0).stop 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 -- 172.21.15.19:0/3526733972 shutdown_connections 2024-09-12T03:21:33.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8eba360 0x55dee8e9c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8ea2680 0x55dee8eb9e20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3526733972 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8e974a0 0x55dee8ea0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 -- 172.21.15.19:0/3526733972 >> 172.21.15.19:0/3526733972 conn(0x55dee8e93a30 msgr2=0x55dee8e9f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 -- 172.21.15.19:0/3526733972 shutdown_connections 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.460+0000 7f5d0864c740 1 -- 172.21.15.19:0/3526733972 wait complete. 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.461+0000 7f5d0864c740 1 -- start start 2024-09-12T03:21:33.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.461+0000 7f5d0864c740 1 Processor -- start 2024-09-12T03:21:33.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.461+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e974a0 0x55dee8ea8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.461+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.461+0000 7f5d0864c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 0x55dee8ea90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d0864c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c6190 con 0x55dee8eba360 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d0864c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c5ac0 con 0x55dee8e974a0 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d0864c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5d001c2d00 con 0x55dee8ea2680 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d0864c740 10 client.0 _ll_register_callbacks cb 0x7f5cf0005020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5cfffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e974a0 0x55dee8ea8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5cff7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40694/0 (socket says 172.21.15.19:40694) 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5cff7fe640 1 -- 172.21.15.19:0/3550168580 learned_addr learned my addr 172.21.15.19:0/3550168580 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 0x55dee8ea90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 -- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 msgr2=0x55dee8ea8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 -- 172.21.15.19:0/3550168580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e974a0 msgr2=0x55dee8ea8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e974a0 0x55dee8ea8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 -- 172.21.15.19:0/3550168580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee8f1bf70 con 0x55dee8eba360 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5d04810640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 0x55dee8ea90a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009bf0 tx=0x7f5cf800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e90df29582d374 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.462+0000 7f5cdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:33.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8032480 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8048070 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5cf8032620 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5cf8032820 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5d0864c740 10 client.15837 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55dee8f1d240 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.463+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ce8048070 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f5cf8011070 con 0x55dee8eba360 2024-09-12T03:21:33.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 10 client.15837 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 20 client.15837 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8f1d6d0 0x55dee8f1fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 4 client.15837 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dee8e04d60 con 0x55dee8f1d6d0 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.464+0000 7f5cfffff640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8f1d6d0 0x55dee8f1fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.465+0000 7f5cfffff640 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8f1d6d0 0x55dee8f1fac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba36fa89f642d0b server_cookie=a3dd596aa2c00d8f in_seq=0 out_seq=0 2024-09-12T03:21:33.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.465+0000 7f5cdf7fe640 10 client.15837 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.468+0000 7f5cdf7fe640 1 -- 172.21.15.19:0/3550168580 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55dee8e04d60 con 0x55dee8f1d6d0 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.468+0000 7f5cdf7fe640 10 client.15837 handle_command_reply: tid=0 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.468+0000 7f5d0864c740 1 client.15837 shutdown 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 msgr2=0x55dee8ea90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 0x55dee8ea90a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009bf0 tx=0x7f5cf800b420 comp rx=0 tx=0).stop 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 shutdown_connections 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dee8eba360 0x55dee8ea90a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dee8ea2680 0x55dee8ea8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dee8f1d6d0 0x55dee8f1fac0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 --2- 172.21.15.19:0/3550168580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dee8e974a0 0x55dee8ea8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 >> 172.21.15.19:0/3550168580 conn(0x55dee8e93a30 msgr2=0x55dee8e92350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5cdf7fe640 0 client.15837 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 shutdown_connections 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 1 -- 172.21.15.19:0/3550168580 wait complete. 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.469+0000 7f5d0864c740 20 client.15837 trim_cache size 0 max 16384 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:21:33.471 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:21:33.472 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 msgr2=0x7f5cec0a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 0x7f5cec0a5110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8012d20 tx=0x7f5cf8012d80 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 msgr2=0x7f5d0011a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c890 tx=0x7f5cf000cd50 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 shutdown_connections 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5d0011b140 0x7f5d00113ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.470+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5cec0a2c50 0x7f5cec0a5110 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.471+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5d00109830 0x7f5d00113760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.471+0000 7f5d06273640 1 --2- 172.21.15.19:0/1947583956 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5d00109450 0x7f5d0011a6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.471+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 >> 172.21.15.19:0/1947583956 conn(0x7f5d0006e140 msgr2=0x7f5d00070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:33.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.471+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 shutdown_connections 2024-09-12T03:21:33.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:33.471+0000 7f5d06273640 1 -- 172.21.15.19:0/1947583956 wait complete. 2024-09-12T03:21:34.317 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:21:34.317 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:34.604 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:35.032 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:34 smithi019 ceph-mon[30953]: pgmap v413: 97 pgs: 97 active+clean; 239 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T03:21:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 -- 172.21.15.19:0/2925733293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8106f10 msgr2=0x7faed810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 --2- 172.21.15.19:0/2925733293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8106f10 0x7faed810ea90 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faed0009ca0 tx=0x7faed002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 -- 172.21.15.19:0/2925733293 shutdown_connections 2024-09-12T03:21:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 --2- 172.21.15.19:0/2925733293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8106f10 0x7faed810ea90 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 --2- 172.21.15.19:0/2925733293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8102790 0x7faed81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 --2- 172.21.15.19:0/2925733293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8101de0 0x7faed81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 -- 172.21.15.19:0/2925733293 >> 172.21.15.19:0/2925733293 conn(0x7faed80fd8c0 msgr2=0x7faed80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 -- 172.21.15.19:0/2925733293 shutdown_connections 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.133+0000 7faee074c640 1 -- 172.21.15.19:0/2925733293 wait complete. 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 Processor -- start 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 -- start start 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8102790 0x7faed8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8106f10 0x7faed819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faed8111f70 con 0x7faed8101de0 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faed8111df0 con 0x7faed8106f10 2024-09-12T03:21:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faee074c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faed81120f0 con 0x7faed8102790 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38298/0 (socket says 172.21.15.19:38298) 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 -- 172.21.15.19:0/1258033234 learned_addr learned my addr 172.21.15.19:0/1258033234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faedece9640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8106f10 0x7faed819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faeddce7640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8102790 0x7faed8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 -- 172.21.15.19:0/1258033234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8102790 msgr2=0x7faed8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8102790 0x7faed8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 -- 172.21.15.19:0/1258033234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8106f10 msgr2=0x7faed819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8106f10 0x7faed819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.134+0000 7faede4e8640 1 -- 172.21.15.19:0/1258033234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed819d380 con 0x7faed8101de0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faede4e8640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faed400e9f0 tx=0x7faed400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed400cd80 con 0x7faed8101de0 2024-09-12T03:21:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed819d670 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faed4005500 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faed81a4f50 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.135+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed4010690 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.136+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faebc005290 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.138+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faed40046c0 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.139+0000 7faecf7fe640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 0x7faec40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.139+0000 7faeddce7640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 0x7faec40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.139+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faed40cf4c0 con 0x7faed8101de0 2024-09-12T03:21:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.139+0000 7faeddce7640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 0x7faec40a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faed81088e0 tx=0x7faec8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.142+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faed40541b0 con 0x7faed8101de0 2024-09-12T03:21:35.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:34 smithi121 ceph-mon[34963]: pgmap v413: 97 pgs: 97 active+clean; 239 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T03:21:35.293 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:34 smithi135 ceph-mon[35474]: pgmap v413: 97 pgs: 97 active+clean; 239 MiB data, 651 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 286 op/s 2024-09-12T03:21:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.372+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faebc005580 con 0x7faed8101de0 2024-09-12T03:21:35.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.372+0000 7faecf7fe640 1 -- 172.21.15.19:0/1258033234 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7faed409c430 con 0x7faed8101de0 2024-09-12T03:21:35.373 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:21:35.373 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:21:35.373 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 msgr2=0x7faec40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 0x7faec40a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7faed81088e0 tx=0x7faec8005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 msgr2=0x7faed81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faed400e9f0 tx=0x7faed400edf0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 shutdown_connections 2024-09-12T03:21:35.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faed8106f10 0x7faed819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faed8102790 0x7faed8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faec40a2b80 0x7faec40a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 --2- 172.21.15.19:0/1258033234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faed8101de0 0x7faed81983d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 >> 172.21.15.19:0/1258033234 conn(0x7faed80fd8c0 msgr2=0x7faed80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 shutdown_connections 2024-09-12T03:21:35.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:35.375+0000 7faee074c640 1 -- 172.21.15.19:0/1258033234 wait complete. 2024-09-12T03:21:35.695 INFO:tasks.workunit.client.0.smithi019.stdout: 67 293834 535 208715 494 77571 244 2024-09-12T03:21:36.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:35 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1258033234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:36.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:35 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1258033234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:36.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:35 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1258033234' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:36.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:21:36.897 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:37.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:36 smithi121 ceph-mon[34963]: pgmap v414: 97 pgs: 97 active+clean; 239 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-09-12T03:21:37.156 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:36 smithi019 ceph-mon[30953]: pgmap v414: 97 pgs: 97 active+clean; 239 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-09-12T03:21:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:36 smithi135 ceph-mon[35474]: pgmap v414: 97 pgs: 97 active+clean; 239 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.0 MiB/s wr, 143 op/s 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 -- 172.21.15.19:0/2088627152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c106f10 msgr2=0x7f568c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 --2- 172.21.15.19:0/2088627152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c106f10 0x7f568c10ea90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5684009ca0 tx=0x7f568402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 -- 172.21.15.19:0/2088627152 shutdown_connections 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 --2- 172.21.15.19:0/2088627152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c106f10 0x7f568c10ea90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 --2- 172.21.15.19:0/2088627152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c102790 0x7f568c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 --2- 172.21.15.19:0/2088627152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c101de0 0x7f568c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 -- 172.21.15.19:0/2088627152 >> 172.21.15.19:0/2088627152 conn(0x7f568c0fd880 msgr2=0x7f568c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 -- 172.21.15.19:0/2088627152 shutdown_connections 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.385+0000 7f5690f9c640 1 -- 172.21.15.19:0/2088627152 wait complete. 2024-09-12T03:21:37.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.386+0000 7f5690f9c640 1 Processor -- start 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.386+0000 7f5690f9c640 1 -- start start 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 0x7f568c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 0x7f568c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 0x7f568c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f568c111f70 con 0x7f568c102790 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f568c111df0 con 0x7f568c106f10 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5690f9c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1120f0 con 0x7f568c101de0 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 0x7f568c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 0x7f568c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 0x7f568c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40734/0 (socket says 172.21.15.19:40734) 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 0x7f568c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568a59c640 1 -- 172.21.15.19:0/4005999052 learned_addr learned my addr 172.21.15.19:0/4005999052 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 0x7f568c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38318/0 (socket says 172.21.15.19:38318) 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 -- 172.21.15.19:0/4005999052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 msgr2=0x7f568c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 0x7f568c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 -- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 msgr2=0x7f568c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 0x7f568c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 -- 172.21.15.19:0/4005999052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c19d380 con 0x7f568c106f10 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.387+0000 7f568ad9d640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 0x7f568c19cca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f568402ade0 tx=0x7f56840319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.388+0000 7f56837fe640 1 -- 172.21.15.19:0/4005999052 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684003b70 con 0x7f568c106f10 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.388+0000 7f56837fe640 1 -- 172.21.15.19:0/4005999052 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5684003d10 con 0x7f568c106f10 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.388+0000 7f56837fe640 1 -- 172.21.15.19:0/4005999052 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568402fc40 con 0x7f568c106f10 2024-09-12T03:21:37.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.388+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c19d610 con 0x7f568c106f10 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.388+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1a4f50 con 0x7f568c106f10 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.390+0000 7f56837fe640 1 -- 172.21.15.19:0/4005999052 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5684004050 con 0x7f568c106f10 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 Processor -- start 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 -- start start 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86ddb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b86d342f0 con 0x563b86dce5d0 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563b86cdbcf0 con 0x563b86dcfb40 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5693375740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b86d302b0 con 0x563b86dd9370 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f56837fe640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f568a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5683fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f568a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38326/0 (socket says 172.21.15.19:38326) 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f568a59c640 1 -- 172.21.15.19:0/3252614335 learned_addr learned my addr 172.21.15.19:0/3252614335 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.391+0000 7f5683fff640 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86ddb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568ad9d640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f56837fe640 1 -- 172.21.15.19:0/4005999052 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f56840eff80 con 0x7f568c106f10 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 -- 172.21.15.19:0/3252614335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 msgr2=0x563b86ddb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86ddb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 -- 172.21.15.19:0/3252614335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 msgr2=0x563b86dce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 -- 172.21.15.19:0/3252614335 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b86cc03a0 con 0x563b86dce5d0 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f5683fff640 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:37.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568a59c640 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f567c009e10 tx=0x7f567c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3fea535d629296e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f568ad9d640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 0x7f56780a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56840bc4f0 tx=0x7f5684040fc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f5680ff9640 1 -- 172.21.15.19:0/3252614335 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c01e070 con 0x563b86dce5d0 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f5680ff9640 1 -- 172.21.15.19:0/3252614335 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c019b10 con 0x563b86dce5d0 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.392+0000 7f5680ff9640 1 -- 172.21.15.19:0/3252614335 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c00fb30 con 0x563b86dce5d0 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 -- 172.21.15.19:0/3252614335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 msgr2=0x563b86dcea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f567c009e10 tx=0x7f567c011810 comp rx=0 tx=0).stop 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 -- 172.21.15.19:0/3252614335 shutdown_connections 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86ddb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcea10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 --2- 172.21.15.19:0/3252614335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 -- 172.21.15.19:0/3252614335 >> 172.21.15.19:0/3252614335 conn(0x563b86d97310 msgr2=0x563b86dcac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 -- 172.21.15.19:0/3252614335 shutdown_connections 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.393+0000 7f5693375740 1 -- 172.21.15.19:0/3252614335 wait complete. 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 -- start start 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 Processor -- start 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86dd8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563b86ddc070 con 0x563b86dd9370 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563b86cef060 con 0x563b86dcfb40 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563b86ddc1f0 con 0x563b86dce5d0 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5693375740 10 client.0 _ll_register_callbacks cb 0x563b86a428c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5683fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86dd8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.451 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38328/0 (socket says 172.21.15.19:38328) 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f5683fff640 1 -- 172.21.15.19:0/1039754859 learned_addr learned my addr 172.21.15.19:0/1039754859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.394+0000 7f568a59c640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 -- 172.21.15.19:0/1039754859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 msgr2=0x563b86dd8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86dd8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 -- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 msgr2=0x563b86dd5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 -- 172.21.15.19:0/1039754859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b86aa53c0 con 0x563b86dd9370 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5683fff640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5674011620 tx=0x7f56740096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cfea2cf4f272cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567401a040 con 0x563b86dd9370 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5674020460 con 0x563b86dd9370 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567c019b10 con 0x563b86dd9370 2024-09-12T03:21:37.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f5674004020 con 0x563b86dd9370 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5693375740 10 client.15855 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.395+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b86de7af0 con 0x563b86dd9370 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f567c019b10 con 0x563b86dd9370 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f56740035f0 con 0x563b86dd9370 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 10 client.15855 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 10 client.15855 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 20 client.15855 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86de8fb0 0x563b86deb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 4 client.15855 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b86d41860 con 0x563b86de8fb0 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f568a59c640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86de8fb0 0x563b86deb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f5671ffb640 10 client.15855 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.396+0000 7f568a59c640 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86de8fb0 0x563b86deb3c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d591fc587057a6a4 server_cookie=33e9fdd53e8858c4 in_seq=0 out_seq=0 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5671ffb640 1 -- 172.21.15.19:0/1039754859 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b86d41860 con 0x563b86de8fb0 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5671ffb640 10 client.15855 handle_command_reply: tid=0 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 client.15855 shutdown 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 msgr2=0x563b86dd6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5674011620 tx=0x7f56740096a0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 shutdown_connections 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd6560 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86de8fb0 0x563b86deb3c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 --2- 172.21.15.19:0/1039754859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86dd8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.454 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 >> 172.21.15.19:0/1039754859 conn(0x563b86d97310 msgr2=0x563b86dd0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5671ffb640 0 client.15855 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 shutdown_connections 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.397+0000 7f5693375740 1 -- 172.21.15.19:0/1039754859 wait complete. 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.398+0000 7f5693375740 20 client.15855 trim_cache size 0 max 16384 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.412+0000 7f5693375740 1 Processor -- start 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.412+0000 7f5693375740 1 -- start start 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86de5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86df0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1ab350 con 0x563b86dce5d0 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1acf50 con 0x563b86dcfb40 2024-09-12T03:21:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5693375740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1aba20 con 0x563b86dd9370 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f568a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86de5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38332/0 (socket says 172.21.15.19:38332) 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 -- 172.21.15.19:0/1610129691 learned_addr learned my addr 172.21.15.19:0/1610129691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86df0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 -- 172.21.15.19:0/1610129691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 msgr2=0x563b86df0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86df0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 -- 172.21.15.19:0/1610129691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 msgr2=0x563b86de5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86de5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 -- 172.21.15.19:0/1610129691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b86dcf300 con 0x563b86dce5d0 2024-09-12T03:21:37.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f568a59c640 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86de5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5683fff640 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5674011f70 tx=0x7f5674001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b69919ad21073d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.413+0000 7f5673fff640 1 -- 172.21.15.19:0/1610129691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5674030070 con 0x563b86dce5d0 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5673fff640 1 -- 172.21.15.19:0/1610129691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5674031040 con 0x563b86dce5d0 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5673fff640 1 -- 172.21.15.19:0/1610129691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56740052a0 con 0x563b86dce5d0 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 -- 172.21.15.19:0/1610129691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 msgr2=0x563b86dcae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5674011f70 tx=0x7f5674001e50 comp rx=0 tx=0).stop 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 -- 172.21.15.19:0/1610129691 shutdown_connections 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dd9370 0x563b86df0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86de5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 --2- 172.21.15.19:0/1610129691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dce5d0 0x563b86dcae10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 -- 172.21.15.19:0/1610129691 >> 172.21.15.19:0/1610129691 conn(0x563b86d97310 msgr2=0x563b86dca700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 -- 172.21.15.19:0/1610129691 shutdown_connections 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.414+0000 7f5693375740 1 -- 172.21.15.19:0/1610129691 wait complete. 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 -- start start 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 Processor -- start 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86de1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1ac7c0 con 0x563b86dd9370 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1ac0f0 con 0x563b86dcfb40 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1a9330 con 0x563b86dce5d0 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5693375740 10 client.0 _ll_register_callbacks cb 0x7f5674020ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5683fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38348/0 (socket says 172.21.15.19:38348) 2024-09-12T03:21:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f568a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5683fff640 1 -- 172.21.15.19:0/3456942862 learned_addr learned my addr 172.21.15.19:0/3456942862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.415+0000 7f5683fff640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86de1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 -- 172.21.15.19:0/3456942862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 msgr2=0x563b86de1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86de1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 -- 172.21.15.19:0/3456942862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 msgr2=0x563b86dd30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 -- 172.21.15.19:0/3456942862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b86dd3760 con 0x563b86dcfb40 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f568a59c640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd2b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567c011d10 tx=0x7f567c009a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395e8e87784547d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c01e040 con 0x563b86dcfb40 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f567c005220 con 0x563b86dcfb40 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5674031040 con 0x563b86dcfb40 2024-09-12T03:21:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f567c005420 con 0x563b86dcfb40 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5693375740 10 client.25385 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563b86e53af0 con 0x563b86dcfb40 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.416+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5674031040 con 0x563b86dcfb40 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f567c003680 con 0x563b86dcfb40 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 10 client.25385 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 10 client.25385 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 20 client.25385 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86e54040 0x563b86e56460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 4 client.25385 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b86d3bd70 con 0x563b86e54040 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5683fff640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86e54040 0x563b86e56460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5683fff640 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86e54040 0x563b86e56460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4650d88fb23156b4 server_cookie=373c5d6bdb438b2 in_seq=0 out_seq=0 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5671ffb640 10 client.25385 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5671ffb640 1 -- 172.21.15.19:0/3456942862 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b86d3bd70 con 0x563b86e54040 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5671ffb640 10 client.25385 handle_command_reply: tid=0 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 client.25385 shutdown 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 msgr2=0x563b86dd2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd2b80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567c011d10 tx=0x7f567c009a10 comp rx=0 tx=0).stop 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 shutdown_connections 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.417+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563b86dd9370 0x563b86dd30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563b86dcfb40 0x563b86dd2b80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x563b86e54040 0x563b86e56460 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 --2- 172.21.15.19:0/3456942862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563b86dce5d0 0x563b86de1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 >> 172.21.15.19:0/3456942862 conn(0x563b86d97310 msgr2=0x563b86dd7820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5671ffb640 0 client.25385 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 shutdown_connections 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 1 -- 172.21.15.19:0/3456942862 wait complete. 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5693375740 20 client.25385 trim_cache size 0 max 16384 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 msgr2=0x7f56780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 0x7f56780a5040 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56840bc4f0 tx=0x7f5684040fc0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 msgr2=0x7f568c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 0x7f568c19cca0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f568402ade0 tx=0x7f56840319f0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 shutdown_connections 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f568c106f10 0x7f568c19cca0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f56780a2b80 0x7f56780a5040 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f568c102790 0x7f568c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 --2- 172.21.15.19:0/4005999052 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f568c101de0 0x7f568c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.418+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 >> 172.21.15.19:0/4005999052 conn(0x7f568c0fd880 msgr2=0x7f568c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.419+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 shutdown_connections 2024-09-12T03:21:37.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:37.419+0000 7f5690f9c640 1 -- 172.21.15.19:0/4005999052 wait complete. 2024-09-12T03:21:38.224 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:21:38.511 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.049+0000 7f85bdeb9640 1 -- 172.21.15.19:0/2397212011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8102dd0 msgr2=0x7f85b81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.049+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/2397212011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8102dd0 0x7f85b81031b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009930 tx=0x7f85a802a420 comp rx=0 tx=0).stop 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.049+0000 7f85bdeb9640 1 -- 172.21.15.19:0/2397212011 shutdown_connections 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.049+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/2397212011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8108300 0x7f85b810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.049+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/2397212011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8103780 0x7f85b8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/2397212011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8102dd0 0x7f85b81031b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 -- 172.21.15.19:0/2397212011 >> 172.21.15.19:0/2397212011 conn(0x7f85b80fc7d0 msgr2=0x7f85b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 -- 172.21.15.19:0/2397212011 shutdown_connections 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 -- 172.21.15.19:0/2397212011 wait complete. 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 Processor -- start 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.050+0000 7f85bdeb9640 1 -- start start 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 0x7f85b8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 0x7f85b819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8112090 con 0x7f85b8108300 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8111f10 con 0x7f85b8102dd0 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85bdeb9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8112210 con 0x7f85b8103780 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47012/0 (socket says 172.21.15.19:47012) 2024-09-12T03:21:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 -- 172.21.15.19:0/848550653 learned_addr learned my addr 172.21.15.19:0/848550653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b7fff640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 0x7f85b819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 0x7f85b8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 -- 172.21.15.19:0/848550653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 msgr2=0x7f85b8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 0x7f85b8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 -- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 msgr2=0x7f85b819cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 0x7f85b819cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 -- 172.21.15.19:0/848550653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b819d600 con 0x7f85b8102dd0 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 0x7f85b8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b7fff640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 0x7f85b819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.051+0000 7f85b77fe640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.052+0000 7f85b4ff9640 1 -- 172.21.15.19:0/848550653 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a803f030 con 0x7f85b8102dd0 2024-09-12T03:21:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.052+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b819d890 con 0x7f85b8102dd0 2024-09-12T03:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.052+0000 7f85b4ff9640 1 -- 172.21.15.19:0/848550653 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85a8003ad0 con 0x7f85b8102dd0 2024-09-12T03:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.052+0000 7f85b4ff9640 1 -- 172.21.15.19:0/848550653 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8035070 con 0x7f85b8102dd0 2024-09-12T03:21:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.052+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b81a51d0 con 0x7f85b8102dd0 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.054+0000 7f85b4ff9640 1 -- 172.21.15.19:0/848550653 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85a8042030 con 0x7f85b8102dd0 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.054+0000 7f85c0292740 1 Processor -- start 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 -- start start 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 0x5596e32d2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e32e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596e32382f0 con 0x5596e32d3b40 2024-09-12T03:21:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5596e31dfcf0 con 0x5596e32e1780 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85c0292740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596e32342b0 con 0x5596e32d25d0 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 0x5596e32d2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 0x5596e32d2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38368/0 (socket says 172.21.15.19:38368) 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b7fff640 1 -- 172.21.15.19:0/3345284178 learned_addr learned my addr 172.21.15.19:0/3345284178 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e32e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b4ff9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 0x7f85a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b77fe640 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b7fff640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 0x7f85a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3345284178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 msgr2=0x5596e32d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3345284178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 msgr2=0x5596e32d2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 0x5596e32d2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3345284178 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e31c43a0 con 0x5596e32e1780 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.055+0000 7f85b4ff9640 1 -- 172.21.15.19:0/848550653 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f85a80ef710 con 0x7f85b8102dd0 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.056+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e32e3b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fefa0 tx=0x7f85ac00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6e9a03dacb1f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.056+0000 7f85b7fff640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 0x7f85a00a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005c40 tx=0x7f85b0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.056+0000 7f859dffb640 1 -- 172.21.15.19:0/3345284178 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac015070 con 0x5596e32e1780 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.056+0000 7f859dffb640 1 -- 172.21.15.19:0/3345284178 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac011920 con 0x5596e32e1780 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.056+0000 7f859dffb640 1 -- 172.21.15.19:0/3345284178 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac010930 con 0x5596e32e1780 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- 172.21.15.19:0/3345284178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 msgr2=0x5596e32e3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e32e3b70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fefa0 tx=0x7f85ac00f730 comp rx=0 tx=0).stop 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- 172.21.15.19:0/3345284178 shutdown_connections 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e32e3b70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 --2- 172.21.15.19:0/3345284178 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d3b40 0x5596e32d2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- 172.21.15.19:0/3345284178 >> 172.21.15.19:0/3345284178 conn(0x5596e329b310 msgr2=0x5596e32cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- 172.21.15.19:0/3345284178 shutdown_connections 2024-09-12T03:21:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- 172.21.15.19:0/3345284178 wait complete. 2024-09-12T03:21:39.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 -- start start 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.057+0000 7f85c0292740 1 Processor -- start 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5596e32e4480 con 0x5596e32e1780 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5596e31f3060 con 0x5596e32d3b40 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5596e32e4600 con 0x5596e32d25d0 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85c0292740 10 client.0 _ll_register_callbacks cb 0x5596e2f468c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38372/0 (socket says 172.21.15.19:38372) 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40790/0 (socket says 172.21.15.19:40790) 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 -- 172.21.15.19:0/1803839762 learned_addr learned my addr 172.21.15.19:0/1803839762 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 -- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 msgr2=0x5596e32e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 -- 172.21.15.19:0/1803839762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 msgr2=0x5596e32de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b7fff640 1 -- 172.21.15.19:0/1803839762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e2fa93c0 con 0x5596e32e1780 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.058+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f85b7fff640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f85b000ff90 tx=0x7f85b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe77e7b1ea3ce69c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85b0004d20 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f85b001e620 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac011920 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85b001e820 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f85c0292740 10 client.15873 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596e32efee0 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.059+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac011920 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f85b0027030 con 0x5596e32e1780 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 10 client.15873 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 10 client.15873 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 20 client.15873 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e32f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 4 client.15873 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596e3245860 con 0x5596e32f12f0 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85b77fe640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e32f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f85b77fe640 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e32f3760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=354b2184d9df7969 server_cookie=ddae4248404efea in_seq=0 out_seq=0 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.060+0000 7f8596ffd640 10 client.15873 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.061+0000 7f8596ffd640 1 -- 172.21.15.19:0/1803839762 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596e3245860 con 0x5596e32f12f0 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.061+0000 7f8596ffd640 10 client.15873 handle_command_reply: tid=0 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.061+0000 7f85c0292740 1 client.15873 shutdown 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 msgr2=0x5596e32de9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f85b000ff90 tx=0x7f85b0004a40 comp rx=0 tx=0).stop 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 shutdown_connections 2024-09-12T03:21:39.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e32f3760 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f8596ffd640 0 client.15873 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32e1780 0x5596e32de9e0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 --2- 172.21.15.19:0/1803839762 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d25d0 0x5596e32e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 >> 172.21.15.19:0/1803839762 conn(0x5596e329b310 msgr2=0x5596e32d4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 shutdown_connections 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 1 -- 172.21.15.19:0/1803839762 wait complete. 2024-09-12T03:21:39.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.062+0000 7f85c0292740 20 client.15873 trim_cache size 0 max 16384 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.076+0000 7f85c0292740 1 Processor -- start 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.076+0000 7f85c0292740 1 -- start start 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32cee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32e1780 0x5596e32f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ab370 con 0x5596e32d25d0 2024-09-12T03:21:39.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81acf70 con 0x5596e32d3b40 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85c0292740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81aba40 con 0x5596e32e1780 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47042/0 (socket says 172.21.15.19:47042) 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3861667843 learned_addr learned my addr 172.21.15.19:0/3861667843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b77fe640 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b7fff640 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32e1780 0x5596e32f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3861667843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32e1780 msgr2=0x5596e32f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32e1780 0x5596e32f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3861667843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 msgr2=0x5596e32cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.077+0000 7f85b6ffd640 1 -- 172.21.15.19:0/3861667843 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596e32dc940 con 0x5596e32d3b40 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.078+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fefa0 tx=0x7f85ac00ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119eab0349608d3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.078+0000 7f859d7fa640 1 -- 172.21.15.19:0/3861667843 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac015040 con 0x5596e32d3b40 2024-09-12T03:21:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.078+0000 7f859d7fa640 1 -- 172.21.15.19:0/3861667843 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac005040 con 0x5596e32d3b40 2024-09-12T03:21:39.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.078+0000 7f859d7fa640 1 -- 172.21.15.19:0/3861667843 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ac004aa0 con 0x5596e32d3b40 2024-09-12T03:21:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- 172.21.15.19:0/3861667843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 msgr2=0x5596e32ed720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f85b80fefa0 tx=0x7f85ac00ba00 comp rx=0 tx=0).stop 2024-09-12T03:21:39.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- 172.21.15.19:0/3861667843 shutdown_connections 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32e1780 0x5596e32f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32d3b40 0x5596e32ed720 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 --2- 172.21.15.19:0/3861667843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- 172.21.15.19:0/3861667843 >> 172.21.15.19:0/3861667843 conn(0x5596e329b310 msgr2=0x5596e32ce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- 172.21.15.19:0/3861667843 shutdown_connections 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- 172.21.15.19:0/3861667843 wait complete. 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.079+0000 7f85c0292740 1 -- start start 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 Processor -- start 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d3b40 0x5596e32e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e335b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ac7e0 con 0x5596e32d25d0 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81ac110 con 0x5596e32e1780 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f85b81a9350 con 0x5596e32d3b40 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.080+0000 7f85c0292740 10 client.0 _ll_register_callbacks cb 0x7f85b00137a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d3b40 0x5596e32e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38404/0 (socket says 172.21.15.19:38404) 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 -- 172.21.15.19:0/1450349074 learned_addr learned my addr 172.21.15.19:0/1450349074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b7fff640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e335b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 -- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d3b40 msgr2=0x5596e32e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d3b40 0x5596e32e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 -- 172.21.15.19:0/1450349074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 msgr2=0x5596e335b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:39.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e335b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 -- 172.21.15.19:0/1450349074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596e32e6830 con 0x5596e32d25d0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b7fff640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e335b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85b77fe640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f85a80b88f0 tx=0x7f85a80ed090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8371adff2b2357e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f8596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8049040 con 0x5596e32d25d0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f85a80ed2d0 con 0x5596e32d25d0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.081+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85ac005040 con 0x5596e32d25d0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f85a80c4b90 con 0x5596e32d25d0 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 10 client.15879 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:39.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5596e335c700 con 0x5596e32d25d0 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85ac005040 con 0x5596e32d25d0 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f85a80b8590 con 0x5596e32d25d0 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 10 client.15879 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 10 client.15879 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 20 client.15879 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e335ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 4 client.15879 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596e323fd70 con 0x5596e32f12f0 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.082+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e335ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:39.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f8596ffd640 10 client.15879 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85b6ffd640 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e335ee80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6dfaf419b57554ed server_cookie=d0bf4c1a3b40b420 in_seq=0 out_seq=0 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f8596ffd640 1 -- 172.21.15.19:0/1450349074 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596e323fd70 con 0x5596e32f12f0 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f8596ffd640 10 client.15879 handle_command_reply: tid=0 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 client.15879 shutdown 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 msgr2=0x5596e32e5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f85a80b88f0 tx=0x7f85a80ed090 comp rx=0 tx=0).stop 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 shutdown_connections 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5596e32f12f0 0x5596e335ee80 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5596e32e1780 0x5596e335b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5596e32d3b40 0x5596e32e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 --2- 172.21.15.19:0/1450349074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5596e32d25d0 0x5596e32e5a10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f8596ffd640 0 client.15879 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:21:39.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 >> 172.21.15.19:0/1450349074 conn(0x5596e329b310 msgr2=0x5596e32dfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 shutdown_connections 2024-09-12T03:21:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 1 -- 172.21.15.19:0/1450349074 wait complete. 2024-09-12T03:21:39.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.083+0000 7f85c0292740 20 client.15879 trim_cache size 0 max 16384 2024-09-12T03:21:39.089 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 msgr2=0x7f85a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 0x7f85a00a5040 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f85b0005c40 tx=0x7f85b0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 msgr2=0x7f85b8198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f85a8009230 tx=0x7f85a8004270 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 shutdown_connections 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85a00a2b80 0x7f85a00a5040 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f85b8108300 0x7f85b819cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f85b8103780 0x7f85b8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 --2- 172.21.15.19:0/848550653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f85b8102dd0 0x7f85b8198650 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 >> 172.21.15.19:0/848550653 conn(0x7f85b80fc7d0 msgr2=0x7f85b8105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:39.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 shutdown_connections 2024-09-12T03:21:39.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:39.084+0000 7f85bdeb9640 1 -- 172.21.15.19:0/848550653 wait complete. 2024-09-12T03:21:39.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:38 smithi121 ceph-mon[34963]: pgmap v415: 97 pgs: 97 active+clean; 240 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-12T03:21:39.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:38 smithi019 ceph-mon[30953]: pgmap v415: 97 pgs: 97 active+clean; 240 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-12T03:21:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:38 smithi135 ceph-mon[35474]: pgmap v415: 97 pgs: 97 active+clean; 240 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-12T03:21:39.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:21:40.253 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:41.023 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:40 smithi019 ceph-mon[30953]: pgmap v416: 97 pgs: 97 active+clean; 247 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 -- 172.21.15.19:0/4065755261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 msgr2=0x7f602010ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 --2- 172.21.15.19:0/4065755261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602010ea80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6018009ca0 tx=0x7f601802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 -- 172.21.15.19:0/4065755261 shutdown_connections 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 --2- 172.21.15.19:0/4065755261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602010ea80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 --2- 172.21.15.19:0/4065755261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 --2- 172.21.15.19:0/4065755261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f60201021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 -- 172.21.15.19:0/4065755261 >> 172.21.15.19:0/4065755261 conn(0x7f60200fd8b0 msgr2=0x7f60200ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 -- 172.21.15.19:0/4065755261 shutdown_connections 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.021+0000 7f6028747640 1 -- 172.21.15.19:0/4065755261 wait complete. 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 Processor -- start 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 -- start start 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f6020198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6020111fb0 con 0x7f6020106f00 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6020111e30 con 0x7f6020101dd0 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6028747640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6020112130 con 0x7f6020102780 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6026ce4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6026ce4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42850/0 (socket says 172.21.15.19:42850) 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f6026ce4640 1 -- 172.21.15.19:0/3575171929 learned_addr learned my addr 172.21.15.19:0/3575171929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.022+0000 7f60264e3640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f6020198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 -- 172.21.15.19:0/3575171929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 msgr2=0x7f60201989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 -- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 msgr2=0x7f602019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 -- 172.21.15.19:0/3575171929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602019d420 con 0x7f6020101dd0 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f6025ce2640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f6026ce4640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60264e3640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f6020198470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60177fe640 1 -- 172.21.15.19:0/3575171929 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c00ccc0 con 0x7f6020101dd0 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60177fe640 1 -- 172.21.15.19:0/3575171929 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c010910 con 0x7f6020101dd0 2024-09-12T03:21:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f60177fe640 1 -- 172.21.15.19:0/3575171929 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018960 con 0x7f6020101dd0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602019d710 con 0x7f6020101dd0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.023+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60201a4ff0 con 0x7f6020101dd0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.026+0000 7f60177fe640 1 -- 172.21.15.19:0/3575171929 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f601c010ac0 con 0x7f6020101dd0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.026+0000 7f60177fe640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 0x7f600c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.026+0000 7f6025ce2640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 0x7f600c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.026+0000 7f60177fe640 1 -- 172.21.15.19:0/3575171929 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601c0cf3a0 con 0x7f6020101dd0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.026+0000 7f602ab20740 1 Processor -- start 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f6025ce2640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 0x7f600c0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60201087c0 tx=0x7f6010005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 -- start start 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c81b40 0x5595c8c80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c8b370 0x5595c8c8d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595c8be62f0 con 0x5595c8c8b370 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5595c8b8dcf0 con 0x5595c8c81b40 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f602ab20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5595c8be22b0 con 0x5595c8c805d0 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:46938/0 (socket says 172.21.15.19:46938) 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f60264e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c81b40 0x5595c8c80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f6025ce2640 1 -- 172.21.15.19:0/1800340 learned_addr learned my addr 172.21.15.19:0/1800340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.027+0000 7f6026ce4640 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c8b370 0x5595c8c8d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 -- 172.21.15.19:0/1800340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c81b40 msgr2=0x5595c8c80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c81b40 0x5595c8c80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 -- 172.21.15.19:0/1800340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c8b370 msgr2=0x5595c8c8d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c8b370 0x5595c8c8d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 -- 172.21.15.19:0/1800340 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c8b723a0 con 0x5595c8c805d0 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6025ce2640 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60100058c0 tx=0x7f6010014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e2467185748bdbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6014ff9640 1 -- 172.21.15.19:0/1800340 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6010026480 con 0x5595c8c805d0 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6014ff9640 1 -- 172.21.15.19:0/1800340 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601001c8d0 con 0x5595c8c805d0 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.028+0000 7f6014ff9640 1 -- 172.21.15.19:0/1800340 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601001dad0 con 0x5595c8c805d0 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- 172.21.15.19:0/1800340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 msgr2=0x5595c8c80a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f60100058c0 tx=0x7f6010014000 comp rx=0 tx=0).stop 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- 172.21.15.19:0/1800340 shutdown_connections 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c8b370 0x5595c8c8d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c805d0 0x5595c8c80a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 --2- 172.21.15.19:0/1800340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c81b40 0x5595c8c80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- 172.21.15.19:0/1800340 >> 172.21.15.19:0/1800340 conn(0x5595c8c49310 msgr2=0x5595c8c7cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- 172.21.15.19:0/1800340 shutdown_connections 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- 172.21.15.19:0/1800340 wait complete. 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 -- start start 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.029+0000 7f602ab20740 1 Processor -- start 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c8adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8c887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5595c8c8e070 con 0x5595c8c81b40 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5595c8ba1060 con 0x5595c8c805d0 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5595c8c8e1f0 con 0x5595c8c8b370 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f602ab20740 10 client.0 _ll_register_callbacks cb 0x5595c88f48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42876/0 (socket says 172.21.15.19:42876) 2024-09-12T03:21:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 -- 172.21.15.19:0/4240313760 learned_addr learned my addr 172.21.15.19:0/4240313760 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6026ce4640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8c887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f60264e3640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c8adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 -- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 msgr2=0x5595c8c887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8c887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 -- 172.21.15.19:0/4240313760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 msgr2=0x5595c8c8adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c8adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 -- 172.21.15.19:0/4240313760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c89573c0 con 0x5595c8c81b40 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f60264e3640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6025ce2640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6010011be0 tx=0x7f6010005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31791ede47cf7ddd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.030+0000 7f6009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601001c5f0 con 0x5595c8c81b40 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601001d7d0 con 0x5595c8c81b40 2024-09-12T03:21:41.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601001c8d0 con 0x5595c8c81b40 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f60100055e0 con 0x5595c8c81b40 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 10 client.15891 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595c8c99b90 con 0x5595c8c81b40 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601001c8d0 con 0x5595c8c81b40 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f601002e070 con 0x5595c8c81b40 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 10 client.15891 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 10 client.15891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 20 client.15891 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8c9aee0 0x5595c8c9d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 4 client.15891 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.031+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c8bf3860 con 0x5595c8c9aee0 2024-09-12T03:21:41.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.032+0000 7f60264e3640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8c9aee0 0x5595c8c9d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.032+0000 7f60264e3640 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8c9aee0 0x5595c8c9d2d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b4a757cada4f082d server_cookie=a2d4e88004876937 in_seq=0 out_seq=0 2024-09-12T03:21:41.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.032+0000 7f6009ffb640 10 client.15891 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f6009ffb640 1 -- 172.21.15.19:0/4240313760 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5595c8bf3860 con 0x5595c8c9aee0 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f6009ffb640 10 client.15891 handle_command_reply: tid=0 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 client.15891 shutdown 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 msgr2=0x5595c8c88060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6010011be0 tx=0x7f6010005430 comp rx=0 tx=0).stop 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 shutdown_connections 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8c9aee0 0x5595c8c9d2d0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8c887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c88060 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 --2- 172.21.15.19:0/4240313760 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c8adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 >> 172.21.15.19:0/4240313760 conn(0x5595c8c49310 msgr2=0x5595c8c82960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f6009ffb640 0 client.15891 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:21:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 shutdown_connections 2024-09-12T03:21:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.033+0000 7f602ab20740 1 -- 172.21.15.19:0/4240313760 wait complete. 2024-09-12T03:21:41.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.034+0000 7f602ab20740 20 client.15891 trim_cache size 0 max 16384 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 Processor -- start 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 -- start start 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8ca0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60201ab420 con 0x5595c8c81b40 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60201ad020 con 0x5595c8c805d0 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f602ab20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60201abaf0 con 0x5595c8c8b370 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f6025ce2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42888/0 (socket says 172.21.15.19:42888) 2024-09-12T03:21:41.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f6025ce2640 1 -- 172.21.15.19:0/1167362546 learned_addr learned my addr 172.21.15.19:0/1167362546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f6026ce4640 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8ca0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 -- 172.21.15.19:0/1167362546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 msgr2=0x5595c8ca0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8ca0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 -- 172.21.15.19:0/1167362546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 msgr2=0x5595c8c9ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f60264e3640 1 -- 172.21.15.19:0/1167362546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c8c7ddd0 con 0x5595c8c805d0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.048+0000 7f6025ce2640 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f60264e3640 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c7ce10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f601c09b960 tx=0x7f601c0ccd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3deb278b3b3693a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f600bfff640 1 -- 172.21.15.19:0/1167362546 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c07f070 con 0x5595c8c805d0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f600bfff640 1 -- 172.21.15.19:0/1167362546 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c0ccf60 con 0x5595c8c805d0 2024-09-12T03:21:41.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f600bfff640 1 -- 172.21.15.19:0/1167362546 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018d60 con 0x5595c8c805d0 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f602ab20740 1 -- 172.21.15.19:0/1167362546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 msgr2=0x5595c8c7ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.049+0000 7f602ab20740 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c7ce10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f601c09b960 tx=0x7f601c0ccd20 comp rx=0 tx=0).stop 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 -- 172.21.15.19:0/1167362546 shutdown_connections 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c8b370 0x5595c8ca0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c81b40 0x5595c8c9ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 --2- 172.21.15.19:0/1167362546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c805d0 0x5595c8c7ce10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 -- 172.21.15.19:0/1167362546 >> 172.21.15.19:0/1167362546 conn(0x5595c8c49310 msgr2=0x5595c8c7c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 -- 172.21.15.19:0/1167362546 shutdown_connections 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 -- 172.21.15.19:0/1167362546 wait complete. 2024-09-12T03:21:41.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 -- start start 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.050+0000 7f602ab20740 1 Processor -- start 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c81b40 0x5595c8c9e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c8b370 0x5595c8c9ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f60201ac890 con 0x5595c8c805d0 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f60201ac1c0 con 0x5595c8c8b370 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f60201a9400 con 0x5595c8c81b40 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f602ab20740 10 client.0 _ll_register_callbacks cb 0x7f6010005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42894/0 (socket says 172.21.15.19:42894) 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 -- 172.21.15.19:0/284951919 learned_addr learned my addr 172.21.15.19:0/284951919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f6025ce2640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c81b40 0x5595c8c9e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f6026ce4640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c8b370 0x5595c8c9ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 -- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c81b40 msgr2=0x5595c8c9e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c81b40 0x5595c8c9e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 -- 172.21.15.19:0/284951919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c8b370 msgr2=0x5595c8c9ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c8b370 0x5595c8c9ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 -- 172.21.15.19:0/284951919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c8c7c4f0 con 0x5595c8c805d0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f60264e3640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f601c09e010 tx=0x7f601c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783e4c422309d64f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f6009ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c07f050 con 0x5595c8c805d0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.051+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f601c0982d0 con 0x5595c8c805d0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f601c0ccf60 con 0x5595c8c805d0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f601c09b390 con 0x5595c8c805d0 2024-09-12T03:21:41.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 10 client.15897 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5595c8c7d960 con 0x5595c8c805d0 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f601c0aa030 con 0x5595c8c805d0 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f601c097b50 con 0x5595c8c805d0 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 10 client.15897 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 10 client.15897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 20 client.15897 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8d06110 0x5595c8d08530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 4 client.15897 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.052+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f601c0ccf60 con 0x5595c8d06110 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6025ce2640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8d06110 0x5595c8d08530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6009ffb640 10 client.15897 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6025ce2640 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8d06110 0x5595c8d08530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f41ffbea6f2b9dd5 server_cookie=a58c980ecc78e8e in_seq=0 out_seq=0 2024-09-12T03:21:41.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6009ffb640 1 -- 172.21.15.19:0/284951919 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f601c0ccf60 con 0x5595c8d06110 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6009ffb640 10 client.15897 handle_command_reply: tid=0 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 client.15897 shutdown 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 msgr2=0x5595c8c9dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f601c09e010 tx=0x7f601c00cf60 comp rx=0 tx=0).stop 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 shutdown_connections 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5595c8d06110 0x5595c8d08530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5595c8c8b370 0x5595c8c9ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5595c8c81b40 0x5595c8c9e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 --2- 172.21.15.19:0/284951919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5595c8c805d0 0x5595c8c9dfb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 >> 172.21.15.19:0/284951919 conn(0x5595c8c49310 msgr2=0x5595c8c7b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.053+0000 7f6009ffb640 0 client.15897 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 shutdown_connections 2024-09-12T03:21:41.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f602ab20740 1 -- 172.21.15.19:0/284951919 wait complete. 2024-09-12T03:21:41.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f602ab20740 20 client.15897 trim_cache size 0 max 16384 2024-09-12T03:21:41.059 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:21:41.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 msgr2=0x7f600c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 0x7f600c0a50c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f60201087c0 tx=0x7f6010005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 msgr2=0x7f6020198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f6020198470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f601c00e9f0 tx=0x7f601c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 shutdown_connections 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f600c0a2c00 0x7f600c0a50c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6020106f00 0x7f602019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6020102780 0x7f60201989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 --2- 172.21.15.19:0/3575171929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6020101dd0 0x7f6020198470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 >> 172.21.15.19:0/3575171929 conn(0x7f60200fd8b0 msgr2=0x7f60200feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 shutdown_connections 2024-09-12T03:21:41.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:41.054+0000 7f6028747640 1 -- 172.21.15.19:0/3575171929 wait complete. 2024-09-12T03:21:41.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:40 smithi121 ceph-mon[34963]: pgmap v416: 97 pgs: 97 active+clean; 247 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-12T03:21:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:40 smithi135 ceph-mon[35474]: pgmap v416: 97 pgs: 97 active+clean; 247 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-09-12T03:21:41.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:21:42.396 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:42.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 -- 172.21.15.19:0/3520715800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 msgr2=0x7effdc10e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 --2- 172.21.15.19:0/3520715800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc10e620 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7effd4009ca0 tx=0x7effd402a950 comp rx=0 tx=0).stop 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 -- 172.21.15.19:0/3520715800 shutdown_connections 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 --2- 172.21.15.19:0/3520715800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc10e620 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 --2- 172.21.15.19:0/3520715800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 0x7effdc10bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 --2- 172.21.15.19:0/3520715800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effdc102e10 0x7effdc1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.960+0000 7effe2c7e640 1 -- 172.21.15.19:0/3520715800 >> 172.21.15.19:0/3520715800 conn(0x7effdc0fc7d0 msgr2=0x7effdc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.961+0000 7effe2c7e640 1 -- 172.21.15.19:0/3520715800 shutdown_connections 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.961+0000 7effe2c7e640 1 -- 172.21.15.19:0/3520715800 wait complete. 2024-09-12T03:21:42.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.962+0000 7effe2c7e640 1 Processor -- start 2024-09-12T03:21:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.962+0000 7effe2c7e640 1 -- start start 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effdc102e10 0x7effdc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 0x7effdc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effdc111f90 con 0x7effdc10c1b0 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7effdc111e10 con 0x7effdc103730 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe2c7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effdc112110 con 0x7effdc102e10 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe121b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe121b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42918/0 (socket says 172.21.15.19:42918) 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effe121b640 1 -- 172.21.15.19:0/224381166 learned_addr learned my addr 172.21.15.19:0/224381166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:42.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.963+0000 7effdbfff640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 0x7effdc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 -- 172.21.15.19:0/224381166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effdc102e10 msgr2=0x7effdc19a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effdc102e10 0x7effdc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 -- 172.21.15.19:0/224381166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 msgr2=0x7effdc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 0x7effdc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 -- 172.21.15.19:0/224381166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effdc19f5e0 con 0x7effdc10c1b0 2024-09-12T03:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.964+0000 7effe121b640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7effd4009dc0 tx=0x7effd4003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.965+0000 7effd9ffb640 1 -- 172.21.15.19:0/224381166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd4034040 con 0x7effdc10c1b0 2024-09-12T03:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.965+0000 7effd9ffb640 1 -- 172.21.15.19:0/224381166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effd4003e50 con 0x7effdc10c1b0 2024-09-12T03:21:42.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.966+0000 7effd9ffb640 1 -- 172.21.15.19:0/224381166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd40046c0 con 0x7effdc10c1b0 2024-09-12T03:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.967+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effdc19f870 con 0x7effdc10c1b0 2024-09-12T03:21:42.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.967+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effdc19fcd0 con 0x7effdc10c1b0 2024-09-12T03:21:42.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.971+0000 7effe5057740 1 Processor -- start 2024-09-12T03:21:42.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 -- start start 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bed090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149beda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c02f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561149b532f0 con 0x561149beeb40 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561149afacf0 con 0x561149bed5d0 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effe5057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561149b4f2b0 con 0x561149c00b80 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.983+0000 7effd9ffb640 1 -- 172.21.15.19:0/224381166 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effd4037020 con 0x7effdc10c1b0 2024-09-12T03:21:42.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.985+0000 7effe0a1a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149beda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.985+0000 7effd9ffb640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 0x7effcc0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:42.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.985+0000 7effdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c02f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.985+0000 7effd9ffb640 1 -- 172.21.15.19:0/224381166 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7effd40efc60 con 0x7effdc10c1b0 2024-09-12T03:21:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.986+0000 7effdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c02f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47002/0 (socket says 172.21.15.19:47002) 2024-09-12T03:21:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.986+0000 7effdbfff640 1 -- 172.21.15.19:0/3549035556 learned_addr learned my addr 172.21.15.19:0/3549035556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:42.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.986+0000 7effdbfff640 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bed090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.988+0000 7effdbfff640 1 -- 172.21.15.19:0/3549035556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 msgr2=0x561149c02f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.988+0000 7effdbfff640 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c02f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.988+0000 7effdbfff640 1 -- 172.21.15.19:0/3549035556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 msgr2=0x561149beda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.988+0000 7effdbfff640 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149beda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:42.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.988+0000 7effdbfff640 1 -- 172.21.15.19:0/3549035556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561149adf3a0 con 0x561149beeb40 2024-09-12T03:21:42.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.990+0000 7effe121b640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 0x7effcc0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.990+0000 7effdbfff640 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bed090 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x561149bee070 tx=0x7effc800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc6b24f9eb1fe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.990+0000 7effc6ffd640 1 -- 172.21.15.19:0/3549035556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8017070 con 0x561149beeb40 2024-09-12T03:21:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.991+0000 7effc6ffd640 1 -- 172.21.15.19:0/3549035556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8010930 con 0x561149beeb40 2024-09-12T03:21:42.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.991+0000 7effc6ffd640 1 -- 172.21.15.19:0/3549035556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800d8d0 con 0x561149beeb40 2024-09-12T03:21:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.995+0000 7effe5057740 1 -- 172.21.15.19:0/3549035556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 msgr2=0x561149bed090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.995+0000 7effe5057740 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bed090 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x561149bee070 tx=0x7effc800b390 comp rx=0 tx=0).stop 2024-09-12T03:21:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe5057740 1 -- 172.21.15.19:0/3549035556 shutdown_connections 2024-09-12T03:21:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe5057740 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c02f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe5057740 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149beda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe5057740 1 --2- 172.21.15.19:0/3549035556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bed090 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe5057740 1 -- 172.21.15.19:0/3549035556 >> 172.21.15.19:0/3549035556 conn(0x561149bb6310 msgr2=0x561149be9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:42.999+0000 7effe121b640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 0x7effcc0a5140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7effd40c4bb0 tx=0x7effd4031fa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.000+0000 7effe5057740 1 -- 172.21.15.19:0/3549035556 shutdown_connections 2024-09-12T03:21:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.003+0000 7effe5057740 1 -- 172.21.15.19:0/3549035556 wait complete. 2024-09-12T03:21:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.003+0000 7effe5057740 1 -- start start 2024-09-12T03:21:43.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.004+0000 7effe5057740 1 Processor -- start 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bfab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561149bf06e0 con 0x561149beeb40 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561149b0e060 con 0x561149bed5d0 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561149bf0860 con 0x561149c00b80 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.009+0000 7effe5057740 10 client.0 _ll_register_callbacks cb 0x5611498618c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.010+0000 7effdbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.010+0000 7effdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.010+0000 7effdbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47596/0 (socket says 172.21.15.19:47596) 2024-09-12T03:21:43.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.010+0000 7effdbfff640 1 -- 172.21.15.19:0/1038891681 learned_addr learned my addr 172.21.15.19:0/1038891681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:43.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.012+0000 7effe0a1a640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bfab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.013+0000 7effdbfff640 1 -- 172.21.15.19:0/1038891681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 msgr2=0x561149c002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.013+0000 7effdbfff640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.013+0000 7effdbfff640 1 -- 172.21.15.19:0/1038891681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 msgr2=0x561149bfab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.013+0000 7effdbfff640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bfab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.013+0000 7effdbfff640 1 -- 172.21.15.19:0/1038891681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611498c43c0 con 0x561149bed5d0 2024-09-12T03:21:43.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.017+0000 7effdbfff640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effc800bcd0 tx=0x7effc800d430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f83722dff9eff3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.018+0000 7effbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:43.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.020+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8017070 con 0x561149bed5d0 2024-09-12T03:21:43.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.020+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7effc80106d0 con 0x561149bed5d0 2024-09-12T03:21:43.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.021+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc8010930 con 0x561149bed5d0 2024-09-12T03:21:43.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.023+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7effc8038040 con 0x561149bed5d0 2024-09-12T03:21:43.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.029+0000 7effe5057740 10 client.25433 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:43.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.029+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561149c17be0 con 0x561149bed5d0 2024-09-12T03:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8010930 con 0x561149bed5d0 2024-09-12T03:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7effc8004c10 con 0x561149bed5d0 2024-09-12T03:21:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 10 client.25433 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 10 client.25433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 20 client.25433 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c18f30 0x561149c1b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 4 client.25433 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:43.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.030+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561149b60860 con 0x561149c18f30 2024-09-12T03:21:43.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.031+0000 7effe0a1a640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c18f30 0x561149c1b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.039+0000 7effe0a1a640 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c18f30 0x561149c1b320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f18531f4f91499b2 server_cookie=f77d89fbc7e9a6d8 in_seq=0 out_seq=0 2024-09-12T03:21:43.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.042+0000 7effbbfff640 10 client.25433 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:21:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.042+0000 7effbbfff640 1 -- 172.21.15.19:0/1038891681 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561149b60860 con 0x561149c18f30 2024-09-12T03:21:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.042+0000 7effbbfff640 10 client.25433 handle_command_reply: tid=0 2024-09-12T03:21:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.042+0000 7effe5057740 1 client.25433 shutdown 2024-09-12T03:21:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 msgr2=0x561149bfa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effc800bcd0 tx=0x7effc800d430 comp rx=0 tx=0).stop 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 shutdown_connections 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c18f30 0x561149c1b320 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149c00b80 0x561149c002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149beeb40 0x561149bfab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 --2- 172.21.15.19:0/1038891681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149bed5d0 0x561149bfa5e0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7effc800bcd0 tx=0x7effc800d430 comp rx=0 tx=0).stop 2024-09-12T03:21:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 >> 172.21.15.19:0/1038891681 conn(0x561149bb6310 msgr2=0x561149bef930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.043+0000 7effbbfff640 0 client.25433 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:21:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.044+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 shutdown_connections 2024-09-12T03:21:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.044+0000 7effe5057740 1 -- 172.21.15.19:0/1038891681 wait complete. 2024-09-12T03:21:43.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.046+0000 7effe5057740 20 client.25433 trim_cache size 0 max 16384 2024-09-12T03:21:43.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.081+0000 7effe5057740 1 Processor -- start 2024-09-12T03:21:43.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.081+0000 7effe5057740 1 -- start start 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effdc06feb0 con 0x561149bf1280 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7effdc071ab0 con 0x561149c00b80 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effe5057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effdc070580 con 0x561149beeb40 2024-09-12T03:21:43.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.082+0000 7effdbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.083 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:42 smithi019 ceph-mon[30953]: pgmap v417: 97 pgs: 97 active+clean; 249 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.083+0000 7effe0a1a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.083+0000 7effdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.083+0000 7effdbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c00230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47022/0 (socket says 172.21.15.19:47022) 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.083+0000 7effdbfff640 1 -- 172.21.15.19:0/2421253265 learned_addr learned my addr 172.21.15.19:0/2421253265 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.084+0000 7effdbfff640 1 -- 172.21.15.19:0/2421253265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 msgr2=0x561149c00230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.084+0000 7effdbfff640 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c00230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.084+0000 7effdbfff640 1 -- 172.21.15.19:0/2421253265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 msgr2=0x561149c208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.084+0000 7effdbfff640 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.084+0000 7effdbfff640 1 -- 172.21.15.19:0/2421253265 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561149bee760 con 0x561149bf1280 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.085+0000 7effdbfff640 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c20e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7effc8040040 tx=0x7effc8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb70d6ed6cc1e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.085+0000 7effc67fc640 1 -- 172.21.15.19:0/2421253265 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8017030 con 0x561149bf1280 2024-09-12T03:21:43.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.085+0000 7effc67fc640 1 -- 172.21.15.19:0/2421253265 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8010930 con 0x561149bf1280 2024-09-12T03:21:43.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.086+0000 7effe5057740 1 -- 172.21.15.19:0/2421253265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 msgr2=0x561149c20e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.086+0000 7effe5057740 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c20e30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7effc8040040 tx=0x7effc8001e70 comp rx=0 tx=0).stop 2024-09-12T03:21:43.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.087+0000 7effe5057740 1 -- 172.21.15.19:0/2421253265 shutdown_connections 2024-09-12T03:21:43.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.087+0000 7effe5057740 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c20e30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.087+0000 7effe5057740 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.087+0000 7effe5057740 1 --2- 172.21.15.19:0/2421253265 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c00230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.087+0000 7effe5057740 1 -- 172.21.15.19:0/2421253265 >> 172.21.15.19:0/2421253265 conn(0x561149bb6310 msgr2=0x561149bf49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.091+0000 7effe5057740 1 -- 172.21.15.19:0/2421253265 shutdown_connections 2024-09-12T03:21:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.091+0000 7effe5057740 1 -- 172.21.15.19:0/2421253265 wait complete. 2024-09-12T03:21:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.092+0000 7effe5057740 1 -- start start 2024-09-12T03:21:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.092+0000 7effe5057740 1 Processor -- start 2024-09-12T03:21:43.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c2cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c284e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7effdc071320 con 0x561149bf1280 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7effdc070c50 con 0x561149c00b80 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7effdc06deb0 con 0x561149beeb40 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.093+0000 7effe5057740 10 client.0 _ll_register_callbacks cb 0x7effd0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.094+0000 7effe0a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.094+0000 7effe0a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42938/0 (socket says 172.21.15.19:42938) 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.094+0000 7effe0a1a640 1 -- 172.21.15.19:0/2877357979 learned_addr learned my addr 172.21.15.19:0/2877357979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effdbfff640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c284e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 -- 172.21.15.19:0/2877357979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 msgr2=0x561149c2cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c2cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 -- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 msgr2=0x561149c284e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c284e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 -- 172.21.15.19:0/2877357979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561149c28d10 con 0x561149bf1280 2024-09-12T03:21:43.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effdbfff640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:21:43.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.095+0000 7effe0a1a640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x561149bf6f40 tx=0x7effd0004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f38dc407f3a04a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:43.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.096+0000 7effbbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.096+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effd0003ca0 con 0x561149bf1280 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.096+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc8010930 con 0x561149bf1280 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.096+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7effd001d420 con 0x561149bf1280 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7effd0003e40 con 0x561149bf1280 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 10 client.15915 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561149c836d0 con 0x561149bf1280 2024-09-12T03:21:43.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8010930 con 0x561149bf1280 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7effd0025060 con 0x561149bf1280 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 10 client.15915 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 10 client.15915 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 20 client.15915 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c83de0 0x561149c86200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 4 client.15915 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:43.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.097+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561149bfaeb0 con 0x561149c83de0 2024-09-12T03:21:43.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.102+0000 7effdbfff640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c83de0 0x561149c86200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.118+0000 7effdbfff640 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c83de0 0x561149c86200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64bb4d83f6d0a3ba server_cookie=1436c876e06b2dc9 in_seq=0 out_seq=0 2024-09-12T03:21:43.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.120+0000 7effbbfff640 10 client.15915 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:21:43.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.123+0000 7effbbfff640 1 -- 172.21.15.19:0/2877357979 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561149bfaeb0 con 0x561149c83de0 2024-09-12T03:21:43.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.123+0000 7effbbfff640 10 client.15915 handle_command_reply: tid=0 2024-09-12T03:21:43.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.123+0000 7effe5057740 1 client.15915 shutdown 2024-09-12T03:21:43.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.130+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 msgr2=0x561149c27fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.130+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x561149bf6f40 tx=0x7effd0004410 comp rx=0 tx=0).stop 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 shutdown_connections 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x561149c83de0 0x561149c86200 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561149c00b80 0x561149c284e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561149bf1280 0x561149c27fa0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 --2- 172.21.15.19:0/2877357979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561149beeb40 0x561149c2cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 >> 172.21.15.19:0/2877357979 conn(0x561149bb6310 msgr2=0x561149be84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.134+0000 7effbbfff640 0 client.15915 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:21:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.139+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 shutdown_connections 2024-09-12T03:21:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.139+0000 7effe5057740 1 -- 172.21.15.19:0/2877357979 wait complete. 2024-09-12T03:21:43.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.139+0000 7effe5057740 20 client.15915 trim_cache size 0 max 16384 2024-09-12T03:21:43.140 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.140+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 msgr2=0x7effcc0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.140+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 0x7effcc0a5140 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7effd40c4bb0 tx=0x7effd4031fa0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.140+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 msgr2=0x7effdc19ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.140+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7effd4009dc0 tx=0x7effd4003ce0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 shutdown_connections 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7effcc0a2c80 0x7effcc0a5140 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7effdc10c1b0 0x7effdc19ef00 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7effdc103730 0x7effdc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 --2- 172.21.15.19:0/224381166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7effdc102e10 0x7effdc19a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:43.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 >> 172.21.15.19:0/224381166 conn(0x7effdc0fc7d0 msgr2=0x7effdc0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 shutdown_connections 2024-09-12T03:21:43.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:43.141+0000 7effe2c7e640 1 -- 172.21.15.19:0/224381166 wait complete. 2024-09-12T03:21:43.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:42 smithi121 ceph-mon[34963]: pgmap v417: 97 pgs: 97 active+clean; 249 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-12T03:21:43.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:42 smithi135 ceph-mon[35474]: pgmap v417: 97 pgs: 97 active+clean; 249 MiB data, 650 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.1 MiB/s wr, 144 op/s 2024-09-12T03:21:43.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:21:44.248 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:44.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 -- 172.21.15.19:0/2221498132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580100a80 msgr2=0x7f0580100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 --2- 172.21.15.19:0/2221498132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580100a80 0x7f0580100ee0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f0574009930 tx=0x7f057402a420 comp rx=0 tx=0).stop 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 -- 172.21.15.19:0/2221498132 shutdown_connections 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 --2- 172.21.15.19:0/2221498132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f058010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 --2- 172.21.15.19:0/2221498132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580100a80 0x7f0580100ee0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 --2- 172.21.15.19:0/2221498132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580106a80 0x7f0580106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 -- 172.21.15.19:0/2221498132 >> 172.21.15.19:0/2221498132 conn(0x7f05800fc7a0 msgr2=0x7f05800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.761+0000 7f0586be6640 1 -- 172.21.15.19:0/2221498132 shutdown_connections 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 -- 172.21.15.19:0/2221498132 wait complete. 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 Processor -- start 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 -- start start 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580100a80 0x7f0580198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f0580198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0580110e80 con 0x7f0580106a80 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0580110d00 con 0x7f0580100a80 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.762+0000 7f0586be6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0580111000 con 0x7f0580101420 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f057ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f0580198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42964/0 (socket says 172.21.15.19:42964) 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 -- 172.21.15.19:0/589084435 learned_addr learned my addr 172.21.15.19:0/589084435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0584982640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580100a80 0x7f0580198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 -- 172.21.15.19:0/589084435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 msgr2=0x7f0580198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f0580198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 -- 172.21.15.19:0/589084435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580100a80 msgr2=0x7f0580198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580100a80 0x7f0580198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 -- 172.21.15.19:0/589084435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058019d3e0 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f057ffff640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f0580198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0585183640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f057800eb10 tx=0x7f057800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f057dffb640 1 -- 172.21.15.19:0/589084435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578004b20 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f057dffb640 1 -- 172.21.15.19:0/589084435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0578004cc0 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058019d6d0 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.763+0000 7f057dffb640 1 -- 172.21.15.19:0/589084435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578010760 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.764+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05801a4fb0 con 0x7f0580106a80 2024-09-12T03:21:44.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f057dffb640 1 -- 172.21.15.19:0/589084435 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0578005020 con 0x7f0580106a80 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 Processor -- start 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 -- start start 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e959090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 0x56327e959a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e968780 0x56327e96ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56327e8bf2f0 con 0x56327e968780 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56327e866cf0 con 0x56327e95ab40 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f0588fbf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56327e8bb2b0 con 0x56327e9595d0 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f057dffb640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.766+0000 7f057ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e968780 0x56327e96ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0584982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 0x56327e959a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e959090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0585183640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0584982640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 0x56327e959a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47060/0 (socket says 172.21.15.19:47060) 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0584982640 1 -- 172.21.15.19:0/2851561133 learned_addr learned my addr 172.21.15.19:0/2851561133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057dffb640 1 -- 172.21.15.19:0/589084435 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f05780cf650 con 0x7f0580106a80 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 -- 172.21.15.19:0/2851561133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 msgr2=0x56327e959a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 0x56327e959a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 -- 172.21.15.19:0/2851561133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e968780 msgr2=0x56327e96ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e968780 0x56327e96ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 -- 172.21.15.19:0/2851561133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327e84b3a0 con 0x56327e95ab40 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0585183640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 0x7f05680a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f057809be50 tx=0x7f0578066b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f057ffff640 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e959090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0574005940 tx=0x7f05740058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de939c1adf90659a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0566ffd640 1 -- 172.21.15.19:0/2851561133 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574035050 con 0x56327e95ab40 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0566ffd640 1 -- 172.21.15.19:0/2851561133 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574005500 con 0x56327e95ab40 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.767+0000 7f0566ffd640 1 -- 172.21.15.19:0/2851561133 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574004b20 con 0x56327e95ab40 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- 172.21.15.19:0/2851561133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 msgr2=0x56327e959090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e959090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0574005940 tx=0x7f05740058d0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- 172.21.15.19:0/2851561133 shutdown_connections 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e968780 0x56327e96ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e9595d0 0x56327e959a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2851561133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e959090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- 172.21.15.19:0/2851561133 >> 172.21.15.19:0/2851561133 conn(0x56327e922310 msgr2=0x56327e955c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- 172.21.15.19:0/2851561133 shutdown_connections 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- 172.21.15.19:0/2851561133 wait complete. 2024-09-12T03:21:44.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.768+0000 7f0588fbf740 1 -- start start 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 Processor -- start 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e9595d0 0x56327e9681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e965a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56327e96b420 con 0x56327e95ab40 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56327e87a060 con 0x56327e9595d0 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56327e96b5a0 con 0x56327e968780 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0588fbf740 10 client.0 _ll_register_callbacks cb 0x56327e5cd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42978/0 (socket says 172.21.15.19:42978) 2024-09-12T03:21:44.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 -- 172.21.15.19:0/173569275 learned_addr learned my addr 172.21.15.19:0/173569275 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f057ffff640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e965a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f057ffff640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e9595d0 0x56327e9681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 -- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 msgr2=0x56327e965a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e965a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 -- 172.21.15.19:0/173569275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e9595d0 msgr2=0x56327e9681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.769+0000 7f0584982640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e9595d0 0x56327e9681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f0584982640 1 -- 172.21.15.19:0/173569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327e6303c0 con 0x56327e95ab40 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f0584982640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0570012720 tx=0x7f0570012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd4fa063184ba54 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f055ffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0570004050 con 0x56327e95ab40 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f05700041f0 con 0x56327e95ab40 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574005500 con 0x56327e95ab40 2024-09-12T03:21:44.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05700043f0 con 0x56327e95ab40 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f0588fbf740 10 client.15927 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56327e976f50 con 0x56327e95ab40 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.770+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574005500 con 0x56327e95ab40 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0570003bf0 con 0x56327e95ab40 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 10 client.15927 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 10 client.15927 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 20 client.15927 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e978410 0x56327e97a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 4 client.15927 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56327e8cc860 con 0x56327e978410 2024-09-12T03:21:44.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f057ffff640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e978410 0x56327e97a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f057ffff640 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e978410 0x56327e97a820 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afc16e7c497666f6 server_cookie=f885b373c17e0c97 in_seq=0 out_seq=0 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.771+0000 7f055ffff640 10 client.15927 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f055ffff640 1 -- 172.21.15.19:0/173569275 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56327e8cc860 con 0x56327e978410 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f055ffff640 10 client.15927 handle_command_reply: tid=0 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 client.15927 shutdown 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 msgr2=0x56327e965450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0570012720 tx=0x7f0570012be0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 shutdown_connections 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e965a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e95ab40 0x56327e965450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e978410 0x56327e97a820 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f055ffff640 0 client.15927 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 --2- 172.21.15.19:0/173569275 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e9595d0 0x56327e9681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 >> 172.21.15.19:0/173569275 conn(0x56327e922310 msgr2=0x56327e95b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 shutdown_connections 2024-09-12T03:21:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.772+0000 7f0588fbf740 1 -- 172.21.15.19:0/173569275 wait complete. 2024-09-12T03:21:44.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.773+0000 7f0588fbf740 20 client.15927 trim_cache size 0 max 16384 2024-09-12T03:21:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.786+0000 7f0588fbf740 1 Processor -- start 2024-09-12T03:21:44.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.786+0000 7f0588fbf740 1 -- start start 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e95ab40 0x56327e9746c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e968780 0x56327e964d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05801ab3a0 con 0x56327e9595d0 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05801acfa0 con 0x56327e968780 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0588fbf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05801aba70 con 0x56327e95ab40 2024-09-12T03:21:44.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42982/0 (socket says 172.21.15.19:42982) 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 -- 172.21.15.19:0/818317584 learned_addr learned my addr 172.21.15.19:0/818317584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f0584982640 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e95ab40 0x56327e9746c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e968780 0x56327e964d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 -- 172.21.15.19:0/818317584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e95ab40 msgr2=0x56327e9746c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e95ab40 0x56327e9746c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 -- 172.21.15.19:0/818317584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e968780 msgr2=0x56327e964d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e968780 0x56327e964d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 -- 172.21.15.19:0/818317584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327e968510 con 0x56327e9595d0 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f057ffff640 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0574004650 tx=0x7f0574000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b67b02c9b5807f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f05667fc640 1 -- 172.21.15.19:0/818317584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0574035070 con 0x56327e9595d0 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.787+0000 7f05667fc640 1 -- 172.21.15.19:0/818317584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574031c90 con 0x56327e9595d0 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f05667fc640 1 -- 172.21.15.19:0/818317584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057403d450 con 0x56327e9595d0 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- 172.21.15.19:0/818317584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 msgr2=0x56327e955e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f0574004650 tx=0x7f0574000bc0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- 172.21.15.19:0/818317584 shutdown_connections 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e968780 0x56327e964d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e95ab40 0x56327e9746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 --2- 172.21.15.19:0/818317584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e955e10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- 172.21.15.19:0/818317584 >> 172.21.15.19:0/818317584 conn(0x56327e922310 msgr2=0x56327e955700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- 172.21.15.19:0/818317584 shutdown_connections 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- 172.21.15.19:0/818317584 wait complete. 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.788+0000 7f0588fbf740 1 -- start start 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 Processor -- start 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e962940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e962e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f05801ac810 con 0x56327e9595d0 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05801ac140 con 0x56327e95ab40 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f05801a93a0 con 0x56327e968780 2024-09-12T03:21:44.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0588fbf740 10 client.0 _ll_register_callbacks cb 0x7f0570009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f057ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f0584982640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e962940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f057ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42998/0 (socket says 172.21.15.19:42998) 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f057ffff640 1 -- 172.21.15.19:0/2491355807 learned_addr learned my addr 172.21.15.19:0/2491355807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.789+0000 7f057ffff640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e962e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 -- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 msgr2=0x56327e962e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e962e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 -- 172.21.15.19:0/2491355807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 msgr2=0x56327e962400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:44.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 -- 172.21.15.19:0/2491355807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327e9e2100 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f057ffff640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0584982640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e962940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05700126f0 tx=0x7f0570012430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0ff16ea12790ff9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f055ffff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057000fbf0 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f057000fd90 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0574031c90 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0570005040 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0588fbf740 10 client.25451 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.790+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56327e9e3390 con 0x56327e95ab40 2024-09-12T03:21:44.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0574031c90 con 0x56327e95ab40 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f057002f030 con 0x56327e95ab40 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 10 client.25451 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 20 client.25451 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e9e3ab0 0x56327e9e5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 4 client.25451 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:21:44.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56327e8c6d70 con 0x56327e9e3ab0 2024-09-12T03:21:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f057ffff640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e9e3ab0 0x56327e9e5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f055ffff640 10 client.25451 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:44.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.791+0000 7f057ffff640 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e9e3ab0 0x56327e9e5ed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1399fbd9772968 server_cookie=158e2958fa40d67d in_seq=0 out_seq=0 2024-09-12T03:21:44.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f055ffff640 1 -- 172.21.15.19:0/2491355807 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56327e8c6d70 con 0x56327e9e3ab0 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f055ffff640 10 client.25451 handle_command_reply: tid=0 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 client.25451 shutdown 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 msgr2=0x56327e962940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e962940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f05700126f0 tx=0x7f0570012430 comp rx=0 tx=0).stop 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 shutdown_connections 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56327e968780 0x56327e962e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56327e9e3ab0 0x56327e9e5ed0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56327e95ab40 0x56327e962940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 --2- 172.21.15.19:0/2491355807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56327e9595d0 0x56327e962400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f055ffff640 0 client.25451 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:44.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 >> 172.21.15.19:0/2491355807 conn(0x56327e922310 msgr2=0x56327e966cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 shutdown_connections 2024-09-12T03:21:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 1 -- 172.21.15.19:0/2491355807 wait complete. 2024-09-12T03:21:44.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.793+0000 7f0588fbf740 20 client.25451 trim_cache size 0 max 16384 2024-09-12T03:21:44.803 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:21:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 msgr2=0x7f05680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 0x7f05680a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f057809be50 tx=0x7f0578066b40 comp rx=0 tx=0).stop 2024-09-12T03:21:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 msgr2=0x7f058019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:44.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f057800eb10 tx=0x7f057800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 shutdown_connections 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f05680a2c00 0x7f05680a50c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0580106a80 0x7f058019cd00 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0580101420 0x7f0580198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 --2- 172.21.15.19:0/589084435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0580100a80 0x7f0580198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 >> 172.21.15.19:0/589084435 conn(0x7f05800fc7a0 msgr2=0x7f0580104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:44.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 shutdown_connections 2024-09-12T03:21:44.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:44.794+0000 7f0586be6640 1 -- 172.21.15.19:0/589084435 wait complete. 2024-09-12T03:21:45.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:44 smithi121 ceph-mon[34963]: pgmap v418: 97 pgs: 97 active+clean; 253 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-12T03:21:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:44 smithi019 ceph-mon[30953]: pgmap v418: 97 pgs: 97 active+clean; 253 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-12T03:21:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:44 smithi135 ceph-mon[35474]: pgmap v418: 97 pgs: 97 active+clean; 253 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-09-12T03:21:45.695 INFO:tasks.workunit.client.0.smithi019.stdout: 84 813685 833 555437 776 182905 164 2024-09-12T03:21:46.631 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:21:46.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:21:46.913 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:46.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:46 smithi019 ceph-mon[30953]: pgmap v419: 97 pgs: 97 active+clean; 276 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-09-12T03:21:47.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:46 smithi121 ceph-mon[34963]: pgmap v419: 97 pgs: 97 active+clean; 276 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-09-12T03:21:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:46 smithi135 ceph-mon[35474]: pgmap v419: 97 pgs: 97 active+clean; 276 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 195 op/s 2024-09-12T03:21:47.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.478+0000 7f91f540b640 1 -- 172.21.15.19:0/3290789822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0101470 msgr2=0x7f91f0101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.478+0000 7f91f540b640 1 --2- 172.21.15.19:0/3290789822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0101470 0x7f91f0101850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91e0009900 tx=0x7f91e002a410 comp rx=0 tx=0).stop 2024-09-12T03:21:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.483+0000 7f91f540b640 1 -- 172.21.15.19:0/3290789822 shutdown_connections 2024-09-12T03:21:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.483+0000 7f91f540b640 1 --2- 172.21.15.19:0/3290789822 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0105de0 0x7f91f010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.483+0000 7f91f540b640 1 --2- 172.21.15.19:0/3290789822 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101e20 0x7f91f01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.483+0000 7f91f540b640 1 --2- 172.21.15.19:0/3290789822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0101470 0x7f91f0101850 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.483+0000 7f91f540b640 1 -- 172.21.15.19:0/3290789822 >> 172.21.15.19:0/3290789822 conn(0x7f91f0077230 msgr2=0x7f91f00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.484+0000 7f91f540b640 1 -- 172.21.15.19:0/3290789822 shutdown_connections 2024-09-12T03:21:47.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.485+0000 7f91f540b640 1 -- 172.21.15.19:0/3290789822 wait complete. 2024-09-12T03:21:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.486+0000 7f91f540b640 1 Processor -- start 2024-09-12T03:21:47.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.486+0000 7f91f540b640 1 -- start start 2024-09-12T03:21:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 0x7f91f0116400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 0x7f91f0110480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 0x7f91f01109c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 0x7f91f0116400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 0x7f91f0110480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 0x7f91f0116400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47092/0 (socket says 172.21.15.19:47092) 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 0x7f91f0110480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43018/0 (socket says 172.21.15.19:43018) 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010eec0 con 0x7f91f0101e20 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91eeffd640 1 -- 172.21.15.19:0/3519212216 learned_addr learned my addr 172.21.15.19:0/3519212216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010ed40 con 0x7f91f0105de0 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91f010f040 con 0x7f91f0101470 2024-09-12T03:21:47.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.487+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 0x7f91f01109c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 -- 172.21.15.19:0/3519212216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 msgr2=0x7f91f0116400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 0x7f91f0116400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 -- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 msgr2=0x7f91f0110480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 0x7f91f0110480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f01110f0 con 0x7f91f0105de0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 0x7f91f01109c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009740 tx=0x7f91e8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ecff9640 1 -- 172.21.15.19:0/3519212216 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8009e50 con 0x7f91f0105de0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ecff9640 1 -- 172.21.15.19:0/3519212216 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e8005020 con 0x7f91f0105de0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91ecff9640 1 -- 172.21.15.19:0/3519212216 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e8004c60 con 0x7f91f0105de0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.488+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91f01b38d0 con 0x7f91f0105de0 2024-09-12T03:21:47.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.489+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91f01b3c70 con 0x7f91f0105de0 2024-09-12T03:21:47.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.491+0000 7f91ecff9640 1 -- 172.21.15.19:0/3519212216 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f91e801d020 con 0x7f91f0105de0 2024-09-12T03:21:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.492+0000 7f91ecff9640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 0x7f91dc0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.492+0000 7f91eeffd640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 0x7f91dc0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.492+0000 7f91f77e4740 1 Processor -- start 2024-09-12T03:21:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.492+0000 7f91ecff9640 1 -- 172.21.15.19:0/3519212216 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91e80ce280 con 0x7f91f0105de0 2024-09-12T03:21:47.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.492+0000 7f91f77e4740 1 -- start start 2024-09-12T03:21:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571cdb60 0x562a571cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571db130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a57133280 con 0x562a571d8d40 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562a570dab10 con 0x562a571ce480 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91f77e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a5712f240 con 0x562a571cdb60 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47706/0 (socket says 172.21.15.19:47706) 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 -- 172.21.15.19:0/1550822109 learned_addr learned my addr 172.21.15.19:0/1550822109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91eeffd640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 0x7f91dc0a52b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e002a910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571cdb60 0x562a571cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 -- 172.21.15.19:0/1550822109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571cdb60 msgr2=0x562a571cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571cdb60 0x562a571cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 -- 172.21.15.19:0/1550822109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 msgr2=0x562a571db130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571db130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.493+0000 7f91e67fc640 1 -- 172.21.15.19:0/1550822109 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a570e71f0 con 0x562a571ce480 2024-09-12T03:21:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.494+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91d80103f0 tx=0x7f91d8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b0a21245fe59c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.494+0000 7f91e57fa640 1 -- 172.21.15.19:0/1550822109 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8016070 con 0x562a571ce480 2024-09-12T03:21:47.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.494+0000 7f91e57fa640 1 -- 172.21.15.19:0/1550822109 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8004d20 con 0x562a571ce480 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.494+0000 7f91e57fa640 1 -- 172.21.15.19:0/1550822109 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d8005560 con 0x562a571ce480 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- 172.21.15.19:0/1550822109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 msgr2=0x562a571d8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f91d80103f0 tx=0x7f91d8009f90 comp rx=0 tx=0).stop 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- 172.21.15.19:0/1550822109 shutdown_connections 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571db130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571ce480 0x562a571d8800 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1550822109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571cdb60 0x562a571cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- 172.21.15.19:0/1550822109 >> 172.21.15.19:0/1550822109 conn(0x562a571ca0f0 msgr2=0x562a571ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- 172.21.15.19:0/1550822109 shutdown_connections 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- 172.21.15.19:0/1550822109 wait complete. 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 -- start start 2024-09-12T03:21:47.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.495+0000 7f91f77e4740 1 Processor -- start 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43028/0 (socket says 172.21.15.19:43028) 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 -- 172.21.15.19:0/1727637723 learned_addr learned my addr 172.21.15.19:0/1727637723 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562a571db910 con 0x562a571cdb60 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562a570edf10 con 0x562a571d8d40 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562a571dba90 con 0x562a571ce480 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91f77e4740 10 client.0 _ll_register_callbacks cb 0x562a570b6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:47.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 -- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 msgr2=0x562a571d6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 -- 172.21.15.19:0/1727637723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 msgr2=0x562a571d67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91eeffd640 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a56ea46a0 con 0x562a571cdb60 2024-09-12T03:21:47.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.496+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f91e00052f0 tx=0x7f91e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e0b05df5368526 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0035040 con 0x562a571cdb60 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91e0004960 con 0x562a571cdb60 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d8004d20 con 0x562a571cdb60 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91e0004b60 con 0x562a571cdb60 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91f77e4740 10 client.15942 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.497+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a571e7620 con 0x562a571cdb60 2024-09-12T03:21:47.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91d8004d20 con 0x562a571cdb60 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f91e0032650 con 0x562a571cdb60 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 10 client.15942 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 10 client.15942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 20 client.15942 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a571e8f60 0x562a571eb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 4 client.15942 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a571407f0 con 0x562a571e8f60 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.498+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a571e8f60 0x562a571eb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.499+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a571e8f60 0x562a571eb3e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662e6f0f4c45cde server_cookie=f88d3c339ea5d41f in_seq=0 out_seq=0 2024-09-12T03:21:47.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.499+0000 7f91d67fc640 10 client.15942 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.503+0000 7f91d67fc640 1 -- 172.21.15.19:0/1727637723 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a571407f0 con 0x562a571e8f60 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.503+0000 7f91d67fc640 10 client.15942 handle_command_reply: tid=0 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.503+0000 7f91f77e4740 1 client.15942 shutdown 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 msgr2=0x562a571d5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f91e00052f0 tx=0x7f91e0009260 comp rx=0 tx=0).stop 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 shutdown_connections 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a571e8f60 0x562a571eb3e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1727637723 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571d5d30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 >> 172.21.15.19:0/1727637723 conn(0x562a571ca0f0 msgr2=0x562a571cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91d67fc640 0 client.15942 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 shutdown_connections 2024-09-12T03:21:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.504+0000 7f91f77e4740 1 -- 172.21.15.19:0/1727637723 wait complete. 2024-09-12T03:21:47.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.505+0000 7f91f77e4740 20 client.15942 trim_cache size 0 max 16384 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.530+0000 7f91f77e4740 1 Processor -- start 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.530+0000 7f91f77e4740 1 -- start start 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01bc310 con 0x562a571d8d40 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01bdf10 con 0x562a571cdb60 2024-09-12T03:21:47.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91f77e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01bc9e0 con 0x562a571ce480 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47730/0 (socket says 172.21.15.19:47730) 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 -- 172.21.15.19:0/3892309664 learned_addr learned my addr 172.21.15.19:0/3892309664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91e67fc640 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 -- 172.21.15.19:0/3892309664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 msgr2=0x562a571cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 -- 172.21.15.19:0/3892309664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 msgr2=0x562a571f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.531+0000 7f91eeffd640 1 -- 172.21.15.19:0/3892309664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a571d8310 con 0x562a571cdb60 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.532+0000 7f91eeffd640 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91e00329f0 tx=0x7f91e0004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c4ecdbbc21c3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.532+0000 7f91e4ff9640 1 -- 172.21.15.19:0/3892309664 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0035040 con 0x562a571cdb60 2024-09-12T03:21:47.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.532+0000 7f91e4ff9640 1 -- 172.21.15.19:0/3892309664 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e00325c0 con 0x562a571cdb60 2024-09-12T03:21:47.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.532+0000 7f91e4ff9640 1 -- 172.21.15.19:0/3892309664 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003ad0 con 0x562a571cdb60 2024-09-12T03:21:47.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- 172.21.15.19:0/3892309664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 msgr2=0x562a571d5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f91e00329f0 tx=0x7f91e0004f90 comp rx=0 tx=0).stop 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- 172.21.15.19:0/3892309664 shutdown_connections 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571d8d40 0x562a571f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 --2- 172.21.15.19:0/3892309664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571cdb60 0x562a571d5650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- 172.21.15.19:0/3892309664 >> 172.21.15.19:0/3892309664 conn(0x562a571ca0f0 msgr2=0x562a571cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- 172.21.15.19:0/3892309664 shutdown_connections 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- 172.21.15.19:0/3892309664 wait complete. 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 -- start start 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 Processor -- start 2024-09-12T03:21:47.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.533+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01bd780 con 0x562a571cdb60 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01bd0b0 con 0x562a571d8d40 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91f01ba2f0 con 0x562a571ce480 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91f77e4740 10 client.0 _ll_register_callbacks cb 0x7f91e0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91ef7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91e67fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43050/0 (socket says 172.21.15.19:43050) 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 -- 172.21.15.19:0/1417073715 learned_addr learned my addr 172.21.15.19:0/1417073715 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 -- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 msgr2=0x562a571d2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 -- 172.21.15.19:0/1417073715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 msgr2=0x562a571d31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 -- 172.21.15.19:0/1417073715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a571d3a00 con 0x562a571cdb60 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91ef7fe640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91eeffd640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f91e0067040 tx=0x7f91e0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee61cb50f385009 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.534+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e0074060 con 0x562a571cdb60 2024-09-12T03:21:47.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91e00035f0 con 0x562a571cdb60 2024-09-12T03:21:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91e00325c0 con 0x562a571cdb60 2024-09-12T03:21:47.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f91e0035040 con 0x562a571cdb60 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 10 client.15948 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562a57253df0 con 0x562a571cdb60 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91e00325c0 con 0x562a571cdb60 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f91e003c4d0 con 0x562a571cdb60 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 10 client.15948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 20 client.15948 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a572544e0 0x562a57256900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 4 client.15948 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a5713ad00 con 0x562a572544e0 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.535+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a572544e0 0x562a57256900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:47.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.536+0000 7f91e67fc640 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a572544e0 0x562a57256900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30dbdcfba1a3489d server_cookie=6d17c57632ec2e41 in_seq=0 out_seq=0 2024-09-12T03:21:47.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.536+0000 7f91d67fc640 10 client.15948 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91d67fc640 1 -- 172.21.15.19:0/1417073715 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a5713ad00 con 0x562a572544e0 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91d67fc640 10 client.15948 handle_command_reply: tid=0 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 client.15948 shutdown 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 msgr2=0x562a571e1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f91e0067040 tx=0x7f91e0005a20 comp rx=0 tx=0).stop 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 shutdown_connections 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562a572544e0 0x562a57256900 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91d67fc640 0 client.15948 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562a571d8d40 0x562a571d31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562a571ce480 0x562a571d2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 --2- 172.21.15.19:0/1417073715 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562a571cdb60 0x562a571e1560 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 >> 172.21.15.19:0/1417073715 conn(0x562a571ca0f0 msgr2=0x562a571c9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 shutdown_connections 2024-09-12T03:21:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.570+0000 7f91f77e4740 1 -- 172.21.15.19:0/1417073715 wait complete. 2024-09-12T03:21:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.571+0000 7f91f77e4740 20 client.15948 trim_cache size 0 max 16384 2024-09-12T03:21:47.572 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:47.572 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:21:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ff4c3bdd-4b7a-4f9b-8204-9ca968f2e371", 2024-09-12T03:21:47.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:21:47.573 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 msgr2=0x7f91dc0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 0x7f91dc0a52b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f91e0000c00 tx=0x7f91e002a910 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 msgr2=0x7f91f01109c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 0x7f91f01109c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f91e8009740 tx=0x7f91e8009c00 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 shutdown_connections 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91f0105de0 0x7f91f01109c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f91dc0a2df0 0x7f91dc0a52b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f91f0101e20 0x7f91f0110480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 --2- 172.21.15.19:0/3519212216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f91f0101470 0x7f91f0116400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.572+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 >> 172.21.15.19:0/3519212216 conn(0x7f91f0077230 msgr2=0x7f91f0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.573+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 shutdown_connections 2024-09-12T03:21:47.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:47.573+0000 7f91f540b640 1 -- 172.21.15.19:0/3519212216 wait complete. 2024-09-12T03:21:48.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:21:48.824 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:49.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: pgmap v420: 97 pgs: 97 active+clean; 315 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 329 op/s 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.167 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.168 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.169 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:49.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.171 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:48 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: pgmap v420: 97 pgs: 97 active+clean; 315 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 329 op/s 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:49.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:48 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 -- 172.21.15.19:0/1159369292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8106f10 msgr2=0x7feeb810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 --2- 172.21.15.19:0/1159369292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8106f10 0x7feeb810ea90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009ca0 tx=0x7feeb002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 -- 172.21.15.19:0/1159369292 shutdown_connections 2024-09-12T03:21:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 --2- 172.21.15.19:0/1159369292 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8106f10 0x7feeb810ea90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 --2- 172.21.15.19:0/1159369292 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8102790 0x7feeb81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 --2- 172.21.15.19:0/1159369292 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 -- 172.21.15.19:0/1159369292 >> 172.21.15.19:0/1159369292 conn(0x7feeb80fd8c0 msgr2=0x7feeb80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 -- 172.21.15.19:0/1159369292 shutdown_connections 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.347+0000 7feebeb71640 1 -- 172.21.15.19:0/1159369292 wait complete. 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.348+0000 7feebeb71640 1 Processor -- start 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.348+0000 7feebeb71640 1 -- start start 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.348+0000 7feebeb71640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.348+0000 7feebeb71640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8102790 0x7feeb8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.348+0000 7feebeb71640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8106f10 0x7feeb819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebeb71640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8111fc0 con 0x7feeb8102790 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebeb71640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8111e40 con 0x7feeb8101de0 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebeb71640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8112140 con 0x7feeb8106f10 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8102790 0x7feeb8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47752/0 (socket says 172.21.15.19:47752) 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebd10e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8106f10 0x7feeb819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 -- 172.21.15.19:0/2964818546 learned_addr learned my addr 172.21.15.19:0/2964818546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 -- 172.21.15.19:0/2964818546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8106f10 msgr2=0x7feeb819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8106f10 0x7feeb819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 -- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8102790 msgr2=0x7feeb8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8102790 0x7feeb8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 -- 172.21.15.19:0/2964818546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb819d390 con 0x7feeb8101de0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feebc90d640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9f0 tx=0x7feeac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.349+0000 7feeb5ffb640 1 -- 172.21.15.19:0/2964818546 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac00cd80 con 0x7feeb8101de0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.350+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb819d680 con 0x7feeb8101de0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.350+0000 7feeb5ffb640 1 -- 172.21.15.19:0/2964818546 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeac010910 con 0x7feeb8101de0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.350+0000 7feeb5ffb640 1 -- 172.21.15.19:0/2964818546 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac0188f0 con 0x7feeb8101de0 2024-09-12T03:21:49.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.350+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a4f60 con 0x7feeb8101de0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.353+0000 7feeb5ffb640 1 -- 172.21.15.19:0/2964818546 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feeac010ac0 con 0x7feeb8101de0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.353+0000 7feec0f4a740 1 Processor -- start 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.353+0000 7feec0f4a740 1 -- start start 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198c7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f31981e2e0 con 0x55f3198b8380 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f3197c5ce0 con 0x55f3198b8ca0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feec0f4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f31981a2a0 con 0x55f3198c7a90 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebc90d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47154/0 (socket says 172.21.15.19:47154) 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feeb7fff640 1 -- 172.21.15.19:0/3460823620 learned_addr learned my addr 172.21.15.19:0/3460823620 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43082/0 (socket says 172.21.15.19:43082) 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 -- 172.21.15.19:0/3460823620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 msgr2=0x55f3198c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 -- 172.21.15.19:0/3460823620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 msgr2=0x55f3198c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feebd10e640 1 -- 172.21.15.19:0/3460823620 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3197aa440 con 0x55f3198b8380 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feeb7fff640 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.354+0000 7feeb5ffb640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 0x7feea80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feebd10e640 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005e30 tx=0x7feeb00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1749b26af19aad78 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feebc90d640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 0x7feea80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feeb5ffb640 1 -- 172.21.15.19:0/2964818546 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feeac0cf3c0 con 0x7feeb8101de0 2024-09-12T03:21:49.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feea2ffd640 1 -- 172.21.15.19:0/3460823620 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0031db0 con 0x55f3198b8380 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feea2ffd640 1 -- 172.21.15.19:0/3460823620 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0036030 con 0x55f3198b8380 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.355+0000 7feea2ffd640 1 -- 172.21.15.19:0/3460823620 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00042a0 con 0x55f3198b8380 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feebc90d640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 0x7feea80a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feeac0cd240 tx=0x7feeac01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- 172.21.15.19:0/3460823620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 msgr2=0x55f3198b8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feeb0005e30 tx=0x7feeb00093e0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- 172.21.15.19:0/3460823620 shutdown_connections 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198c7a90 0x55f3198c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 --2- 172.21.15.19:0/3460823620 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198b8760 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- 172.21.15.19:0/3460823620 >> 172.21.15.19:0/3460823620 conn(0x55f3198b4a30 msgr2=0x55f3198b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- 172.21.15.19:0/3460823620 shutdown_connections 2024-09-12T03:21:49.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- 172.21.15.19:0/3460823620 wait complete. 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 -- start start 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.356+0000 7feec0f4a740 1 Processor -- start 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8ca0 0x55f3198c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f3198ca4e0 con 0x55f3198b8ca0 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f3197d9050 con 0x55f3198c7a90 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f3198ca660 con 0x55f3198b8380 2024-09-12T03:21:49.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feec0f4a740 10 client.0 _ll_register_callbacks cb 0x55f319819160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47158/0 (socket says 172.21.15.19:47158) 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 -- 172.21.15.19:0/2541472327 learned_addr learned my addr 172.21.15.19:0/2541472327 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feeb7fff640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8ca0 0x55f3198c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 -- 172.21.15.19:0/2541472327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 msgr2=0x55f3198c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 -- 172.21.15.19:0/2541472327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8ca0 msgr2=0x55f3198c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8ca0 0x55f3198c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.357+0000 7feebd10e640 1 -- 172.21.15.19:0/2541472327 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f31958f3c0 con 0x55f3198b8380 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7feebd10e640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feeb0031ac0 tx=0x7feeb0031920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75d168ec373b9e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7fee97fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:21:49.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0059060 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb0036030 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feeb0004040 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feeb0004b40 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7feec0f4a740 10 client.25510 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3198ca4e0 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.358+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb0036030 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7feeb0043070 con 0x55f3198b8380 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 10 client.25510 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 10 client.25510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 20 client.25510 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f3198d7110 0x55f3198d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 4 client.25510 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:49.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f31982b850 con 0x55f3198d7110 2024-09-12T03:21:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feeb7fff640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f3198d7110 0x55f3198d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7fee97fff640 10 client.25510 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:49.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.359+0000 7feeb7fff640 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f3198d7110 0x55f3198d9530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662f035f98559d0 server_cookie=ac6aeb148d362f79 in_seq=0 out_seq=0 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7fee97fff640 1 -- 172.21.15.19:0/2541472327 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f31982b850 con 0x55f3198d7110 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7fee97fff640 10 client.25510 handle_command_reply: tid=0 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 client.25510 shutdown 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 msgr2=0x55f3198c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feeb0031ac0 tx=0x7feeb0031920 comp rx=0 tx=0).stop 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 shutdown_connections 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f3198d7110 0x55f3198d9530 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8ca0 0x55f3198c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7fee97fff640 0 client.25510 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 --2- 172.21.15.19:0/2541472327 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198c7420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 >> 172.21.15.19:0/2541472327 conn(0x55f3198b4a30 msgr2=0x55f3198ba790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 shutdown_connections 2024-09-12T03:21:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 1 -- 172.21.15.19:0/2541472327 wait complete. 2024-09-12T03:21:49.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.363+0000 7feec0f4a740 20 client.25510 trim_cache size 0 max 16384 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: pgmap v420: 97 pgs: 97 active+clean; 315 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 329 op/s 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:48 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.387+0000 7feec0f4a740 1 Processor -- start 2024-09-12T03:21:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 -- start start 2024-09-12T03:21:49.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198d3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198c7a90 0x55f3198df320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ab300 con 0x55f3198c7a90 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81acf00 con 0x55f3198b8ca0 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.388+0000 7feec0f4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ab9d0 con 0x55f3198b8380 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.394+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.395+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47774/0 (socket says 172.21.15.19:47774) 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.395+0000 7feeb7fff640 1 -- 172.21.15.19:0/1902448272 learned_addr learned my addr 172.21.15.19:0/1902448272 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.396+0000 7feeb7fff640 1 -- 172.21.15.19:0/1902448272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 msgr2=0x55f3198d3780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.396+0000 7feeb7fff640 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198d3780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.396+0000 7feeb7fff640 1 -- 172.21.15.19:0/1902448272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198c7a90 msgr2=0x55f3198df320 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.396+0000 7feeb7fff640 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198c7a90 0x55f3198df320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.396+0000 7feeb7fff640 1 -- 172.21.15.19:0/1902448272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3198b5ba0 con 0x55f3198b8ca0 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.397+0000 7feeb7fff640 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f3198c3f60 tx=0x7feea4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6870fa7201067d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.398+0000 7feea27fc640 1 -- 172.21.15.19:0/1902448272 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4005220 con 0x55f3198b8ca0 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.398+0000 7feea27fc640 1 -- 172.21.15.19:0/1902448272 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea4004c80 con 0x55f3198b8ca0 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.398+0000 7feea27fc640 1 -- 172.21.15.19:0/1902448272 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea401e9e0 con 0x55f3198b8ca0 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.400+0000 7feec0f4a740 1 -- 172.21.15.19:0/1902448272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 msgr2=0x55f3198dede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.400+0000 7feec0f4a740 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55f3198c3f60 tx=0x7feea4005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.406+0000 7feec0f4a740 1 -- 172.21.15.19:0/1902448272 shutdown_connections 2024-09-12T03:21:49.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.406+0000 7feec0f4a740 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198c7a90 0x55f3198df320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.406+0000 7feec0f4a740 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198b8ca0 0x55f3198dede0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.406+0000 7feec0f4a740 1 --2- 172.21.15.19:0/1902448272 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8380 0x55f3198d3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.406+0000 7feec0f4a740 1 -- 172.21.15.19:0/1902448272 >> 172.21.15.19:0/1902448272 conn(0x55f3198b4a30 msgr2=0x55f3198bfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.408+0000 7feec0f4a740 1 -- 172.21.15.19:0/1902448272 shutdown_connections 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.409+0000 7feec0f4a740 1 -- 172.21.15.19:0/1902448272 wait complete. 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.409+0000 7feec0f4a740 1 -- start start 2024-09-12T03:21:49.415 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.409+0000 7feec0f4a740 1 Processor -- start 2024-09-12T03:21:49.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.411+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8ca0 0x55f3198bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ac770 con 0x55f3198b8380 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ac0a0 con 0x55f3198c7a90 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81a92e0 con 0x55f3198b8ca0 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feec0f4a740 10 client.0 _ll_register_callbacks cb 0x7feea4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47790/0 (socket says 172.21.15.19:47790) 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 -- 172.21.15.19:0/4177864373 learned_addr learned my addr 172.21.15.19:0/4177864373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 -- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8ca0 msgr2=0x55f3198bd160 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8ca0 0x55f3198bd160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.412+0000 7feebd10e640 1 -- 172.21.15.19:0/4177864373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 msgr2=0x55f3198bcc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.413+0000 7feebd10e640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198bcc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.413+0000 7feebd10e640 1 -- 172.21.15.19:0/4177864373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3198ceed0 con 0x55f3198c7a90 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.413+0000 7feebd10e640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feeb002fbf0 tx=0x7feeb002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2e62794fccb74c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.413+0000 7fee97fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.414+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0063060 con 0x55f3198c7a90 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.414+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea4004c80 con 0x55f3198c7a90 2024-09-12T03:21:49.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.414+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7feeb0003fc0 con 0x55f3198c7a90 2024-09-12T03:21:49.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.414+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feeb0059070 con 0x55f3198c7a90 2024-09-12T03:21:49.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.416+0000 7feec0f4a740 10 client.25487 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:49.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.416+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f3198d01c0 con 0x55f3198c7a90 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.417+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea4004c80 con 0x55f3198c7a90 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.417+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7feeb0043070 con 0x55f3198c7a90 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 10 client.25487 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 10 client.25487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 20 client.25487 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f31993dfc0 0x55f3199403e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:49.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 4 client.25487 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:21:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.419+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f319825d60 con 0x55f31993dfc0 2024-09-12T03:21:49.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.421+0000 7feeb7fff640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f31993dfc0 0x55f3199403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.422+0000 7feeb7fff640 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f31993dfc0 0x55f3199403e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d10fb56b7869c07 server_cookie=fd60d7c04c69b52f in_seq=0 out_seq=0 2024-09-12T03:21:49.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.422+0000 7fee97fff640 10 client.25487 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.427+0000 7fee97fff640 1 -- 172.21.15.19:0/4177864373 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55f319825d60 con 0x55f31993dfc0 2024-09-12T03:21:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.427+0000 7fee97fff640 10 client.25487 handle_command_reply: tid=0 2024-09-12T03:21:49.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.427+0000 7feec0f4a740 1 client.25487 shutdown 2024-09-12T03:21:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.429+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 msgr2=0x55f3198ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.429+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feeb002fbf0 tx=0x7feeb002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 shutdown_connections 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f31993dfc0 0x55f3199403e0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f3198c7a90 0x55f3198ce8e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f3198b8ca0 0x55f3198bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 --2- 172.21.15.19:0/4177864373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f3198b8380 0x55f3198bcc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 >> 172.21.15.19:0/4177864373 conn(0x55f3198b4a30 msgr2=0x55f3198b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.432+0000 7fee97fff640 0 client.25487 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:49.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.433+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 shutdown_connections 2024-09-12T03:21:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.434+0000 7feec0f4a740 1 -- 172.21.15.19:0/4177864373 wait complete. 2024-09-12T03:21:49.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.434+0000 7feec0f4a740 20 client.25487 trim_cache size 0 max 16384 2024-09-12T03:21:49.441 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (2945 inodes in the stack)", 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ff4c3bdd-4b7a-4f9b-8204-9ca968f2e371": { 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ff4c3bdd-4b7a-4f9b-8204-9ca968f2e371", 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:21:49.442 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 msgr2=0x7feea80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 0x7feea80a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7feeac0cd240 tx=0x7feeac01f000 comp rx=0 tx=0).stop 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 msgr2=0x7feeb81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feeac00e9f0 tx=0x7feeac00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 shutdown_connections 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7feeb8106f10 0x7feeb819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.442+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7feea80a2b80 0x7feea80a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.443+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7feeb8102790 0x7feeb8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.443+0000 7feebeb71640 1 --2- 172.21.15.19:0/2964818546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feeb8101de0 0x7feeb81983e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.443+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 >> 172.21.15.19:0/2964818546 conn(0x7feeb80fd8c0 msgr2=0x7feeb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.443+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 shutdown_connections 2024-09-12T03:21:49.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:49.443+0000 7feebeb71640 1 -- 172.21.15.19:0/2964818546 wait complete. 2024-09-12T03:21:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:21:50.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:21:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.330 INFO:tasks.cephfs.filesystem:scrub status for tag:ff4c3bdd-4b7a-4f9b-8204-9ca968f2e371 - {'path': '/', 'tag': 'ff4c3bdd-4b7a-4f9b-8204-9ca968f2e371', 'options': 'recursive,force'} 2024-09-12T03:21:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:21:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:51.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:50 smithi019 ceph-mon[30953]: pgmap v421: 97 pgs: 97 active+clean; 326 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.6 MiB/s wr, 364 op/s 2024-09-12T03:21:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:50 smithi135 ceph-mon[35474]: pgmap v421: 97 pgs: 97 active+clean; 326 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.6 MiB/s wr, 364 op/s 2024-09-12T03:21:51.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:50 smithi121 ceph-mon[34963]: pgmap v421: 97 pgs: 97 active+clean; 326 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.6 MiB/s wr, 364 op/s 2024-09-12T03:21:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:52 smithi019 ceph-mon[30953]: pgmap v422: 97 pgs: 97 active+clean; 331 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-12T03:21:53.201 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:52 smithi121 ceph-mon[34963]: pgmap v422: 97 pgs: 97 active+clean; 331 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-12T03:21:53.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:52 smithi135 ceph-mon[35474]: pgmap v422: 97 pgs: 97 active+clean; 331 MiB data, 736 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.4 MiB/s wr, 367 op/s 2024-09-12T03:21:54.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:53 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:54.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:53 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:53 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:21:55.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:54 smithi135 ceph-mon[35474]: pgmap v423: 97 pgs: 97 active+clean; 331 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.3 MiB/s wr, 377 op/s 2024-09-12T03:21:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:54 smithi121 ceph-mon[34963]: pgmap v423: 97 pgs: 97 active+clean; 331 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.3 MiB/s wr, 377 op/s 2024-09-12T03:21:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:54 smithi019 ceph-mon[30953]: pgmap v423: 97 pgs: 97 active+clean; 331 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.3 MiB/s wr, 377 op/s 2024-09-12T03:21:55.552 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T03:21:55.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:21:55.695 INFO:tasks.workunit.client.0.smithi019.stdout: 108 973293 1203 672523 1191 201110 662 2024-09-12T03:21:55.853 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:56.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 -- 172.21.15.19:0/2012997657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 msgr2=0x7f03a81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 --2- 172.21.15.19:0/2012997657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a81029b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0398009900 tx=0x7f039802a480 comp rx=0 tx=0).stop 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 -- 172.21.15.19:0/2012997657 shutdown_connections 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 --2- 172.21.15.19:0/2012997657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 --2- 172.21.15.19:0/2012997657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a81029b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 --2- 172.21.15.19:0/2012997657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 0x7f03a8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 -- 172.21.15.19:0/2012997657 >> 172.21.15.19:0/2012997657 conn(0x7f03a80fc7d0 msgr2=0x7f03a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 -- 172.21.15.19:0/2012997657 shutdown_connections 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.423+0000 7f03afd42640 1 -- 172.21.15.19:0/2012997657 wait complete. 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 Processor -- start 2024-09-12T03:21:56.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 -- start start 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 0x7f03a80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8110f80 con 0x7f03a8102570 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8110e00 con 0x7f03a8100e20 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03afd42640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03a8111100 con 0x7f03a8102ef0 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ae2df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ae2df640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53156/0 (socket says 172.21.15.19:53156) 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ae2df640 1 -- 172.21.15.19:0/1993972976 learned_addr learned my addr 172.21.15.19:0/1993972976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03adade640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 0x7f03a80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 -- 172.21.15.19:0/1993972976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 msgr2=0x7f03a80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 -- 172.21.15.19:0/1993972976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 msgr2=0x7f03a80fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 0x7f03a80fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.424+0000 7f03ad2dd640 1 -- 172.21.15.19:0/1993972976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a81a6a80 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f03ae2df640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f03ad2dd640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a80fd1c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0398005940 tx=0x7f0398003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039803f070 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0398031c30 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a81a6d10 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398035030 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.425+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a81a71a0 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.426+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a8101200 con 0x7f03a8102570 2024-09-12T03:21:56.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.428+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0398042070 con 0x7f03a8102570 2024-09-12T03:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.429+0000 7f039effd640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 0x7f03940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.429+0000 7f03adade640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 0x7f03940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.429+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f03980efb70 con 0x7f03a8102570 2024-09-12T03:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.429+0000 7f03adade640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 0x7f03940a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009bd0 tx=0x7f03a4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:56.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.432+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03980bc490 con 0x7f03a8102570 2024-09-12T03:21:56.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.666+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03a810a0a0 con 0x7f03a8102570 2024-09-12T03:21:56.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.666+0000 7f039effd640 1 -- 172.21.15.19:0/1993972976 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7f03980c0790 con 0x7f03a8102570 2024-09-12T03:21:56.667 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:21:56.667 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:21:56.668 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:21:56.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 msgr2=0x7f03940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 0x7f03940a5040 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009bd0 tx=0x7f03a4005f90 comp rx=0 tx=0).stop 2024-09-12T03:21:56.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 msgr2=0x7f03a80fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a80fd1c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0398005940 tx=0x7f0398003c10 comp rx=0 tx=0).stop 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 shutdown_connections 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03a8102ef0 0x7f03a80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03940a2b80 0x7f03940a5040 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f03a8102570 0x7f03a80fd1c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 --2- 172.21.15.19:0/1993972976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03a8100e20 0x7f03a80fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 >> 172.21.15.19:0/1993972976 conn(0x7f03a80fc7d0 msgr2=0x7f03a806a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 shutdown_connections 2024-09-12T03:21:56.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:56.669+0000 7f03afd42640 1 -- 172.21.15.19:0/1993972976 wait complete. 2024-09-12T03:21:57.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:56 smithi135 ceph-mon[35474]: pgmap v424: 97 pgs: 97 active+clean; 338 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 368 op/s 2024-09-12T03:21:57.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:56 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1993972976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:57.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:56 smithi019 ceph-mon[30953]: pgmap v424: 97 pgs: 97 active+clean; 338 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 368 op/s 2024-09-12T03:21:57.343 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:56 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1993972976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:57.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:56 smithi121 ceph-mon[34963]: pgmap v424: 97 pgs: 97 active+clean; 338 MiB data, 740 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 368 op/s 2024-09-12T03:21:57.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:56 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1993972976' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:21:57.757 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T03:21:57.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T03:21:58.034 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 -- 172.21.15.19:0/3285939536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0103740 msgr2=0x7f1be0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 --2- 172.21.15.19:0/3285939536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0103740 0x7f1be0107b80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009930 tx=0x7f1bcc02a420 comp rx=0 tx=0).stop 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 -- 172.21.15.19:0/3285939536 shutdown_connections 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 --2- 172.21.15.19:0/3285939536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0108290 0x7f1be010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 --2- 172.21.15.19:0/3285939536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0103740 0x7f1be0107b80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 --2- 172.21.15.19:0/3285939536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102d90 0x7f1be0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 -- 172.21.15.19:0/3285939536 >> 172.21.15.19:0/3285939536 conn(0x7f1be00fc7d0 msgr2=0x7f1be00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 -- 172.21.15.19:0/3285939536 shutdown_connections 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.547+0000 7f1be6c79640 1 -- 172.21.15.19:0/3285939536 wait complete. 2024-09-12T03:21:58.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 Processor -- start 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 -- start start 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0103740 0x7f1be019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0108290 0x7f1be019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0112050 con 0x7f1be0102d90 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0111ed0 con 0x7f1be0108290 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be6c79640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01121d0 con 0x7f1be0103740 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0103740 0x7f1be019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52950/0 (socket says 172.21.15.19:52950) 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be4a15640 1 -- 172.21.15.19:0/2950362018 learned_addr learned my addr 172.21.15.19:0/2950362018 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.548+0000 7f1be5216640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0108290 0x7f1be019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 -- 172.21.15.19:0/2950362018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0103740 msgr2=0x7f1be019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0103740 0x7f1be019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 -- 172.21.15.19:0/2950362018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0108290 msgr2=0x7f1be019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0108290 0x7f1be019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 -- 172.21.15.19:0/2950362018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be019f6c0 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be4a15640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c920 tx=0x7f1bd400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1bddffb640 1 -- 172.21.15.19:0/2950362018 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4007c70 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1bddffb640 1 -- 172.21.15.19:0/2950362018 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd4007e10 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be019f9b0 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.549+0000 7f1bddffb640 1 -- 172.21.15.19:0/2950362018 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4018890 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.550+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01a7290 con 0x7f1be0102d90 2024-09-12T03:21:58.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1bddffb640 1 -- 172.21.15.19:0/2950362018 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bd4018a50 con 0x7f1be0102d90 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1be9052740 1 Processor -- start 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1be9052740 1 -- start start 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1be9052740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c863b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1be9052740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.552+0000 7f1be9052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c870d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be9052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55909c7c92a0 con 0x55909c86e940 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be9052740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55909c770b10 con 0x55909c863760 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be9052740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55909c7c5260 con 0x55909c864080 2024-09-12T03:21:58.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c870d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c870d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52966/0 (socket says 172.21.15.19:52966) 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53190/0 (socket says 172.21.15.19:53190) 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1bdffff640 1 -- 172.21.15.19:0/1495313671 learned_addr learned my addr 172.21.15.19:0/1495313671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be5216640 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c863b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 -- 172.21.15.19:0/1495313671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 msgr2=0x55909c863b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c863b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 -- 172.21.15.19:0/1495313671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 msgr2=0x55909c870d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c870d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1bddffb640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 0x7f1bd00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 -- 172.21.15.19:0/1495313671 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909c53ab90 con 0x55909c864080 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be5216640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 0x7f1bd00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1bddffb640 1 -- 172.21.15.19:0/2950362018 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bd40cf0f0 con 0x7f1be0102d90 2024-09-12T03:21:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.553+0000 7f1be4a15640 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40ccbd0 tx=0x7f1bd4097cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5893301aec58bfb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1bcaffd640 1 -- 172.21.15.19:0/1495313671 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd409b730 con 0x55909c864080 2024-09-12T03:21:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1bcaffd640 1 -- 172.21.15.19:0/1495313671 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd409c060 con 0x55909c864080 2024-09-12T03:21:58.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1bcaffd640 1 -- 172.21.15.19:0/1495313671 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd409b2c0 con 0x55909c864080 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1be9052740 1 -- 172.21.15.19:0/1495313671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 msgr2=0x55909c86e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1be9052740 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40ccbd0 tx=0x7f1bd4097cc0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1be5216640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 0x7f1bd00a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.554+0000 7f1be9052740 1 -- 172.21.15.19:0/1495313671 shutdown_connections 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c870d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86e400 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 --2- 172.21.15.19:0/1495313671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c863b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 -- 172.21.15.19:0/1495313671 >> 172.21.15.19:0/1495313671 conn(0x55909c85fcf0 msgr2=0x55909c8600d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 -- 172.21.15.19:0/1495313671 shutdown_connections 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 -- 172.21.15.19:0/1495313671 wait complete. 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 -- start start 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.555+0000 7f1be9052740 1 Processor -- start 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c86b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c86e940 0x55909c86c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55909c871510 con 0x55909c863760 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55909c7516e0 con 0x55909c86e940 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55909c871690 con 0x55909c864080 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be9052740 10 client.0 _ll_register_callbacks cb 0x55909c775a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c86b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53202/0 (socket says 172.21.15.19:53202) 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 -- 172.21.15.19:0/3918112007 learned_addr learned my addr 172.21.15.19:0/3918112007 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be5216640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c86e940 0x55909c86c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 -- 172.21.15.19:0/3918112007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c86e940 msgr2=0x55909c86c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c86e940 0x55909c86c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 -- 172.21.15.19:0/3918112007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 msgr2=0x55909c86b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c86b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1bdffff640 1 -- 172.21.15.19:0/3918112007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909c87a9d0 con 0x55909c864080 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.556+0000 7f1be4a15640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c86b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009230 tx=0x7f1bcc031920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c89e82819a9d5c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bbffff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:21:58.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc035040 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bcc004440 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd409c060 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc005040 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1be9052740 10 client.25528 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909c87d340 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.557+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd409c060 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1bcc0035f0 con 0x55909c864080 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 10 client.25528 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 10 client.25528 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 20 client.25528 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:58.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c87e800 0x55909c880c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 4 client.25528 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909c7d69d0 con 0x55909c87e800 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be4a15640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c87e800 0x55909c880c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1be4a15640 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c87e800 0x55909c880c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a86b4903956908b server_cookie=960e27439e314979 in_seq=0 out_seq=0 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.558+0000 7f1bbffff640 10 client.25528 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1bbffff640 1 -- 172.21.15.19:0/3918112007 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55909c7d69d0 con 0x55909c87e800 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1bbffff640 10 client.25528 handle_command_reply: tid=0 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 client.25528 shutdown 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 msgr2=0x55909c86bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009230 tx=0x7f1bcc031920 comp rx=0 tx=0).stop 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 shutdown_connections 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c87e800 0x55909c880c30 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c86e940 0x55909c86c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c86bd20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1bbffff640 0 client.25528 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 --2- 172.21.15.19:0/3918112007 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c86b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 >> 172.21.15.19:0/3918112007 conn(0x55909c85fcf0 msgr2=0x55909c8659d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 shutdown_connections 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 1 -- 172.21.15.19:0/3918112007 wait complete. 2024-09-12T03:21:58.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.559+0000 7f1be9052740 20 client.25528 trim_cache size 0 max 16384 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 Processor -- start 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 -- start start 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c8661c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01af600 con 0x55909c86e940 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b1200 con 0x55909c863760 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.578+0000 7f1be9052740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01afcd0 con 0x55909c864080 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53216/0 (socket says 172.21.15.19:53216) 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:52994/0 (socket says 172.21.15.19:52994) 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bdffff640 1 -- 172.21.15.19:0/908475768 learned_addr learned my addr 172.21.15.19:0/908475768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be4a15640 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c8661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 -- 172.21.15.19:0/908475768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 msgr2=0x55909c85ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 -- 172.21.15.19:0/908475768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 msgr2=0x55909c8661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c8661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 -- 172.21.15.19:0/908475768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909c863eb0 con 0x55909c86e940 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bdffff640 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1be5216640 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8003e20 tx=0x7f1bd8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b6a7683279f968 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bca7fc640 1 -- 172.21.15.19:0/908475768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004600 con 0x55909c86e940 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bca7fc640 1 -- 172.21.15.19:0/908475768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd80047a0 con 0x55909c86e940 2024-09-12T03:21:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.579+0000 7f1bca7fc640 1 -- 172.21.15.19:0/908475768 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd80058e0 con 0x55909c86e940 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- 172.21.15.19:0/908475768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 msgr2=0x55909c886540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8003e20 tx=0x7f1bd8016000 comp rx=0 tx=0).stop 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- 172.21.15.19:0/908475768 shutdown_connections 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c86e940 0x55909c886540 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c864080 0x55909c85ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 --2- 172.21.15.19:0/908475768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c863760 0x55909c8661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- 172.21.15.19:0/908475768 >> 172.21.15.19:0/908475768 conn(0x55909c85fcf0 msgr2=0x55909c85e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:21:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- 172.21.15.19:0/908475768 shutdown_connections 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- 172.21.15.19:0/908475768 wait complete. 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.580+0000 7f1be9052740 1 -- start start 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 Processor -- start 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c864080 0x55909c8734a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 0x55909c8e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b0a70 con 0x55909c863760 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b03a0 con 0x55909c864080 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ad5e0 con 0x55909c86e940 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be9052740 10 client.0 _ll_register_callbacks cb 0x7f1bcc0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:21:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be5216640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 0x55909c8e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be4a15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53000/0 (socket says 172.21.15.19:53000) 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1be4a15640 1 -- 172.21.15.19:0/255989277 learned_addr learned my addr 172.21.15.19:0/255989277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.581+0000 7f1bdffff640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c864080 0x55909c8734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 -- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 msgr2=0x55909c8e2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 0x55909c8e2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 -- 172.21.15.19:0/255989277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c864080 msgr2=0x55909c8734a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c864080 0x55909c8734a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 -- 172.21.15.19:0/255989277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909c8e2b90 con 0x55909c863760 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be5216640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 0x55909c8e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be4a15640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4097dc0 tx=0x7f1bd400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb649c55272bbf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1bbffff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:21:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd409b4a0 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bd4099c60 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd80047a0 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1bd409b2c0 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be9052740 10 client.15987 fetch_fsmap learned FSMap version 30 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.582+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55909c8e9f20 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd80047a0 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f1bd409bbc0 con 0x55909c863760 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 10 client.15987 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 10 client.15987 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 20 client.15987 populate_metadata read hostname 'smithi019' 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c8ea250 0x55909c8ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 4 client.15987 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T03:21:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55909c7d0d20 con 0x55909c8ea250 2024-09-12T03:21:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1bdffff640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c8ea250 0x55909c8ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:21:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1bbffff640 10 client.15987 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:21:58.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:21:58.583+0000 7f1bdffff640 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c8ea250 0x55909c8ec670 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78660162803391b7 server_cookie=5985ef7fea471d6d in_seq=0 out_seq=0 2024-09-12T03:21:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:58 smithi135 ceph-mon[35474]: pgmap v425: 97 pgs: 97 active+clean; 338 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 294 op/s 2024-09-12T03:21:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:58 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:58 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:58 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:21:59.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:58 smithi121 ceph-mon[34963]: pgmap v425: 97 pgs: 97 active+clean; 338 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 294 op/s 2024-09-12T03:21:59.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:58 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:59.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:58 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:59.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:58 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:21:59.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:58 smithi019 ceph-mon[30953]: pgmap v425: 97 pgs: 97 active+clean; 338 MiB data, 742 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 294 op/s 2024-09-12T03:21:59.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:58 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:59.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:58 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:21:59.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:58 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:00.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:59 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:00.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:21:59 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:00.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:59 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:00.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:21:59 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:59 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:00.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:21:59 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:01.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:00 smithi135 ceph-mon[35474]: pgmap v426: 97 pgs: 97 active+clean; 340 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 167 op/s 2024-09-12T03:22:01.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:00 smithi121 ceph-mon[34963]: pgmap v426: 97 pgs: 97 active+clean; 340 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 167 op/s 2024-09-12T03:22:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:00 smithi019 ceph-mon[30953]: pgmap v426: 97 pgs: 97 active+clean; 340 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 167 op/s 2024-09-12T03:22:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:02 smithi135 ceph-mon[35474]: pgmap v427: 97 pgs: 97 active+clean; 348 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-12T03:22:03.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:02 smithi121 ceph-mon[34963]: pgmap v427: 97 pgs: 97 active+clean; 348 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-12T03:22:03.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:02 smithi019 ceph-mon[30953]: pgmap v427: 97 pgs: 97 active+clean; 348 MiB data, 776 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 144 op/s 2024-09-12T03:22:05.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:04 smithi135 ceph-mon[35474]: pgmap v428: 97 pgs: 97 active+clean; 360 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T03:22:05.344 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:04 smithi019 ceph-mon[30953]: pgmap v428: 97 pgs: 97 active+clean; 360 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.343+0000 7f1bbffff640 1 -- 172.21.15.19:0/255989277 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55909c7d0d20 con 0x55909c8ea250 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.343+0000 7f1bbffff640 10 client.15987 handle_command_reply: tid=0 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.343+0000 7f1be9052740 1 client.15987 shutdown 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 msgr2=0x55909c872f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4097dc0 tx=0x7f1bd400cf10 comp rx=0 tx=0).stop 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 shutdown_connections 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55909c86e940 0x55909c8e2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55909c8ea250 0x55909c8ec670 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55909c864080 0x55909c8734a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1bbffff640 0 client.15987 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 --2- 172.21.15.19:0/255989277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55909c863760 0x55909c872f60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 >> 172.21.15.19:0/255989277 conn(0x55909c85fcf0 msgr2=0x55909c86bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 shutdown_connections 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 1 -- 172.21.15.19:0/255989277 wait complete. 2024-09-12T03:22:05.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.344+0000 7f1be9052740 20 client.15987 trim_cache size 0 max 16384 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 10, 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "85516998": { 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 10, 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:22:05.347 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 144.7, 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:22:05.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:05.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:05.349 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 msgr2=0x7f1bd00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 0x7f1bd00a5040 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8005fd0 tx=0x7f1bd8009500 comp rx=0 tx=0).stop 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 msgr2=0x7f1be019a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400c920 tx=0x7f1bd400cde0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 shutdown_connections 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0108290 0x7f1be019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0103740 0x7f1be019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bd00a2b80 0x7f1bd00a5040 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 --2- 172.21.15.19:0/2950362018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0102d90 0x7f1be019a710 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 >> 172.21.15.19:0/2950362018 conn(0x7f1be00fc7d0 msgr2=0x7f1be0100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 shutdown_connections 2024-09-12T03:22:05.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:05.345+0000 7f1be6c79640 1 -- 172.21.15.19:0/2950362018 wait complete. 2024-09-12T03:22:05.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:04 smithi121 ceph-mon[34963]: pgmap v428: 97 pgs: 97 active+clean; 360 MiB data, 835 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 341 op/s 2024-09-12T03:22:05.696 INFO:tasks.workunit.client.0.smithi019.stdout: 114 71039 397 48655 318 15050 23 2024-09-12T03:22:06.125 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '85516998', quiesce took 10.6 seconds. Will release after: 14.1 2024-09-12T03:22:07.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:06 smithi135 ceph-mon[35474]: pgmap v429: 97 pgs: 97 active+clean; 366 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-12T03:22:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:06 smithi121 ceph-mon[34963]: pgmap v429: 97 pgs: 97 active+clean; 366 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-12T03:22:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:06 smithi019 ceph-mon[30953]: pgmap v429: 97 pgs: 97 active+clean; 366 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.4 MiB/s wr, 389 op/s 2024-09-12T03:22:09.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:08 smithi135 ceph-mon[35474]: pgmap v430: 97 pgs: 97 active+clean; 376 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-09-12T03:22:09.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:08 smithi121 ceph-mon[34963]: pgmap v430: 97 pgs: 97 active+clean; 376 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-09-12T03:22:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:08 smithi019 ceph-mon[30953]: pgmap v430: 97 pgs: 97 active+clean; 376 MiB data, 966 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 450 op/s 2024-09-12T03:22:11.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:10 smithi135 ceph-mon[35474]: pgmap v431: 97 pgs: 97 active+clean; 379 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-12T03:22:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:10 smithi121 ceph-mon[34963]: pgmap v431: 97 pgs: 97 active+clean; 379 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-12T03:22:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:10 smithi019 ceph-mon[30953]: pgmap v431: 97 pgs: 97 active+clean; 379 MiB data, 959 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 466 op/s 2024-09-12T03:22:13.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:12 smithi135 ceph-mon[35474]: pgmap v432: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-12T03:22:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:12 smithi121 ceph-mon[34963]: pgmap v432: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-12T03:22:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:12 smithi019 ceph-mon[30953]: pgmap v432: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 467 op/s 2024-09-12T03:22:15.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:14 smithi135 ceph-mon[35474]: pgmap v433: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 445 op/s 2024-09-12T03:22:15.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:14 smithi121 ceph-mon[34963]: pgmap v433: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 445 op/s 2024-09-12T03:22:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:14 smithi019 ceph-mon[30953]: pgmap v433: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 445 op/s 2024-09-12T03:22:15.696 INFO:tasks.workunit.client.0.smithi019.stdout: 114 185923 0 125230 0 36725 0 2024-09-12T03:22:17.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:16 smithi135 ceph-mon[35474]: pgmap v434: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-09-12T03:22:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:16 smithi121 ceph-mon[34963]: pgmap v434: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-09-12T03:22:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:16 smithi019 ceph-mon[30953]: pgmap v434: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-09-12T03:22:19.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:18 smithi135 ceph-mon[35474]: pgmap v435: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-12T03:22:19.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:18 smithi121 ceph-mon[34963]: pgmap v435: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-12T03:22:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:18 smithi019 ceph-mon[30953]: pgmap v435: 97 pgs: 97 active+clean; 380 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-09-12T03:22:20.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:22:20.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:22:20.507 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 -- 172.21.15.19:0/3666083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102570 msgr2=0x7f51501029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 --2- 172.21.15.19:0/3666083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102570 0x7f51501029b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5138009900 tx=0x7f513802a480 comp rx=0 tx=0).stop 2024-09-12T03:22:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 -- 172.21.15.19:0/3666083876 shutdown_connections 2024-09-12T03:22:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 --2- 172.21.15.19:0/3666083876 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150102ef0 0x7f515010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 --2- 172.21.15.19:0/3666083876 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102570 0x7f51501029b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 --2- 172.21.15.19:0/3666083876 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150100e20 0x7f5150101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 -- 172.21.15.19:0/3666083876 >> 172.21.15.19:0/3666083876 conn(0x7f51500fc7d0 msgr2=0x7f51500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 -- 172.21.15.19:0/3666083876 shutdown_connections 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.952+0000 7f5154b4e640 1 -- 172.21.15.19:0/3666083876 wait complete. 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 Processor -- start 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 -- start start 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 0x7f51501998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102ef0 0x7f51501a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5150110f80 con 0x7f5150102ef0 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5150110e00 con 0x7f5150102570 2024-09-12T03:22:20.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5154b4e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5150111100 con 0x7f5150100e20 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f514e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5147fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 0x7f51501998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f514e59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:52494/0 (socket says 172.21.15.19:52494) 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f514e59c640 1 -- 172.21.15.19:0/636889874 learned_addr learned my addr 172.21.15.19:0/636889874 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f514ed9d640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102ef0 0x7f51501a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5147fff640 1 -- 172.21.15.19:0/636889874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 msgr2=0x7f51501993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.953+0000 7f5147fff640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.954+0000 7f5147fff640 1 -- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102ef0 msgr2=0x7f51501a4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.954+0000 7f5147fff640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102ef0 0x7f51501a4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:20.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.954+0000 7f5147fff640 1 -- 172.21.15.19:0/636889874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51501a4970 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.954+0000 7f514e59c640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.954+0000 7f5147fff640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 0x7f51501998f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5138005940 tx=0x7f5138003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.957+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513803f070 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.957+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5138031c30 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.957+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51501a4b40 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.957+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138035030 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.957+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51501a4ff0 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.958+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5150101200 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.959+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5138042070 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.960+0000 7f51477fe640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 0x7f513c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.960+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51380ef5b0 con 0x7f5150102570 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.960+0000 7f514e59c640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 0x7f513c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:20.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.960+0000 7f514e59c640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 0x7f513c0a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5140011550 tx=0x7f5140005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:20.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:20.964+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f51380bbed0 con 0x7f5150102570 2024-09-12T03:22:21.023 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:21.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.195+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f515006a560 con 0x7f5150102570 2024-09-12T03:22:21.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.196+0000 7f51477fe640 1 -- 172.21.15.19:0/636889874 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7f51380c01d0 con 0x7f5150102570 2024-09-12T03:22:21.196 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:22:21.196 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:22:21.197 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 msgr2=0x7f513c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 0x7f513c0a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5140011550 tx=0x7f5140005f90 comp rx=0 tx=0).stop 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 msgr2=0x7f51501998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 0x7f51501998f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5138005940 tx=0x7f5138003c10 comp rx=0 tx=0).stop 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 shutdown_connections 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f513c0a2c00 0x7f513c0a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5150102ef0 0x7f51501a4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5150102570 0x7f51501998f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 --2- 172.21.15.19:0/636889874 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5150100e20 0x7f51501993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 >> 172.21.15.19:0/636889874 conn(0x7f51500fc7d0 msgr2=0x7f51500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 shutdown_connections 2024-09-12T03:22:21.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.198+0000 7f5154b4e640 1 -- 172.21.15.19:0/636889874 wait complete. 2024-09-12T03:22:21.207 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:20 smithi019 ceph-mon[30953]: pgmap v436: 97 pgs: 97 active+clean; 382 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 550 KiB/s wr, 81 op/s 2024-09-12T03:22:21.207 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:20 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:20 smithi135 ceph-mon[35474]: pgmap v436: 97 pgs: 97 active+clean; 382 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 550 KiB/s wr, 81 op/s 2024-09-12T03:22:21.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:20 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:20 smithi121 ceph-mon[34963]: pgmap v436: 97 pgs: 97 active+clean; 382 MiB data, 962 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 550 KiB/s wr, 81 op/s 2024-09-12T03:22:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:20 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:22:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 -- 172.21.15.19:0/1482623740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 msgr2=0x7ff1800748f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 --2- 172.21.15.19:0/1482623740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff1800748f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009900 tx=0x7ff17c02a480 comp rx=0 tx=0).stop 2024-09-12T03:22:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 -- 172.21.15.19:0/1482623740 shutdown_connections 2024-09-12T03:22:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 --2- 172.21.15.19:0/1482623740 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff180110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 --2- 172.21.15.19:0/1482623740 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 0x7ff18010df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 --2- 172.21.15.19:0/1482623740 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff1800748f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 -- 172.21.15.19:0/1482623740 >> 172.21.15.19:0/1482623740 conn(0x7ff1800fd8c0 msgr2=0x7ff1800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 -- 172.21.15.19:0/1482623740 shutdown_connections 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.718+0000 7ff18800a640 1 -- 172.21.15.19:0/1482623740 wait complete. 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 Processor -- start 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 -- start start 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 0x7ff18019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff18019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff180114170 con 0x7ff180074510 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff180113ff0 con 0x7ff18010e440 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff18800a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801142f0 con 0x7ff180074e30 2024-09-12T03:22:21.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff185da6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff185da6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47792/0 (socket says 172.21.15.19:47792) 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.719+0000 7ff185da6640 1 -- 172.21.15.19:0/2903671891 learned_addr learned my addr 172.21.15.19:0/2903671891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff1855a5640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 0x7ff18019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff1865a7640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff18019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 -- 172.21.15.19:0/2903671891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 msgr2=0x7ff18019ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 0x7ff18019ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 -- 172.21.15.19:0/2903671891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 msgr2=0x7ff18019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff18019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 -- 172.21.15.19:0/2903671891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18019f6d0 con 0x7ff180074510 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff1865a7640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff18019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff185da6640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006010 tx=0x7ff17c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff176ffd640 1 -- 172.21.15.19:0/2903671891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c03f030 con 0x7ff180074510 2024-09-12T03:22:21.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff176ffd640 1 -- 172.21.15.19:0/2903671891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c003ce0 con 0x7ff180074510 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.720+0000 7ff176ffd640 1 -- 172.21.15.19:0/2903671891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c035070 con 0x7ff180074510 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.721+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18019f960 con 0x7ff180074510 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.721+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18019fdf0 con 0x7ff180074510 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.723+0000 7ff176ffd640 1 -- 172.21.15.19:0/2903671891 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff17c042030 con 0x7ff180074510 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.723+0000 7ff18a3e3740 1 Processor -- start 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.723+0000 7ff18a3e3740 1 -- start start 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d31880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 0x555807d3c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555807c972e0 con 0x555807d3c680 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555807c3ece0 con 0x555807d314a0 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555807c932a0 con 0x555807d31dc0 2024-09-12T03:22:21.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff176ffd640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47798/0 (socket says 172.21.15.19:47798) 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1855a5640 1 -- 172.21.15.19:0/299588884 learned_addr learned my addr 172.21.15.19:0/299588884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1865a7640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 0x555807d3c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff176ffd640 1 -- 172.21.15.19:0/2903671891 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff17c0ef540 con 0x7ff180074510 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1865a7640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d31880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 -- 172.21.15.19:0/299588884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 msgr2=0x555807d31880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d31880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 -- 172.21.15.19:0/299588884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 msgr2=0x555807d3ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff185da6640 1 -- 172.21.15.19:0/299588884 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555807c23440 con 0x555807d31dc0 2024-09-12T03:22:21.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.724+0000 7ff1855a5640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.725+0000 7ff1865a7640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 0x7ff16c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff178005c40 tx=0x7ff178012a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.725+0000 7ff185da6640 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 0x555807d3c140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0ed0a0 tx=0x7ff17c0b1350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d872f881e669ca17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.725+0000 7ff16bfff640 1 -- 172.21.15.19:0/299588884 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c049070 con 0x555807d31dc0 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.725+0000 7ff16bfff640 1 -- 172.21.15.19:0/299588884 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17c03b580 con 0x555807d31dc0 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.725+0000 7ff16bfff640 1 -- 172.21.15.19:0/299588884 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0bbe70 con 0x555807d31dc0 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- 172.21.15.19:0/299588884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 msgr2=0x555807d3c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 0x555807d3c140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff17c0ed0a0 tx=0x7ff17c0b1350 comp rx=0 tx=0).stop 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- 172.21.15.19:0/299588884 shutdown_connections 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d3c680 0x555807d3ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d31dc0 0x555807d3c140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/299588884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d31880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- 172.21.15.19:0/299588884 >> 172.21.15.19:0/299588884 conn(0x555807d2da30 msgr2=0x555807d2de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- 172.21.15.19:0/299588884 shutdown_connections 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- 172.21.15.19:0/299588884 wait complete. 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 -- start start 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.726+0000 7ff18a3e3740 1 Processor -- start 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d3c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d39880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555807d3f010 con 0x555807d31dc0 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555807c52050 con 0x555807d314a0 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555807d3f190 con 0x555807d3c680 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff18a3e3740 10 client.0 _ll_register_callbacks cb 0x555807c92160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47804/0 (socket says 172.21.15.19:47804) 2024-09-12T03:22:21.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 -- 172.21.15.19:0/1904368331 learned_addr learned my addr 172.21.15.19:0/1904368331 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff185da6640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d3c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1865a7640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d39880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 -- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 msgr2=0x555807d39880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d39880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 -- 172.21.15.19:0/1904368331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 msgr2=0x555807d3c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d3c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 -- 172.21.15.19:0/1904368331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555807a083c0 con 0x555807d31dc0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff185da6640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1855a5640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff170010a20 tx=0x7ff170010ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1b7647b77ade93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.727+0000 7ff1697fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17000fc50 con 0x555807d31dc0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555807ca4850 con 0x555807d31dc0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff17000fdf0 con 0x555807d31dc0 2024-09-12T03:22:21.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff170005020 con 0x555807d31dc0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 10 client.16005 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555807d4aa40 con 0x555807d31dc0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x555807ca4850 con 0x555807d31dc0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff170019410 con 0x555807d31dc0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 10 client.16005 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 10 client.16005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 20 client.16005 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807d4bea0 0x555807d4e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 4 client.16005 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.728+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555807c9ed60 con 0x555807d4bea0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.729+0000 7ff185da6640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807d4bea0 0x555807d4e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.729+0000 7ff185da6640 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807d4bea0 0x555807d4e300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a47d31791c83ede4 server_cookie=1906473372404ab5 in_seq=0 out_seq=0 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.729+0000 7ff1697fa640 10 client.16005 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff1697fa640 1 -- 172.21.15.19:0/1904368331 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555807c9ed60 con 0x555807d4bea0 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff1697fa640 10 client.16005 handle_command_reply: tid=0 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 client.16005 shutdown 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 msgr2=0x555807d392e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff170010a20 tx=0x7ff170010ee0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 shutdown_connections 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d392e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807d4bea0 0x555807d4e300 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/1904368331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d3c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff1697fa640 0 client.16005 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:21.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 >> 172.21.15.19:0/1904368331 conn(0x555807d2da30 msgr2=0x555807d33710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 shutdown_connections 2024-09-12T03:22:21.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 1 -- 172.21.15.19:0/1904368331 wait complete. 2024-09-12T03:22:21.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.730+0000 7ff18a3e3740 20 client.16005 trim_cache size 0 max 16384 2024-09-12T03:22:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.747+0000 7ff18a3e3740 1 Processor -- start 2024-09-12T03:22:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.747+0000 7ff18a3e3740 1 -- start start 2024-09-12T03:22:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d314a0 0x555807d53c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d3c680 0x555807d36540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801af750 con 0x555807d31dc0 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801b1350 con 0x555807d3c680 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801afe20 con 0x555807d314a0 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47812/0 (socket says 172.21.15.19:47812) 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 -- 172.21.15.19:0/2124275472 learned_addr learned my addr 172.21.15.19:0/2124275472 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff185da6640 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d314a0 0x555807d53c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1865a7640 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d3c680 0x555807d36540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 -- 172.21.15.19:0/2124275472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d314a0 msgr2=0x555807d53c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d314a0 0x555807d53c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 -- 172.21.15.19:0/2124275472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d3c680 msgr2=0x555807d36540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d3c680 0x555807d36540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 -- 172.21.15.19:0/2124275472 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555807d31bf0 con 0x555807d31dc0 2024-09-12T03:22:21.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.748+0000 7ff1855a5640 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff17000a5e0 tx=0x7ff170010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76efb4e90b13da9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.749+0000 7ff16b7fe640 1 -- 172.21.15.19:0/2124275472 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170028040 con 0x555807d31dc0 2024-09-12T03:22:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.749+0000 7ff16b7fe640 1 -- 172.21.15.19:0/2124275472 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17000a2c0 con 0x555807d31dc0 2024-09-12T03:22:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.749+0000 7ff16b7fe640 1 -- 172.21.15.19:0/2124275472 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170005420 con 0x555807d31dc0 2024-09-12T03:22:21.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.749+0000 7ff18a3e3740 1 -- 172.21.15.19:0/2124275472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 msgr2=0x555807d54150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.749+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff17000a5e0 tx=0x7ff170010640 comp rx=0 tx=0).stop 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 -- 172.21.15.19:0/2124275472 shutdown_connections 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d3c680 0x555807d36540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d54150 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/2124275472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d314a0 0x555807d53c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 -- 172.21.15.19:0/2124275472 >> 172.21.15.19:0/2124275472 conn(0x555807d2da30 msgr2=0x555807d3b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 -- 172.21.15.19:0/2124275472 shutdown_connections 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 -- 172.21.15.19:0/2124275472 wait complete. 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 -- start start 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 Processor -- start 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d42160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.750+0000 7ff18a3e3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801b0bc0 con 0x555807d31dc0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801b04f0 con 0x555807d314a0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff18a3e3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff1801ad730 con 0x555807d3c680 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff18a3e3740 10 client.0 _ll_register_callbacks cb 0x7ff17000a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff185da6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d42160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47826/0 (socket says 172.21.15.19:47826) 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 -- 172.21.15.19:0/4166882014 learned_addr learned my addr 172.21.15.19:0/4166882014 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1865a7640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 -- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 msgr2=0x555807d42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 -- 172.21.15.19:0/4166882014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 msgr2=0x555807d42160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:21.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d42160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 -- 172.21.15.19:0/4166882014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555807db5d50 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1855a5640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff1700195c0 tx=0x7ff1700193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aa212ede2c1e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1697fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170028070 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff170005c00 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.751+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff17000a2c0 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff170019a20 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 10 client.16011 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555807db7000 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff17000a2c0 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7ff170004a60 con 0x555807d31dc0 2024-09-12T03:22:21.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 10 client.16011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 20 client.16011 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807db7750 0x555807db9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 4 client.16011 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555807c1b210 con 0x555807db7750 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.752+0000 7ff185da6640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807db7750 0x555807db9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff185da6640 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807db7750 0x555807db9b70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f74b585009fc46ec server_cookie=273de93e8641131e in_seq=0 out_seq=0 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff1697fa640 10 client.16011 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff1697fa640 1 -- 172.21.15.19:0/4166882014 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555807c1b210 con 0x555807db7750 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff1697fa640 10 client.16011 handle_command_reply: tid=0 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 client.16011 shutdown 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 msgr2=0x555807d426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff1700195c0 tx=0x7ff1700193e0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 shutdown_connections 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555807d3c680 0x555807d42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555807d31dc0 0x555807d426a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555807db7750 0x555807db9b70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 --2- 172.21.15.19:0/4166882014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555807d314a0 0x555807d42160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff1697fa640 0 client.16011 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 >> 172.21.15.19:0/4166882014 conn(0x555807d2da30 msgr2=0x555807d2c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 shutdown_connections 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 1 -- 172.21.15.19:0/4166882014 wait complete. 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.753+0000 7ff18a3e3740 20 client.16011 trim_cache size 0 max 16384 2024-09-12T03:22:21.758 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:22:21.759 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:22:21.759 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:22:21.759 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:22:21.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 msgr2=0x7ff16c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 0x7ff16c0a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff178005c40 tx=0x7ff178012a90 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 msgr2=0x7ff18019a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff17c006010 tx=0x7ff17c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 shutdown_connections 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff18010e440 0x7ff18019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff180074e30 0x7ff18019ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff16c0a2b80 0x7ff16c0a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 --2- 172.21.15.19:0/2903671891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff180074510 0x7ff18019a720 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:21.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 >> 172.21.15.19:0/2903671891 conn(0x7ff1800fd8c0 msgr2=0x7ff1800ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:21.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 shutdown_connections 2024-09-12T03:22:21.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:21.754+0000 7ff18800a640 1 -- 172.21.15.19:0/2903671891 wait complete. 2024-09-12T03:22:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/636889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:22.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:21 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:22:22.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/636889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:22.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:21 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:22:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/636889874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:22:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:22:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:21 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:22:22.626 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:22:22.627 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:22:23.110 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 85516998 --release --await' 2024-09-12T03:22:23.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 85516998 --release --await 2024-09-12T03:22:23.220 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:23.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:22 smithi135 ceph-mon[35474]: pgmap v437: 97 pgs: 97 active+clean; 391 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-12T03:22:23.301 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:22 smithi121 ceph-mon[34963]: pgmap v437: 97 pgs: 97 active+clean; 391 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-12T03:22:23.342 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:22 smithi019 ceph-mon[30953]: pgmap v437: 97 pgs: 97 active+clean; 391 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-09-12T03:22:23.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.693+0000 7fea5d523640 1 -- 172.21.15.19:0/2397620154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58102d30 msgr2=0x7fea58103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.693+0000 7fea5d523640 1 --2- 172.21.15.19:0/2397620154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58102d30 0x7fea58103110 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fea48009cd0 tx=0x7fea4802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 -- 172.21.15.19:0/2397620154 shutdown_connections 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 --2- 172.21.15.19:0/2397620154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea581082a0 0x7fea5810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 --2- 172.21.15.19:0/2397620154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581036e0 0x7fea58107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 --2- 172.21.15.19:0/2397620154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea58102d30 0x7fea58103110 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 -- 172.21.15.19:0/2397620154 >> 172.21.15.19:0/2397620154 conn(0x7fea580fc7d0 msgr2=0x7fea580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 -- 172.21.15.19:0/2397620154 shutdown_connections 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 -- 172.21.15.19:0/2397620154 wait complete. 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 Processor -- start 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.694+0000 7fea5d523640 1 -- start start 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea58102d30 0x7fea581985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581082a0 0x7fea5819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fea581120a0 con 0x7fea581036e0 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea58111f20 con 0x7fea58102d30 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea5d523640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fea58112220 con 0x7fea581082a0 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47850/0 (socket says 172.21.15.19:47850) 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 -- 172.21.15.19:0/4170528998 learned_addr learned my addr 172.21.15.19:0/4170528998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea577fe640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581082a0 0x7fea5819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea56ffd640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea58102d30 0x7fea581985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 -- 172.21.15.19:0/4170528998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581082a0 msgr2=0x7fea5819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581082a0 0x7fea5819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 -- 172.21.15.19:0/4170528998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea58102d30 msgr2=0x7fea581985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea58102d30 0x7fea581985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 -- 172.21.15.19:0/4170528998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea5819d590 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.695+0000 7fea567fc640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d750 tx=0x7fea4000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.696+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40017070 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.696+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea40005500 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.696+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea5819d880 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.696+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea40004e00 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.696+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea581a5160 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.697+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea5806a850 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.698+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea40004fa0 con 0x7fea581036e0 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.699+0000 7fea4ffff640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 0x7fea440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:23.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.699+0000 7fea56ffd640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 0x7fea440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.699+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fea400ced60 con 0x7fea581036e0 2024-09-12T03:22:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.700+0000 7fea56ffd640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 0x7fea440a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea48006010 tx=0x7fea48009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:23.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.702+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea4009b620 con 0x7fea581036e0 2024-09-12T03:22:23.787 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:23.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.919+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea5810a2b0 con 0x7fea581036e0 2024-09-12T03:22:23.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.921+0000 7fea4ffff640 1 -- 172.21.15.19:0/4170528998 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7fea400bb030 con 0x7fea581036e0 2024-09-12T03:22:23.921 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:22:23.922 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:22:23.922 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:22:23.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.936+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 msgr2=0x7fea440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.936+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 0x7fea440a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fea48006010 tx=0x7fea48009360 comp rx=0 tx=0).stop 2024-09-12T03:22:23.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.936+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 msgr2=0x7fea58198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:23.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.936+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fea4000d750 tx=0x7fea4000dc10 comp rx=0 tx=0).stop 2024-09-12T03:22:23.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 shutdown_connections 2024-09-12T03:22:23.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fea581082a0 0x7fea5819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fea440a2b80 0x7fea440a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fea581036e0 0x7fea58198b20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 --2- 172.21.15.19:0/4170528998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea58102d30 0x7fea581985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.938+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 >> 172.21.15.19:0/4170528998 conn(0x7fea580fc7d0 msgr2=0x7fea58105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.942+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 shutdown_connections 2024-09-12T03:22:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:23.943+0000 7fea5d523640 1 -- 172.21.15.19:0/4170528998 wait complete. 2024-09-12T03:22:24.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:23 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4170528998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:24.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:23 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4170528998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:24.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:23 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4170528998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:22:24.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- 172.21.15.19:0/829589462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4100e20 msgr2=0x7f76f4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 --2- 172.21.15.19:0/829589462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4100e20 0x7f76f4101200 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009900 tx=0x7f76e402a480 comp rx=0 tx=0).stop 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- 172.21.15.19:0/829589462 shutdown_connections 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 --2- 172.21.15.19:0/829589462 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76f4102ef0 0x7f76f410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 --2- 172.21.15.19:0/829589462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 0x7f76f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 --2- 172.21.15.19:0/829589462 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4100e20 0x7f76f4101200 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- 172.21.15.19:0/829589462 >> 172.21.15.19:0/829589462 conn(0x7f76f40fc7d0 msgr2=0x7f76f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- 172.21.15.19:0/829589462 shutdown_connections 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- 172.21.15.19:0/829589462 wait complete. 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 Processor -- start 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.064+0000 7f76fadf8640 1 -- start start 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76f4100e20 0x7f76f4109dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 0x7f76f410a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76f410fe70 con 0x7f76f4102ef0 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76f410fcf0 con 0x7f76f4102570 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76fadf8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76f410fff0 con 0x7f76f4100e20 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47856/0 (socket says 172.21.15.19:47856) 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 0x7f76f410a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 -- 172.21.15.19:0/826621999 learned_addr learned my addr 172.21.15.19:0/826621999 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 -- 172.21.15.19:0/826621999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76f4100e20 msgr2=0x7f76f4109dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76f4100e20 0x7f76f4109dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 -- 172.21.15.19:0/826621999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 msgr2=0x7f76f410a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 0x7f76f410a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 -- 172.21.15.19:0/826621999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41aad80 con 0x7f76f4102ef0 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f9395640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e970 tx=0x7f76ec00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.065+0000 7f76f1ffb640 1 -- 172.21.15.19:0/826621999 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00ccc0 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.066+0000 7f76f1ffb640 1 -- 172.21.15.19:0/826621999 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec00ce60 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.066+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41ab070 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.066+0000 7f76f1ffb640 1 -- 172.21.15.19:0/826621999 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec010490 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.066+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41ab560 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.068+0000 7f76f1ffb640 1 -- 172.21.15.19:0/826621999 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76ec005020 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.069+0000 7f76f1ffb640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.069+0000 7f76fd1d1740 1 Processor -- start 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.069+0000 7f76f8b94640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.069+0000 7f76fd1d1740 1 -- start start 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.069+0000 7f76f1ffb640 1 -- 172.21.15.19:0/826621999 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f76ec0cf530 con 0x7f76f4102ef0 2024-09-12T03:22:25.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334626d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 0x55e334629630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e334581620 con 0x55e33461c060 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e334528e50 con 0x55e33461c980 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e33457d5e0 con 0x55e334627240 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f8b94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f8b94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47862/0 (socket says 172.21.15.19:47862) 2024-09-12T03:22:25.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f8b94640 1 -- 172.21.15.19:0/1547400118 learned_addr learned my addr 172.21.15.19:0/1547400118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f3fff640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334626d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 0x55e334629630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f8b94640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 0x7f76e00a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76e40096c0 tx=0x7f76e4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 -- 172.21.15.19:0/1547400118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 msgr2=0x55e334626d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334626d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 -- 172.21.15.19:0/1547400118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 msgr2=0x55e33461c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 -- 172.21.15.19:0/1547400118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e334519d40 con 0x55e334627240 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f8b94640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76f9395640 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 0x55e334629630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cd040 tx=0x7f76ec09b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155b8b6084c902f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.070+0000 7f76eaffd640 1 -- 172.21.15.19:0/1547400118 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec09f070 con 0x55e334627240 2024-09-12T03:22:25.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76eaffd640 1 -- 172.21.15.19:0/1547400118 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec0a4a10 con 0x55e334627240 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 -- 172.21.15.19:0/1547400118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 msgr2=0x55e334629630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 0x55e334629630 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0cd040 tx=0x7f76ec09b7c0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 -- 172.21.15.19:0/1547400118 shutdown_connections 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e334627240 0x55e334629630 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334626d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/1547400118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c060 0x55e33461c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 -- 172.21.15.19:0/1547400118 >> 172.21.15.19:0/1547400118 conn(0x55e3346185f0 msgr2=0x55e3346189d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 -- 172.21.15.19:0/1547400118 shutdown_connections 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.071+0000 7f76fd1d1740 1 -- 172.21.15.19:0/1547400118 wait complete. 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 -- start start 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 Processor -- start 2024-09-12T03:22:25.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334624680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e334624bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e334629db0 con 0x55e334627240 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e334509f10 con 0x55e33461c980 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e334629f30 con 0x55e33461c060 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76fd1d1740 10 client.0 _ll_register_callbacks cb 0x55e33452ddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f9395640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e334624bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f8b94640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334624680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334624680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56226/0 (socket says 172.21.15.19:56226) 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f8b94640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:52606/0 (socket says 172.21.15.19:52606) 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.072+0000 7f76f3fff640 1 -- 172.21.15.19:0/889839681 learned_addr learned my addr 172.21.15.19:0/889839681 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 -- 172.21.15.19:0/889839681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 msgr2=0x55e334624680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334624680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 -- 172.21.15.19:0/889839681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 msgr2=0x55e334624bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e334624bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 -- 172.21.15.19:0/889839681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e334290580 con 0x55e33461c060 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76f8b94640 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005770 tx=0x7f76e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94970280b9f12040 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e404c020 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec0a4a10 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.073+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f76e4031740 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76e4035040 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 10 client.25570 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e33435a960 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76ec0a4a10 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f76e4048aa0 con 0x55e33461c060 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 10 client.25570 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:25.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 10 client.25570 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 20 client.25570 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e334636e20 0x55e3346392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 4 client.25570 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.074+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e33458eb90 con 0x55e334636e20 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.075+0000 7f76f3fff640 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e334636e20 0x55e3346392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.075+0000 7f76f3fff640 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e334636e20 0x55e3346392c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=978039146848645f server_cookie=c63c9f3e53061dc0 in_seq=0 out_seq=0 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.075+0000 7f76d3fff640 10 client.25570 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76d3fff640 1 -- 172.21.15.19:0/889839681 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e33458eb90 con 0x55e334636e20 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76d3fff640 10 client.25570 handle_command_reply: tid=0 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 client.25570 shutdown 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 msgr2=0x55e334624140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005770 tx=0x7f76e4009260 comp rx=0 tx=0).stop 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 shutdown_connections 2024-09-12T03:22:25.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e334624bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e334636e20 0x55e3346392c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e334624680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/889839681 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e334624140 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76d3fff640 0 client.25570 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 >> 172.21.15.19:0/889839681 conn(0x55e3346185f0 msgr2=0x55e33461e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 shutdown_connections 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 1 -- 172.21.15.19:0/889839681 wait complete. 2024-09-12T03:22:25.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.076+0000 7f76fd1d1740 20 client.25570 trim_cache size 0 max 16384 2024-09-12T03:22:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 Processor -- start 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 -- start start 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33461e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e334627240 0x55e33463f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b3b60 con 0x55e33461c980 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b5760 con 0x55e334627240 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.094+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b4230 con 0x55e33461c060 2024-09-12T03:22:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47878/0 (socket says 172.21.15.19:47878) 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 -- 172.21.15.19:0/757460317 learned_addr learned my addr 172.21.15.19:0/757460317 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f9395640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e334627240 0x55e33463f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f8b94640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33461e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 -- 172.21.15.19:0/757460317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 msgr2=0x55e33461e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33461e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 -- 172.21.15.19:0/757460317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e334627240 msgr2=0x55e33463f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e334627240 0x55e33463f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 -- 172.21.15.19:0/757460317 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3346192c0 con 0x55e33461c980 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f8b94640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33461e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76f3fff640 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55e334625780 tx=0x7f76dc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8bbb5fc149bc77 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76ea7fc640 1 -- 172.21.15.19:0/757460317 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc01ca50 con 0x55e33461c980 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76ea7fc640 1 -- 172.21.15.19:0/757460317 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc01cbf0 con 0x55e33461c980 2024-09-12T03:22:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.095+0000 7f76ea7fc640 1 -- 172.21.15.19:0/757460317 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc025640 con 0x55e33461c980 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 -- 172.21.15.19:0/757460317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 msgr2=0x55e33463eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55e334625780 tx=0x7f76dc009500 comp rx=0 tx=0).stop 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 -- 172.21.15.19:0/757460317 shutdown_connections 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e334627240 0x55e33463f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e33461c980 0x55e33463eb90 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/757460317 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33461e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.096+0000 7f76fd1d1740 1 -- 172.21.15.19:0/757460317 >> 172.21.15.19:0/757460317 conn(0x55e3346185f0 msgr2=0x55e33461dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- 172.21.15.19:0/757460317 shutdown_connections 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- 172.21.15.19:0/757460317 wait complete. 2024-09-12T03:22:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- start start 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 Processor -- start 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33462f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b4fd0 con 0x55e334627240 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b4900 con 0x55e33461c980 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b1b40 con 0x55e33461c060 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.097+0000 7f76fd1d1740 10 client.0 _ll_register_callbacks cb 0x7f76e40042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f9395640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56252/0 (socket says 172.21.15.19:56252) 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f9395640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47882/0 (socket says 172.21.15.19:47882) 2024-09-12T03:22:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f8b94640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33462f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 -- 172.21.15.19:0/2311313254 learned_addr learned my addr 172.21.15.19:0/2311313254 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 -- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 msgr2=0x55e33462f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33462f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 -- 172.21.15.19:0/2311313254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 msgr2=0x55e33462c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 -- 172.21.15.19:0/2311313254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e33462c980 con 0x55e33461c980 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f9395640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76f3fff640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76dc000f80 tx=0x7f76dc01cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b66c968a4c76186 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.098+0000 7f76d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc004ab0 con 0x55e33461c980 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f76dc004c50 con 0x55e33461c980 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76dc01cbf0 con 0x55e33461c980 2024-09-12T03:22:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f76dc004e50 con 0x55e33461c980 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 10 client.25553 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3346a2580 con 0x55e33461c980 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76dc01cbf0 con 0x55e33461c980 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f76dc024070 con 0x55e33461c980 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 10 client.25553 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 10 client.25553 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 20 client.25553 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e3346a2ce0 0x55e3346a5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 4 client.25553 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "85516998", "release": true, "await": true}] 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.099+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "85516998", "release": true, "await": true}) -- 0x55e3345890a0 con 0x55e3346a2ce0 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.100+0000 7f76f8b94640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e3346a2ce0 0x55e3346a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.100+0000 7f76f8b94640 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e3346a2ce0 0x55e3346a5100 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc2ca19b3bc24d5 server_cookie=1312f19ab51c7d06 in_seq=0 out_seq=0 2024-09-12T03:22:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.100+0000 7f76d3fff640 10 client.25553 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:25.139 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:25.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:24 smithi019 ceph-mon[30953]: pgmap v438: 97 pgs: 97 active+clean; 398 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-09-12T03:22:25.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:24 smithi135 ceph-mon[35474]: pgmap v438: 97 pgs: 97 active+clean; 398 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-09-12T03:22:25.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:24 smithi121 ceph-mon[34963]: pgmap v438: 97 pgs: 97 active+clean; 398 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 373 KiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76d3fff640 1 -- 172.21.15.19:0/2311313254 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e3345890a0 con 0x55e3346a2ce0 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76d3fff640 10 client.25553 handle_command_reply: tid=0 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 client.25553 shutdown 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 msgr2=0x55e33462f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f76dc000f80 tx=0x7f76dc01cee0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 shutdown_connections 2024-09-12T03:22:25.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e334627240 0x55e33462c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e3346a2ce0 0x55e3346a5100 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e33461c980 0x55e33462f8f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 --2- 172.21.15.19:0/2311313254 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e33461c060 0x55e33462f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76d3fff640 0 client.25553 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.548+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 >> 172.21.15.19:0/2311313254 conn(0x55e3346185f0 msgr2=0x55e334617750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 shutdown_connections 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fd1d1740 1 -- 172.21.15.19:0/2311313254 wait complete. 2024-09-12T03:22:25.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fd1d1740 20 client.25553 trim_cache size 0 max 16384 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 30, 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 12, 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "85516998": { 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 12, 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:22:25.551 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 144.7, 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:25.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 msgr2=0x7f76e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 0x7f76e00a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f76e40096c0 tx=0x7f76e4000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 msgr2=0x7f76f410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00e970 tx=0x7f76ec00ed70 comp rx=0 tx=0).stop 2024-09-12T03:22:25.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 shutdown_connections 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76e00a2b80 0x7f76e00a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f76f4102ef0 0x7f76f410a850 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f76f4102570 0x7f76f410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 --2- 172.21.15.19:0/826621999 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f76f4100e20 0x7f76f4109dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.549+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 >> 172.21.15.19:0/826621999 conn(0x7f76f40fc7d0 msgr2=0x7f76f40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.550+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 shutdown_connections 2024-09-12T03:22:25.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.550+0000 7f76fadf8640 1 -- 172.21.15.19:0/826621999 wait complete. 2024-09-12T03:22:25.696 INFO:tasks.workunit.client.0.smithi019.stdout: 114 14573 24 9644 2 2372 9 2024-09-12T03:22:25.719 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 -- 172.21.15.19:0/4207033472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c102570 msgr2=0x7faa5c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 --2- 172.21.15.19:0/4207033472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c102570 0x7faa5c1029b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7faa44009900 tx=0x7faa4402a480 comp rx=0 tx=0).stop 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 -- 172.21.15.19:0/4207033472 shutdown_connections 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 --2- 172.21.15.19:0/4207033472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 --2- 172.21.15.19:0/4207033472 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c102570 0x7faa5c1029b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 --2- 172.21.15.19:0/4207033472 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c100e20 0x7faa5c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 -- 172.21.15.19:0/4207033472 >> 172.21.15.19:0/4207033472 conn(0x7faa5c0fc7d0 msgr2=0x7faa5c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 -- 172.21.15.19:0/4207033472 shutdown_connections 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.706+0000 7faa61d1e640 1 -- 172.21.15.19:0/4207033472 wait complete. 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 Processor -- start 2024-09-12T03:22:25.720 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 -- start start 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c102570 0x7faa5c19ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c110f80 con 0x7faa5c100e20 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c110e00 con 0x7faa5c102ef0 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa61d1e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c111100 con 0x7faa5c102570 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c102570 0x7faa5c19ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47886/0 (socket says 172.21.15.19:47886) 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa5b7fe640 1 -- 172.21.15.19:0/3783371125 learned_addr learned my addr 172.21.15.19:0/3783371125 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.707+0000 7faa5bfff640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 -- 172.21.15.19:0/3783371125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c102570 msgr2=0x7faa5c19ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c102570 0x7faa5c19ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 -- 172.21.15.19:0/3783371125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 msgr2=0x7faa5c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 -- 172.21.15.19:0/3783371125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c19f670 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5bfff640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e9f0 tx=0x7faa4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa597fa640 1 -- 172.21.15.19:0/3783371125 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c00cd80 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa597fa640 1 -- 172.21.15.19:0/3783371125 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa4c005500 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c19f960 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa597fa640 1 -- 172.21.15.19:0/3783371125 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4c010690 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.708+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c1a7240 con 0x7faa5c100e20 2024-09-12T03:22:25.722 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.710+0000 7faa597fa640 1 -- 172.21.15.19:0/3783371125 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa4c0046c0 con 0x7faa5c100e20 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 Processor -- start 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 -- start start 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa597fa640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 0x7faa480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56481632f2f0 con 0x5648163c95d0 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5648162d6cf0 con 0x5648163cab40 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa640f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56481632b2b0 con 0x5648163d4370 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa53fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47898/0 (socket says 172.21.15.19:47898) 2024-09-12T03:22:25.723 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa5bfff640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 0x7faa480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa53fff640 1 -- 172.21.15.19:0/682388001 learned_addr learned my addr 172.21.15.19:0/682388001 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa5bfff640 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.711+0000 7faa597fa640 1 -- 172.21.15.19:0/3783371125 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa4c0cf3b0 con 0x7faa5c100e20 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 -- 172.21.15.19:0/682388001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 msgr2=0x5648163d6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 -- 172.21.15.19:0/682388001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 msgr2=0x5648163c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 -- 172.21.15.19:0/682388001 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648162bb3a0 con 0x5648163c95d0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa5bfff640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 0x7faa480a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa54009fd0 tx=0x7faa540125f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa53fff640 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faa4402a9a0 tx=0x7faa44005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5630c02e5c70b7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa51ffb640 1 -- 172.21.15.19:0/682388001 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44035030 con 0x5648163c95d0 2024-09-12T03:22:25.724 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa51ffb640 1 -- 172.21.15.19:0/682388001 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa44040070 con 0x5648163c95d0 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.712+0000 7faa51ffb640 1 -- 172.21.15.19:0/682388001 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa440318c0 con 0x5648163c95d0 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- 172.21.15.19:0/682388001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 msgr2=0x5648163c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7faa4402a9a0 tx=0x7faa44005e80 comp rx=0 tx=0).stop 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- 172.21.15.19:0/682388001 shutdown_connections 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163c9a10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 --2- 172.21.15.19:0/682388001 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- 172.21.15.19:0/682388001 >> 172.21.15.19:0/682388001 conn(0x564816392310 msgr2=0x5648163c5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- 172.21.15.19:0/682388001 shutdown_connections 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- 172.21.15.19:0/682388001 wait complete. 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.713+0000 7faa640f7740 1 -- start start 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 Processor -- start 2024-09-12T03:22:25.725 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5648163d7010 con 0x5648163c95d0 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5648162ea060 con 0x5648163cab40 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5648163d7190 con 0x5648163d4370 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa640f7740 10 client.0 _ll_register_callbacks cb 0x56481603d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47908/0 (socket says 172.21.15.19:47908) 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa5b7fe640 1 -- 172.21.15.19:0/151286137 learned_addr learned my addr 172.21.15.19:0/151286137 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.726 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa5bfff640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 -- 172.21.15.19:0/151286137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 msgr2=0x5648163d1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 -- 172.21.15.19:0/151286137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 msgr2=0x5648163d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa53fff640 1 -- 172.21.15.19:0/151286137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648160a03c0 con 0x5648163cab40 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.714+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa53fff640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d1040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa44006010 tx=0x7faa44004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec05dfab3b2aecaa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44035070 con 0x5648163cab40 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa44040070 con 0x5648163cab40 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa44005220 con 0x5648163cab40 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa44005420 con 0x5648163cab40 2024-09-12T03:22:25.727 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa640f7740 10 client.25565 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5648163e2ae0 con 0x5648163cab40 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa44040070 con 0x5648163cab40 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.715+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faa4404d020 con 0x5648163cab40 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 10 client.25565 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 10 client.25565 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 20 client.25565 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5648163e3e30 0x5648163e6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 4 client.25565 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56481633c860 con 0x5648163e3e30 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5648163e3e30 0x5648163e6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5648163e3e30 0x5648163e6220 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd6b4a1367eed7bf server_cookie=455befcc184a02b6 in_seq=0 out_seq=0 2024-09-12T03:22:25.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.716+0000 7faa3affd640 10 client.25565 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa3affd640 1 -- 172.21.15.19:0/151286137 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56481633c860 con 0x5648163e3e30 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa3affd640 10 client.25565 handle_command_reply: tid=0 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 client.25565 shutdown 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 msgr2=0x5648163d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d1040 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7faa44006010 tx=0x7faa44004530 comp rx=0 tx=0).stop 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 shutdown_connections 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5648163e3e30 0x5648163e6220 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163d4370 0x5648163d1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d1040 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa3affd640 0 client.25565 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 --2- 172.21.15.19:0/151286137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163c95d0 0x5648163d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 >> 172.21.15.19:0/151286137 conn(0x564816392310 msgr2=0x5648163cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 shutdown_connections 2024-09-12T03:22:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 1 -- 172.21.15.19:0/151286137 wait complete. 2024-09-12T03:22:25.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.719+0000 7faa640f7740 20 client.25565 trim_cache size 0 max 16384 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 Processor -- start 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 -- start start 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163c95d0 0x5648163e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1af660 con 0x5648163d4370 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b1260 con 0x5648163cab40 2024-09-12T03:22:25.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa640f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1afd30 con 0x5648163c95d0 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa5bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa5bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47916/0 (socket says 172.21.15.19:47916) 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa5bfff640 1 -- 172.21.15.19:0/1579417784 learned_addr learned my addr 172.21.15.19:0/1579417784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163c95d0 0x5648163e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.733+0000 7faa53fff640 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 -- 172.21.15.19:0/1579417784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163c95d0 msgr2=0x5648163e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163c95d0 0x5648163e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 -- 172.21.15.19:0/1579417784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 msgr2=0x5648163d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 -- 172.21.15.19:0/1579417784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648163d19c0 con 0x5648163d4370 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa5bfff640 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faa54009e30 tx=0x7faa54004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bab94c16cd217c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa517fa640 1 -- 172.21.15.19:0/1579417784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5402f070 con 0x5648163d4370 2024-09-12T03:22:25.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa517fa640 1 -- 172.21.15.19:0/1579417784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54012960 con 0x5648163d4370 2024-09-12T03:22:25.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.734+0000 7faa517fa640 1 -- 172.21.15.19:0/1579417784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54020400 con 0x5648163d4370 2024-09-12T03:22:25.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- 172.21.15.19:0/1579417784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 msgr2=0x5648163e9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7faa54009e30 tx=0x7faa54004ab0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- 172.21.15.19:0/1579417784 shutdown_connections 2024-09-12T03:22:25.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e9240 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163cab40 0x5648163d2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 --2- 172.21.15.19:0/1579417784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163c95d0 0x5648163e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- 172.21.15.19:0/1579417784 >> 172.21.15.19:0/1579417784 conn(0x564816392310 msgr2=0x5648163d28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- 172.21.15.19:0/1579417784 shutdown_connections 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- 172.21.15.19:0/1579417784 wait complete. 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 -- start start 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.735+0000 7faa640f7740 1 Processor -- start 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 0x5648163e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 0x5648163e6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b0ad0 con 0x5648163d4370 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1b0400 con 0x5648163c95d0 2024-09-12T03:22:25.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c1ad640 con 0x5648163cab40 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 0x5648163e6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 0x5648163e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47922/0 (socket says 172.21.15.19:47922) 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5bfff640 1 -- 172.21.15.19:0/2728696114 learned_addr learned my addr 172.21.15.19:0/2728696114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa640f7740 10 client.0 _ll_register_callbacks cb 0x7faa4c09cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 -- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 msgr2=0x5648163e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 0x5648163e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 -- 172.21.15.19:0/2728696114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 msgr2=0x5648163e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa53fff640 1 -- 172.21.15.19:0/2728696114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648163e79b0 con 0x5648163cab40 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 0x5648163e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.736+0000 7faa5bfff640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa53fff640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 0x5648163e6c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa44004c10 tx=0x7faa440050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a70ef80cf354f99 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa44035070 con 0x5648163cab40 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa54012960 con 0x5648163cab40 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa44031c60 con 0x5648163cab40 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faa44005850 con 0x5648163cab40 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 10 client.25591 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56481644e650 con 0x5648163cab40 2024-09-12T03:22:25.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa54012960 con 0x5648163cab40 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7faa4404d040 con 0x5648163cab40 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 10 client.25591 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 10 client.25591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.737+0000 7faa640f7740 20 client.25591 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x56481644ed10 0x564816451100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa640f7740 4 client.25591 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:25.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564816336d70 con 0x56481644ed10 2024-09-12T03:22:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x56481644ed10 0x564816451100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa5b7fe640 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x56481644ed10 0x564816451100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4c592520afc5b1ce server_cookie=347dd0a20e21edf8 in_seq=0 out_seq=0 2024-09-12T03:22:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa3affd640 10 client.25591 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:22:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa3affd640 1 -- 172.21.15.19:0/2728696114 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564816336d70 con 0x56481644ed10 2024-09-12T03:22:25.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa3affd640 10 client.25591 handle_command_reply: tid=0 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.738+0000 7faa640f7740 1 client.25591 shutdown 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 msgr2=0x5648163e6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 0x5648163e6c40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faa44004c10 tx=0x7faa440050b0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 shutdown_connections 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5648163d4370 0x5648163e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x56481644ed10 0x564816451100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5648163cab40 0x5648163e6c40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 --2- 172.21.15.19:0/2728696114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5648163c95d0 0x5648163e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 >> 172.21.15.19:0/2728696114 conn(0x564816392310 msgr2=0x5648163d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa3affd640 0 client.25591 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 shutdown_connections 2024-09-12T03:22:25.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 1 -- 172.21.15.19:0/2728696114 wait complete. 2024-09-12T03:22:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.739+0000 7faa640f7740 20 client.25591 trim_cache size 0 max 16384 2024-09-12T03:22:25.745 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:22:25.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 msgr2=0x7faa480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 0x7faa480a50c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7faa54009fd0 tx=0x7faa540125f0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 msgr2=0x7faa5c19a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7faa4c00e9f0 tx=0x7faa4c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 shutdown_connections 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa5c102ef0 0x7faa5c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa5c102570 0x7faa5c19ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa480a2c00 0x7faa480a50c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 --2- 172.21.15.19:0/3783371125 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa5c100e20 0x7faa5c19a6c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 >> 172.21.15.19:0/3783371125 conn(0x7faa5c0fc7d0 msgr2=0x7faa5c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 shutdown_connections 2024-09-12T03:22:25.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:25.740+0000 7faa61d1e640 1 -- 172.21.15.19:0/3783371125 wait complete. 2024-09-12T03:22:26.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:22:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:26 smithi135 ceph-mon[35474]: pgmap v439: 97 pgs: 97 active+clean; 405 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-09-12T03:22:27.339 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:26 smithi019 ceph-mon[30953]: pgmap v439: 97 pgs: 97 active+clean; 405 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-09-12T03:22:27.368 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '85516998', total seconds elapsed: 31.8 2024-09-12T03:22:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:26 smithi121 ceph-mon[34963]: pgmap v439: 97 pgs: 97 active+clean; 405 MiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 2.2 MiB/s wr, 133 op/s 2024-09-12T03:22:27.457 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.940+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/277831320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b441027a0 msgr2=0x7f0b441067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.940+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/277831320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b441027a0 0x7f0b441067f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009900 tx=0x7f0b3402a410 comp rx=0 tx=0).stop 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/277831320 shutdown_connections 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/277831320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b44106f20 0x7f0b4410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/277831320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b441027a0 0x7f0b441067f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/277831320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 0x7f0b441021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/277831320 >> 172.21.15.19:0/277831320 conn(0x7f0b440fd8f0 msgr2=0x7f0b440ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/277831320 shutdown_connections 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.941+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/277831320 wait complete. 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 Processor -- start 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 -- start start 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 0x7f0b44198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 0x7f0b441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44111fd0 con 0x7f0b44106f20 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44111e50 con 0x7f0b441027a0 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4c2b6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b44112150 con 0x7f0b44101df0 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4a853640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4a853640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47938/0 (socket says 172.21.15.19:47938) 2024-09-12T03:22:27.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 0x7f0b44198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.942+0000 7f0b4a853640 1 -- 172.21.15.19:0/1091819684 learned_addr learned my addr 172.21.15.19:0/1091819684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b49851640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 0x7f0b441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 -- 172.21.15.19:0/1091819684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 msgr2=0x7f0b44198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 0x7f0b44198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 -- 172.21.15.19:0/1091819684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 msgr2=0x7f0b441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 0x7f0b441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 -- 172.21.15.19:0/1091819684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4419d420 con 0x7f0b44106f20 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b49851640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 0x7f0b441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4a853640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00cce0 tx=0x7f0b3c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b3b7fe640 1 -- 172.21.15.19:0/1091819684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00f450 con 0x7f0b44106f20 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b3b7fe640 1 -- 172.21.15.19:0/1091819684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b3c00fad0 con 0x7f0b44106f20 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4419d710 con 0x7f0b44106f20 2024-09-12T03:22:27.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b3b7fe640 1 -- 172.21.15.19:0/1091819684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c018c10 con 0x7f0b44106f20 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.943+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b441a4ff0 con 0x7f0b44106f20 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.945+0000 7f0b3b7fe640 1 -- 172.21.15.19:0/1091819684 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b3c00f5f0 con 0x7f0b44106f20 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 Processor -- start 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 -- start start 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d6593a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65a4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603d64f92f0 con 0x5603d65935d0 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5603d64a0cf0 con 0x5603d6594b40 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603d64f52b0 con 0x5603d65a2780 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b3b7fe640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 0x7f0b300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.946+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d6593a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d6593a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47950/0 (socket says 172.21.15.19:47950) 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56322/0 (socket says 172.21.15.19:56322) 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b4a052640 1 -- 172.21.15.19:0/3816631655 learned_addr learned my addr 172.21.15.19:0/3816631655 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65a4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b4a853640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 0x7f0b300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b3b7fe640 1 -- 172.21.15.19:0/1091819684 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0b3c0cf660 con 0x7f0b44106f20 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 -- 172.21.15.19:0/3816631655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 msgr2=0x5603d65a4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65a4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 -- 172.21.15.19:0/3816631655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 msgr2=0x5603d6593a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d6593a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 -- 172.21.15.19:0/3816631655 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d64853a0 con 0x5603d6594b40 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b4a853640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 0x7f0b300a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c09bbd0 tx=0x7f0b3c066aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b49851640 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0b340096c0 tx=0x7f0b34000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a254131ff5cf75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b38dea640 1 -- 172.21.15.19:0/3816631655 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b34035040 con 0x5603d6594b40 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b38dea640 1 -- 172.21.15.19:0/3816631655 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b340046c0 con 0x5603d6594b40 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.947+0000 7f0b38dea640 1 -- 172.21.15.19:0/3816631655 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b340038a0 con 0x5603d6594b40 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 -- 172.21.15.19:0/3816631655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 msgr2=0x5603d6593090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0b340096c0 tx=0x7f0b34000bc0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 -- 172.21.15.19:0/3816631655 shutdown_connections 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65a4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d6593a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/3816631655 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d6594b40 0x5603d6593090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 -- 172.21.15.19:0/3816631655 >> 172.21.15.19:0/3816631655 conn(0x5603d655c310 msgr2=0x5603d658fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 -- 172.21.15.19:0/3816631655 shutdown_connections 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.948+0000 7f0b4e68f740 1 -- 172.21.15.19:0/3816631655 wait complete. 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 -- start start 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 Processor -- start 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d6594b40 0x5603d659f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d659fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5603d65a5420 con 0x5603d65935d0 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5603d64b4060 con 0x5603d65a2780 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5603d65a55a0 con 0x5603d6594b40 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4e68f740 10 client.0 _ll_register_callbacks cb 0x5603d62078c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47956/0 (socket says 172.21.15.19:47956) 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 -- 172.21.15.19:0/645104465 learned_addr learned my addr 172.21.15.19:0/645104465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b4a052640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d6594b40 0x5603d659f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 -- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d6594b40 msgr2=0x5603d659f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d6594b40 0x5603d659f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 -- 172.21.15.19:0/645104465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 msgr2=0x5603d659fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d659fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.949+0000 7f0b49851640 1 -- 172.21.15.19:0/645104465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d626a3c0 con 0x5603d65935d0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b49851640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0b3402ac00 tx=0x7f0b340095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b1f56f560a5f3b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3404e020 con 0x5603d65935d0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0b34005410 con 0x5603d65935d0 2024-09-12T03:22:27.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b340046c0 con 0x5603d65935d0 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b34035070 con 0x5603d65935d0 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 10 client.16065 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603d65b0ef0 con 0x5603d65935d0 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b340046c0 con 0x5603d65935d0 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0b3403a040 con 0x5603d65935d0 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 10 client.16065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.950+0000 7f0b4e68f740 20 client.16065 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d65b2240 0x5603d65b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b4e68f740 4 client.16065 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d6506860 con 0x5603d65b2240 2024-09-12T03:22:27.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b4a052640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d65b2240 0x5603d65b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b4a052640 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d65b2240 0x5603d65b4630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b563d8e686e03f9 server_cookie=21305ec0587848d7 in_seq=0 out_seq=0 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.951+0000 7f0b2dffb640 10 client.16065 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b2dffb640 1 -- 172.21.15.19:0/645104465 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603d6506860 con 0x5603d65b2240 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b2dffb640 10 client.16065 handle_command_reply: tid=0 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 client.16065 shutdown 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 msgr2=0x5603d65a2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0b3402ac00 tx=0x7f0b340095b0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 shutdown_connections 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d65b2240 0x5603d65b4630 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d659fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d6594b40 0x5603d659f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b2dffb640 0 client.16065 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:22:27.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/645104465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d65935d0 0x5603d65a2190 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 >> 172.21.15.19:0/645104465 conn(0x5603d655c310 msgr2=0x5603d6595960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 shutdown_connections 2024-09-12T03:22:27.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 1 -- 172.21.15.19:0/645104465 wait complete. 2024-09-12T03:22:27.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.952+0000 7f0b4e68f740 20 client.16065 trim_cache size 0 max 16384 2024-09-12T03:22:27.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 Processor -- start 2024-09-12T03:22:27.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 -- start start 2024-09-12T03:22:27.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65935d0 0x5603d65ae840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d65b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441ab2d0 con 0x5603d6594b40 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441aced0 con 0x5603d65a2780 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441ab9a0 con 0x5603d65935d0 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.966+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47970/0 (socket says 172.21.15.19:47970) 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 -- 172.21.15.19:0/1175328597 learned_addr learned my addr 172.21.15.19:0/1175328597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b49851640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65935d0 0x5603d65ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b49851640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d65b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 -- 172.21.15.19:0/1175328597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65935d0 msgr2=0x5603d65ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65935d0 0x5603d65ae840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 -- 172.21.15.19:0/1175328597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 msgr2=0x5603d65b7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d65b7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 -- 172.21.15.19:0/1175328597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d659d990 con 0x5603d6594b40 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b49851640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d65b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x5603d65a0a80 tx=0x7f0b40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a15693dbf43cc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b2ffff640 1 -- 172.21.15.19:0/1175328597 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40022070 con 0x5603d6594b40 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b2ffff640 1 -- 172.21.15.19:0/1175328597 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4001eb00 con 0x5603d6594b40 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.967+0000 7f0b2ffff640 1 -- 172.21.15.19:0/1175328597 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4001dad0 con 0x5603d6594b40 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1175328597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 msgr2=0x5603d659e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x5603d65a0a80 tx=0x7f0b40005f90 comp rx=0 tx=0).stop 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1175328597 shutdown_connections 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65a2780 0x5603d65b7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d659e420 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1175328597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65935d0 0x5603d65ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1175328597 >> 172.21.15.19:0/1175328597 conn(0x5603d655c310 msgr2=0x5603d659dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1175328597 shutdown_connections 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1175328597 wait complete. 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 -- start start 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.968+0000 7f0b4e68f740 1 Processor -- start 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65935d0 0x5603d65b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65b7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441ac740 con 0x5603d6594b40 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441ac070 con 0x5603d65935d0 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0b441a92b0 con 0x5603d65a2780 2024-09-12T03:22:27.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4e68f740 10 client.0 _ll_register_callbacks cb 0x7f0b40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b49851640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65b7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:47980/0 (socket says 172.21.15.19:47980) 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 -- 172.21.15.19:0/1051209859 learned_addr learned my addr 172.21.15.19:0/1051209859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b49851640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65935d0 0x5603d65b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 -- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 msgr2=0x5603d65b7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65b7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 -- 172.21.15.19:0/1051209859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65935d0 msgr2=0x5603d65b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65935d0 0x5603d65b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 -- 172.21.15.19:0/1051209859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d65b7ae0 con 0x5603d6594b40 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b4a052640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009ef0 tx=0x7f0b40009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af376d1702db0e91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:27.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.969+0000 7f0b2dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b40022040 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0b4001d380 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b4001eb00 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0b4001d5c0 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 10 client.16071 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5603d661c640 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4001eb00 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f0b40027450 con 0x5603d6594b40 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 10 client.16071 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 10 client.16071 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 20 client.16071 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d661cd30 0x5603d661f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:27.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 4 client.16071 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.970+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5603d6500d70 con 0x5603d661cd30 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b49851640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d661cd30 0x5603d661f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b49851640 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d661cd30 0x5603d661f150 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=78eb37b7d1830777 server_cookie=6eccc93e522ee177 in_seq=0 out_seq=0 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b2dffb640 10 client.16071 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b2dffb640 1 -- 172.21.15.19:0/1051209859 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5603d6500d70 con 0x5603d661cd30 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b2dffb640 10 client.16071 handle_command_reply: tid=0 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 client.16071 shutdown 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 msgr2=0x5603d65b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009ef0 tx=0x7f0b40009cb0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 shutdown_connections 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x5603d661cd30 0x5603d661f150 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5603d65a2780 0x5603d65b7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5603d6594b40 0x5603d65b5390 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b2dffb640 0 client.16071 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 --2- 172.21.15.19:0/1051209859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5603d65935d0 0x5603d65b4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 >> 172.21.15.19:0/1051209859 conn(0x5603d655c310 msgr2=0x5603d659e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 shutdown_connections 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.971+0000 7f0b4e68f740 1 -- 172.21.15.19:0/1051209859 wait complete. 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4e68f740 20 client.16071 trim_cache size 0 max 16384 2024-09-12T03:22:27.981 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 msgr2=0x7f0b300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 0x7f0b300a5040 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c09bbd0 tx=0x7f0b3c066aa0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 msgr2=0x7f0b4419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00cce0 tx=0x7f0b3c00a620 comp rx=0 tx=0).stop 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 shutdown_connections 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0b300a2b80 0x7f0b300a5040 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0b44106f20 0x7f0b4419cd40 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0b441027a0 0x7f0b441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 --2- 172.21.15.19:0/1091819684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0b44101df0 0x7f0b44198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 >> 172.21.15.19:0/1091819684 conn(0x7f0b440fd8f0 msgr2=0x7f0b440fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 shutdown_connections 2024-09-12T03:22:27.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:27.972+0000 7f0b4c2b6640 1 -- 172.21.15.19:0/1091819684 wait complete. 2024-09-12T03:22:29.086 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:28 smithi019 ceph-mon[30953]: pgmap v440: 97 pgs: 97 active+clean; 421 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.5 MiB/s wr, 190 op/s 2024-09-12T03:22:29.088 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:22:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:28 smithi135 ceph-mon[35474]: pgmap v440: 97 pgs: 97 active+clean; 421 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.5 MiB/s wr, 190 op/s 2024-09-12T03:22:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:28 smithi121 ceph-mon[34963]: pgmap v440: 97 pgs: 97 active+clean; 421 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 3.5 MiB/s wr, 190 op/s 2024-09-12T03:22:29.471 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 -- 172.21.15.19:0/1725840685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 msgr2=0x7f2474074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 --2- 172.21.15.19:0/1725840685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f2474074970 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2470009860 tx=0x7f247002a3d0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 -- 172.21.15.19:0/1725840685 shutdown_connections 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 --2- 172.21.15.19:0/1725840685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474112b90 0x7f2474114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 --2- 172.21.15.19:0/1725840685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474074eb0 0x7f2474112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 --2- 172.21.15.19:0/1725840685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f2474074970 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 -- 172.21.15.19:0/1725840685 >> 172.21.15.19:0/1725840685 conn(0x7f24740fd880 msgr2=0x7f24740ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 -- 172.21.15.19:0/1725840685 shutdown_connections 2024-09-12T03:22:30.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.020+0000 7f247c267640 1 -- 172.21.15.19:0/1725840685 wait complete. 2024-09-12T03:22:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 Processor -- start 2024-09-12T03:22:30.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 -- start start 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 0x7f24741a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 0x7f247419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24741185c0 con 0x7f2474112b90 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247a003640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247a003640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56368/0 (socket says 172.21.15.19:56368) 2024-09-12T03:22:30.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247a003640 1 -- 172.21.15.19:0/2771689771 learned_addr learned my addr 172.21.15.19:0/2771689771 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2474118440 con 0x7f2474074590 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.021+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2474118740 con 0x7f2474074eb0 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a804640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 0x7f247419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f2479802640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 0x7f24741a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a003640 1 -- 172.21.15.19:0/2771689771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 msgr2=0x7f24741a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a003640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 0x7f24741a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a003640 1 -- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 msgr2=0x7f247419e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a003640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 0x7f247419e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a003640 1 -- 172.21.15.19:0/2771689771 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247419efe0 con 0x7f2474074590 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.022+0000 7f247a804640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 0x7f247419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f2479802640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 0x7f24741a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f247a003640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2470009980 tx=0x7f24700309a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f246b7fe640 1 -- 172.21.15.19:0/2771689771 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470040070 con 0x7f2474074590 2024-09-12T03:22:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f246b7fe640 1 -- 172.21.15.19:0/2771689771 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2470005620 con 0x7f2474074590 2024-09-12T03:22:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247419f270 con 0x7f2474074590 2024-09-12T03:22:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f246b7fe640 1 -- 172.21.15.19:0/2771689771 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470004920 con 0x7f2474074590 2024-09-12T03:22:30.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.023+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24741ab5c0 con 0x7f2474074590 2024-09-12T03:22:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f246b7fe640 1 -- 172.21.15.19:0/2771689771 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2470043070 con 0x7f2474074590 2024-09-12T03:22:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 Processor -- start 2024-09-12T03:22:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 -- start start 2024-09-12T03:22:30.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39feb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39feb9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 0x55b39fec6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39fe1f2f0 con 0x55b39feb95d0 2024-09-12T03:22:30.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b39fdc6cf0 con 0x55b39fec4370 2024-09-12T03:22:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247e640740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b39fe1b2b0 con 0x55b39febab40 2024-09-12T03:22:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f2479802640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 0x55b39fec6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f2479802640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 0x55b39fec6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56374/0 (socket says 172.21.15.19:56374) 2024-09-12T03:22:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f2479802640 1 -- 172.21.15.19:0/2126438780 learned_addr learned my addr 172.21.15.19:0/2126438780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:30.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.027+0000 7f247a003640 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39feb9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39feb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f246b7fe640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 0x7f24600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 0x7f24600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f246b7fe640 1 -- 172.21.15.19:0/2771689771 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f24700efdd0 con 0x7f2474074590 2024-09-12T03:22:30.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 -- 172.21.15.19:0/2126438780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 msgr2=0x55b39fec6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 0x55b39fec6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 -- 172.21.15.19:0/2126438780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 msgr2=0x55b39feb9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39feb9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 -- 172.21.15.19:0/2126438780 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39fdab3a0 con 0x55b39febab40 2024-09-12T03:22:30.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.028+0000 7f247a804640 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39feb9090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2474076600 tx=0x7f246c00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbed074d3d46f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.029+0000 7f247a804640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 0x7f24600a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f246c008000 tx=0x7f246c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.029+0000 7f2468dea640 1 -- 172.21.15.19:0/2126438780 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c0137e0 con 0x55b39febab40 2024-09-12T03:22:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.029+0000 7f2468dea640 1 -- 172.21.15.19:0/2126438780 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c022030 con 0x55b39febab40 2024-09-12T03:22:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.029+0000 7f2468dea640 1 -- 172.21.15.19:0/2126438780 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c010ba0 con 0x55b39febab40 2024-09-12T03:22:30.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- 172.21.15.19:0/2126438780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 msgr2=0x55b39feb9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39feb9090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2474076600 tx=0x7f246c00f790 comp rx=0 tx=0).stop 2024-09-12T03:22:30.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- 172.21.15.19:0/2126438780 shutdown_connections 2024-09-12T03:22:30.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39fec4370 0x55b39fec6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39feb9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 --2- 172.21.15.19:0/2126438780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39feb9090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- 172.21.15.19:0/2126438780 >> 172.21.15.19:0/2126438780 conn(0x55b39fe82310 msgr2=0x55b39feb5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- 172.21.15.19:0/2126438780 shutdown_connections 2024-09-12T03:22:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- 172.21.15.19:0/2126438780 wait complete. 2024-09-12T03:22:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 -- start start 2024-09-12T03:22:30.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.030+0000 7f247e640740 1 Processor -- start 2024-09-12T03:22:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fec3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39fec1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b39fec7010 con 0x55b39fec4370 2024-09-12T03:22:30.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b39fdda060 con 0x55b39feb95d0 2024-09-12T03:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b39fec7190 con 0x55b39febab40 2024-09-12T03:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247e640740 10 client.0 _ll_register_callbacks cb 0x55b39fb2d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a003640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fec3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48020/0 (socket says 172.21.15.19:48020) 2024-09-12T03:22:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 -- 172.21.15.19:0/2751289478 learned_addr learned my addr 172.21.15.19:0/2751289478 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 -- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 msgr2=0x55b39fec1030 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39fec1030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 -- 172.21.15.19:0/2751289478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 msgr2=0x55b39fec3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fec3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.031+0000 7f247a804640 1 -- 172.21.15.19:0/2751289478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39fb903c0 con 0x55b39fec4370 2024-09-12T03:22:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f247a003640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fec3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f247a804640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f246c0104e0 tx=0x7f246c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3222571f1082c2af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:30.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c035060 con 0x55b39fec4370 2024-09-12T03:22:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f246c0137e0 con 0x55b39fec4370 2024-09-12T03:22:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246c022030 con 0x55b39fec4370 2024-09-12T03:22:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f246c0129a0 con 0x55b39fec4370 2024-09-12T03:22:30.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f247e640740 10 client.16083 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.032+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39fed2ab0 con 0x55b39fec4370 2024-09-12T03:22:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f246c010920 con 0x55b39fec4370 2024-09-12T03:22:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f246c027470 con 0x55b39fec4370 2024-09-12T03:22:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 10 client.16083 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:30.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 10 client.16083 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:22:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 20 client.16083 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39fed3f10 0x55b39fed6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 4 client.16083 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.033+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f246c022030 con 0x55b39fed3f10 2024-09-12T03:22:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.034+0000 7f247a003640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39fed3f10 0x55b39fed6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.034+0000 7f247a003640 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39fed3f10 0x55b39fed6300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c96ac0a8597507a7 server_cookie=d009c0c9207bfafc in_seq=0 out_seq=0 2024-09-12T03:22:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.034+0000 7f245dffb640 10 client.16083 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:22:30.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f245dffb640 1 -- 172.21.15.19:0/2751289478 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f246c022030 con 0x55b39fed3f10 2024-09-12T03:22:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f245dffb640 10 client.16083 handle_command_reply: tid=0 2024-09-12T03:22:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 client.16083 shutdown 2024-09-12T03:22:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 msgr2=0x55b39fec1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f246c0104e0 tx=0x7f246c005fa0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 shutdown_connections 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39fed3f10 0x55b39fed6300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39fec1690 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39fec1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f245dffb640 0 client.16083 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 --2- 172.21.15.19:0/2751289478 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fec3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 >> 172.21.15.19:0/2751289478 conn(0x55b39fe82310 msgr2=0x55b39febb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 shutdown_connections 2024-09-12T03:22:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 1 -- 172.21.15.19:0/2751289478 wait complete. 2024-09-12T03:22:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.036+0000 7f247e640740 20 client.16083 trim_cache size 0 max 16384 2024-09-12T03:22:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 Processor -- start 2024-09-12T03:22:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 -- start start 2024-09-12T03:22:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 0x55b39febc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39fec4370 0x55b39febc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b3ba0 con 0x55b39feb95d0 2024-09-12T03:22:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b57a0 con 0x55b39febab40 2024-09-12T03:22:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.052+0000 7f247e640740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b4270 con 0x55b39fec4370 2024-09-12T03:22:30.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48026/0 (socket says 172.21.15.19:48026) 2024-09-12T03:22:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 -- 172.21.15.19:0/4219341816 learned_addr learned my addr 172.21.15.19:0/4219341816 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f2479802640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 0x55b39febc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a804640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39fec4370 0x55b39febc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 -- 172.21.15.19:0/4219341816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39fec4370 msgr2=0x55b39febc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39fec4370 0x55b39febc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 -- 172.21.15.19:0/4219341816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 msgr2=0x55b39febc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 0x55b39febc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 -- 172.21.15.19:0/4219341816 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b39fed68c0 con 0x55b39feb95d0 2024-09-12T03:22:30.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f2479802640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 0x55b39febc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f247a003640 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2470009830 tx=0x7f24700f22e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa1fc9089129d9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.053+0000 7f245ffff640 1 -- 172.21.15.19:0/4219341816 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247004a040 con 0x55b39feb95d0 2024-09-12T03:22:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.054+0000 7f245ffff640 1 -- 172.21.15.19:0/4219341816 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24700bcd40 con 0x55b39feb95d0 2024-09-12T03:22:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.054+0000 7f245ffff640 1 -- 172.21.15.19:0/4219341816 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24700c52b0 con 0x55b39feb95d0 2024-09-12T03:22:30.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.054+0000 7f247e640740 1 -- 172.21.15.19:0/4219341816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 msgr2=0x55b39fed02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.054+0000 7f247e640740 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2470009830 tx=0x7f24700f22e0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- 172.21.15.19:0/4219341816 shutdown_connections 2024-09-12T03:22:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39fec4370 0x55b39febc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39febab40 0x55b39febc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- 172.21.15.19:0/4219341816 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39feb95d0 0x55b39fed02b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- 172.21.15.19:0/4219341816 >> 172.21.15.19:0/4219341816 conn(0x55b39fe82310 msgr2=0x55b39fec30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- 172.21.15.19:0/4219341816 shutdown_connections 2024-09-12T03:22:30.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- 172.21.15.19:0/4219341816 wait complete. 2024-09-12T03:22:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- start start 2024-09-12T03:22:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 Processor -- start 2024-09-12T03:22:30.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fecca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39febdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b5010 con 0x55b39fec4370 2024-09-12T03:22:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b4940 con 0x55b39feb95d0 2024-09-12T03:22:30.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f24741b1b80 con 0x55b39febab40 2024-09-12T03:22:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.055+0000 7f247e640740 10 client.0 _ll_register_callbacks cb 0x7f24700b8960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a804640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:48032/0 (socket says 172.21.15.19:48032) 2024-09-12T03:22:30.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a804640 1 -- 172.21.15.19:0/2186134187 learned_addr learned my addr 172.21.15.19:0/2186134187 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fecca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f2479802640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39febdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 -- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 msgr2=0x55b39febdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39febdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 -- 172.21.15.19:0/2186134187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 msgr2=0x55b39febe150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 -- 172.21.15.19:0/2186134187 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b39febe790 con 0x55b39feb95d0 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a804640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247a003640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fecca10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b39fed2a80 tx=0x7f24700c0b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc050f1df0368d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f245dffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:22:30.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247004a040 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f24700ba940 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.056+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24700bcd40 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24700bbfa0 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 10 client.25598 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b39ff3e8f0 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24700bcd40 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f2470031600 con 0x55b39feb95d0 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 10 client.25598 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 10 client.25598 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 20 client.25598 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39ff3ee50 0x55b39ff41270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:30.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 4 client.25598 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.057+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b39fe2c860 con 0x55b39ff3ee50 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.058+0000 7f2479802640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39ff3ee50 0x55b39ff41270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.058+0000 7f2479802640 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39ff3ee50 0x55b39ff41270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b708d8bf6f7b6ecb server_cookie=52fc6f62a9f8f5b1 in_seq=0 out_seq=0 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.058+0000 7f245dffb640 10 client.25598 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f245dffb640 1 -- 172.21.15.19:0/2186134187 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b39fe2c860 con 0x55b39ff3ee50 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f245dffb640 10 client.25598 handle_command_reply: tid=0 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 client.25598 shutdown 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 msgr2=0x55b39fecca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fecca10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55b39fed2a80 tx=0x7f24700c0b60 comp rx=0 tx=0).stop 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 shutdown_connections 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55b39ff3ee50 0x55b39ff41270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f245dffb640 0 client.25598 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:22:30.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b39fec4370 0x55b39febe150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b39febab40 0x55b39febdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 --2- 172.21.15.19:0/2186134187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b39feb95d0 0x55b39fecca10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 >> 172.21.15.19:0/2186134187 conn(0x55b39fe82310 msgr2=0x55b39febf830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 shutdown_connections 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 1 -- 172.21.15.19:0/2186134187 wait complete. 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.059+0000 7f247e640740 20 client.25598 trim_cache size 0 max 16384 2024-09-12T03:22:30.082 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 msgr2=0x7f24600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 0x7f24600a5040 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f246c008000 tx=0x7f246c01f000 comp rx=0 tx=0).stop 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 msgr2=0x7f24741a46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2470009980 tx=0x7f24700309a0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 shutdown_connections 2024-09-12T03:22:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f24600a2b80 0x7f24600a5040 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2474112b90 0x7f247419e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2474074eb0 0x7f24741a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 --2- 172.21.15.19:0/2771689771 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2474074590 0x7f24741a46f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.060+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 >> 172.21.15.19:0/2771689771 conn(0x7f24740fd880 msgr2=0x7f24740fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.061+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 shutdown_connections 2024-09-12T03:22:30.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:30.061+0000 7f247c267640 1 -- 172.21.15.19:0/2771689771 wait complete. 2024-09-12T03:22:31.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:22:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:30 smithi135 ceph-mon[35474]: pgmap v441: 97 pgs: 97 active+clean; 432 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-09-12T03:22:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:30 smithi121 ceph-mon[34963]: pgmap v441: 97 pgs: 97 active+clean; 432 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-09-12T03:22:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:30 smithi019 ceph-mon[30953]: pgmap v441: 97 pgs: 97 active+clean; 432 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 360 KiB/s rd, 4.4 MiB/s wr, 222 op/s 2024-09-12T03:22:31.487 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:32.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 -- 172.21.15.19:0/1108609251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc1082a0 msgr2=0x7fe4dc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1108609251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc10ec40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009cd0 tx=0x7fe4d402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 -- 172.21.15.19:0/1108609251 shutdown_connections 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1108609251 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc10ec40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1108609251 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1036e0 0x7fe4dc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1108609251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc102d30 0x7fe4dc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.038+0000 7fe4e380d640 1 -- 172.21.15.19:0/1108609251 >> 172.21.15.19:0/1108609251 conn(0x7fe4dc0fc7d0 msgr2=0x7fe4dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.039+0000 7fe4e380d640 1 -- 172.21.15.19:0/1108609251 shutdown_connections 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.039+0000 7fe4e380d640 1 -- 172.21.15.19:0/1108609251 wait complete. 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.039+0000 7fe4e380d640 1 Processor -- start 2024-09-12T03:22:32.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.039+0000 7fe4e380d640 1 -- start start 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc1036e0 0x7fe4dc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1120a0 con 0x7fe4dc102d30 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc111f20 con 0x7fe4dc1036e0 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e380d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc112220 con 0x7fe4dc1082a0 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50498/0 (socket says 172.21.15.19:50498) 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1528525074 learned_addr learned my addr 172.21.15.19:0/1528525074 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc1036e0 0x7fe4dc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1528525074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1082a0 msgr2=0x7fe4dc19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1528525074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc1036e0 msgr2=0x7fe4dc198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc1036e0 0x7fe4dc198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.040+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1528525074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4dc19d510 con 0x7fe4dc102d30 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9f0 tx=0x7fe4d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4d27fc640 1 -- 172.21.15.19:0/1528525074 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d800cd80 con 0x7fe4dc102d30 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4d27fc640 1 -- 172.21.15.19:0/1528525074 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4d8010910 con 0x7fe4dc102d30 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4dc19d800 con 0x7fe4dc102d30 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4d27fc640 1 -- 172.21.15.19:0/1528525074 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80188f0 con 0x7fe4dc102d30 2024-09-12T03:22:32.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.041+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4dc1a50e0 con 0x7fe4dc102d30 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.043+0000 7fe4d27fc640 1 -- 172.21.15.19:0/1528525074 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe4d8010ac0 con 0x7fe4dc102d30 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 Processor -- start 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 -- start start 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 0x55850c52b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c52ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c53cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55850c4912f0 con 0x55850c52b5d0 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55850c438cf0 con 0x55850c52cb40 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.044+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55850c48d2b0 con 0x55850c53a780 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c52ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e1daa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 0x55850c52b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4d27fc640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 0x7fe4c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c52ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50510/0 (socket says 172.21.15.19:50510) 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1030024286 learned_addr learned my addr 172.21.15.19:0/1030024286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c53cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e1daa640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 0x55850c52b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34554/0 (socket says 172.21.15.19:34554) 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 0x7fe4c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1030024286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 msgr2=0x55850c52b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 0x55850c52b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1030024286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 msgr2=0x55850c52ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c52ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1030024286 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850c41d3a0 con 0x55850c53a780 2024-09-12T03:22:32.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4d27fc640 1 -- 172.21.15.19:0/1528525074 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d80cf420 con 0x7fe4dc102d30 2024-09-12T03:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.045+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c53cb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc06a370 tx=0x7fe4cc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d473d3c6af015dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 0x7fe4c80a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4004ab0 tx=0x7fe4d402a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4c77fe640 1 -- 172.21.15.19:0/1030024286 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004050 con 0x55850c53a780 2024-09-12T03:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4c77fe640 1 -- 172.21.15.19:0/1030024286 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc005500 con 0x55850c53a780 2024-09-12T03:22:32.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4c77fe640 1 -- 172.21.15.19:0/1030024286 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004d90 con 0x55850c53a780 2024-09-12T03:22:32.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1030024286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 msgr2=0x55850c53cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.046+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c53cb70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc06a370 tx=0x7fe4cc011810 comp rx=0 tx=0).stop 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1030024286 shutdown_connections 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c53cb70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c52ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1030024286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52cb40 0x55850c52b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1030024286 >> 172.21.15.19:0/1030024286 conn(0x55850c4f4310 msgr2=0x55850c527c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1030024286 shutdown_connections 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1030024286 wait complete. 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 -- start start 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.047+0000 7fe4e5be6740 1 Processor -- start 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c53a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c5379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55850c53d480 con 0x55850c52cb40 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55850c44c060 con 0x55850c52b5d0 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55850c53d600 con 0x55850c53a780 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e5be6740 10 client.0 _ll_register_callbacks cb 0x55850c19f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e0da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c53a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e0da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50524/0 (socket says 172.21.15.19:50524) 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1436387105 learned_addr learned my addr 172.21.15.19:0/1436387105 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c5379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1436387105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 msgr2=0x55850c5379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c5379e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1436387105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 msgr2=0x55850c5373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1436387105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850c2023c0 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c5379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.048+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c53a140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80986b0 tx=0x7fe4d809b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2695502f8f32271b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d809b330 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d8010430 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc005500 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4d80a48a0 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4e5be6740 10 client.25610 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55850c548f40 con 0x55850c52b5d0 2024-09-12T03:22:32.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.049+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc005500 con 0x55850c52b5d0 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe4d809aba0 con 0x55850c52b5d0 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 10 client.25610 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 10 client.25610 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 20 client.25610 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c54a450 0x55850c54c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 4 client.25610 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55850c49e860 con 0x55850c54a450 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c54a450 0x55850c54c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c54a450 0x55850c54c8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b642fc06f660f5d server_cookie=154e1deda1e586a8 in_seq=0 out_seq=0 2024-09-12T03:22:32.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.050+0000 7fe4c4ff9640 10 client.25610 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1436387105 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55850c49e860 con 0x55850c54a450 2024-09-12T03:22:32.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4c4ff9640 10 client.25610 handle_command_reply: tid=0 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 client.25610 shutdown 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 msgr2=0x55850c53a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c53a140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80986b0 tx=0x7fe4d809b910 comp rx=0 tx=0).stop 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 shutdown_connections 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c54a450 0x55850c54c8a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c5379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c5373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1436387105 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c53a140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.051+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 >> 172.21.15.19:0/1436387105 conn(0x55850c4f4310 msgr2=0x55850c52d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.052+0000 7fe4c4ff9640 0 client.25610 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.052+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 shutdown_connections 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.052+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1436387105 wait complete. 2024-09-12T03:22:32.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.052+0000 7fe4e5be6740 20 client.25610 trim_cache size 0 max 16384 2024-09-12T03:22:32.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 Processor -- start 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 -- start start 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c546720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c552170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ab3e0 con 0x55850c52cb40 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1acfe0 con 0x55850c52b5d0 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.065+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1abab0 con 0x55850c53a780 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50532/0 (socket says 172.21.15.19:50532) 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1533867699 learned_addr learned my addr 172.21.15.19:0/1533867699 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c546720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c552170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1533867699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 msgr2=0x55850c552170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c552170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1533867699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 msgr2=0x55850c546720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c546720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 -- 172.21.15.19:0/1533867699 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55850c53a4b0 con 0x55850c52cb40 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005b90 tx=0x7fe4cc004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7a2ecda1481222 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.066+0000 7fe4c6ffd640 1 -- 172.21.15.19:0/1533867699 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc031070 con 0x55850c52cb40 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4c6ffd640 1 -- 172.21.15.19:0/1533867699 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc009d50 con 0x55850c52cb40 2024-09-12T03:22:32.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4c6ffd640 1 -- 172.21.15.19:0/1533867699 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc01d040 con 0x55850c52cb40 2024-09-12T03:22:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1533867699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 msgr2=0x55850c539360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005b90 tx=0x7fe4cc004540 comp rx=0 tx=0).stop 2024-09-12T03:22:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1533867699 shutdown_connections 2024-09-12T03:22:32.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c53a780 0x55850c552170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52cb40 0x55850c539360 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1533867699 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c52b5d0 0x55850c546720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1533867699 >> 172.21.15.19:0/1533867699 conn(0x55850c4f4310 msgr2=0x55850c538c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1533867699 shutdown_connections 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.067+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1533867699 wait complete. 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 -- start start 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 Processor -- start 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c52cb40 0x55850c53ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 0x55850c53efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac850 con 0x55850c52b5d0 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1ac180 con 0x55850c53a780 2024-09-12T03:22:32.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4dc1a93e0 con 0x55850c52cb40 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.068+0000 7fe4e5be6740 10 client.0 _ll_register_callbacks cb 0x7fe4d800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50544/0 (socket says 172.21.15.19:50544) 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1231301452 learned_addr learned my addr 172.21.15.19:0/1231301452 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 0x55850c53efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c52cb40 0x55850c53ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1231301452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c52cb40 msgr2=0x55850c53ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c52cb40 0x55850c53ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 msgr2=0x55850c53efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 0x55850c53efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 -- 172.21.15.19:0/1231301452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55850c5b4140 con 0x55850c52b5d0 2024-09-12T03:22:32.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e1daa640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 0x55850c53efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e15a9640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80a4370 tx=0x7fe4d809b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9aca908e692f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d80a4af0 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4d80a4c90 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.069+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4cc009d50 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe4d8097a30 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 10 client.16107 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55850c5b4140 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4cc009d50 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe4d80862e0 con 0x55850c52b5d0 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 10 client.16107 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 10 client.16107 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:22:32.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 20 client.16107 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c5b5770 0x55850c5b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 4 client.16107 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55850c54cc30 con 0x55850c5b5770 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.070+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c5b5770 0x55850c5b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e0da8640 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c5b5770 0x55850c5b7b90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0fc555908a09d95 server_cookie=c08e6f9e0c4d73ad in_seq=0 out_seq=0 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4c4ff9640 10 client.16107 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4c4ff9640 1 -- 172.21.15.19:0/1231301452 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55850c54cc30 con 0x55850c5b5770 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4c4ff9640 10 client.16107 handle_command_reply: tid=0 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 client.16107 shutdown 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 msgr2=0x55850c53e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80a4370 tx=0x7fe4d809b3e0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 shutdown_connections 2024-09-12T03:22:32.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55850c5b5770 0x55850c5b7b90 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55850c53a780 0x55850c53efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4c4ff9640 0 client.16107 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55850c52cb40 0x55850c53ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 --2- 172.21.15.19:0/1231301452 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55850c52b5d0 0x55850c53e550 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.071+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 >> 172.21.15.19:0/1231301452 conn(0x55850c4f4310 msgr2=0x55850c539470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.072+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 shutdown_connections 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.072+0000 7fe4e5be6740 1 -- 172.21.15.19:0/1231301452 wait complete. 2024-09-12T03:22:32.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.072+0000 7fe4e5be6740 20 client.16107 trim_cache size 0 max 16384 2024-09-12T03:22:32.078 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.072+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 msgr2=0x7fe4c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.072+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 0x7fe4c80a5040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4004ab0 tx=0x7fe4d402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 msgr2=0x7fe4dc198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800e9f0 tx=0x7fe4d800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 shutdown_connections 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe4dc1082a0 0x7fe4dc19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe4dc1036e0 0x7fe4dc198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4c80a2b80 0x7fe4c80a5040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 --2- 172.21.15.19:0/1528525074 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe4dc102d30 0x7fe4dc198560 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 >> 172.21.15.19:0/1528525074 conn(0x7fe4dc0fc7d0 msgr2=0x7fe4dc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 shutdown_connections 2024-09-12T03:22:32.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:32.073+0000 7fe4e380d640 1 -- 172.21.15.19:0/1528525074 wait complete. 2024-09-12T03:22:33.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:22:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:33 smithi135 ceph-mon[35474]: pgmap v442: 97 pgs: 97 active+clean; 445 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-09-12T03:22:33.307 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:33 smithi121 ceph-mon[34963]: pgmap v442: 97 pgs: 97 active+clean; 445 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-09-12T03:22:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:33 smithi019 ceph-mon[30953]: pgmap v442: 97 pgs: 97 active+clean; 445 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 533 KiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-09-12T03:22:33.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.963+0000 7fcd680c7640 1 -- 172.21.15.19:0/897473943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60109830 msgr2=0x7fcd60109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.964+0000 7fcd680c7640 1 --2- 172.21.15.19:0/897473943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60109830 0x7fcd60109c90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009860 tx=0x7fcd5c02a380 comp rx=0 tx=0).stop 2024-09-12T03:22:33.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.965+0000 7fcd680c7640 1 -- 172.21.15.19:0/897473943 shutdown_connections 2024-09-12T03:22:33.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.965+0000 7fcd680c7640 1 --2- 172.21.15.19:0/897473943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd6011b140 0x7fcd6011d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.965+0000 7fcd680c7640 1 --2- 172.21.15.19:0/897473943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd60109830 0x7fcd60109c90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.965+0000 7fcd680c7640 1 --2- 172.21.15.19:0/897473943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109450 0x7fcd60110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.965+0000 7fcd680c7640 1 -- 172.21.15.19:0/897473943 >> 172.21.15.19:0/897473943 conn(0x7fcd6006e140 msgr2=0x7fcd6006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:33.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.969+0000 7fcd680c7640 1 -- 172.21.15.19:0/897473943 shutdown_connections 2024-09-12T03:22:33.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.970+0000 7fcd680c7640 1 -- 172.21.15.19:0/897473943 wait complete. 2024-09-12T03:22:33.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.972+0000 7fcd680c7640 1 Processor -- start 2024-09-12T03:22:33.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.980+0000 7fcd680c7640 1 -- start start 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd60109450 0x7fcd601a72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 0x7fcd601a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 0x7fcd601abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60120df0 con 0x7fcd6011b140 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60120c70 con 0x7fcd60109830 2024-09-12T03:22:33.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.981+0000 7fcd680c7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60120f70 con 0x7fcd60109450 2024-09-12T03:22:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.983+0000 7fcd65662640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 0x7fcd601a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.983+0000 7fcd65e63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd60109450 0x7fcd601a72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.983+0000 7fcd65662640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 0x7fcd601a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34602/0 (socket says 172.21.15.19:34602) 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.983+0000 7fcd65662640 1 -- 172.21.15.19:0/180679127 learned_addr learned my addr 172.21.15.19:0/180679127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.983+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 0x7fcd601abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.984+0000 7fcd66664640 1 -- 172.21.15.19:0/180679127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd60109450 msgr2=0x7fcd601a72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.984+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd60109450 0x7fcd601a72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.984+0000 7fcd66664640 1 -- 172.21.15.19:0/180679127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 msgr2=0x7fcd601a77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.984+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 0x7fcd601a77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:33.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.984+0000 7fcd66664640 1 -- 172.21.15.19:0/180679127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd601ac260 con 0x7fcd6011b140 2024-09-12T03:22:33.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.986+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 0x7fcd601abb80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005fd0 tx=0x7fcd5000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.987+0000 7fcd56ffd640 1 -- 172.21.15.19:0/180679127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd50017070 con 0x7fcd6011b140 2024-09-12T03:22:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.987+0000 7fcd56ffd640 1 -- 172.21.15.19:0/180679127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd500047a0 con 0x7fcd6011b140 2024-09-12T03:22:33.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.987+0000 7fcd56ffd640 1 -- 172.21.15.19:0/180679127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd50017070 con 0x7fcd6011b140 2024-09-12T03:22:33.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.988+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd601ac550 con 0x7fcd6011b140 2024-09-12T03:22:33.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.992+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd601b3e30 con 0x7fcd6011b140 2024-09-12T03:22:33.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.997+0000 7fcd6a4a0740 1 Processor -- start 2024-09-12T03:22:33.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.998+0000 7fcd56ffd640 1 -- 172.21.15.19:0/180679127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd50004950 con 0x7fcd6011b140 2024-09-12T03:22:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 -- start start 2024-09-12T03:22:34.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 0x5654b76ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f7370 0x5654b76f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5654b76522f0 con 0x5654b76edb40 2024-09-12T03:22:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5654b75f9cf0 con 0x5654b76f7370 2024-09-12T03:22:34.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:33.999+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5654b764e2b0 con 0x5654b76ec5d0 2024-09-12T03:22:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.000+0000 7fcd65662640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 0x5654b76ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.000+0000 7fcd65e63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.000+0000 7fcd65e63640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76eca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48592/0 (socket says 172.21.15.19:48592) 2024-09-12T03:22:34.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.000+0000 7fcd65e63640 1 -- 172.21.15.19:0/3244416201 learned_addr learned my addr 172.21.15.19:0/3244416201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.001+0000 7fcd56ffd640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 0x7fcd4c0a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.001+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 0x7fcd4c0a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.002+0000 7fcd66664640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 0x7fcd4c0a5480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cf000 tx=0x7fcd500ced10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.003+0000 7fcd56ffd640 1 -- 172.21.15.19:0/180679127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcd50005020 con 0x7fcd6011b140 2024-09-12T03:22:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.008+0000 7fcd65662640 1 -- 172.21.15.19:0/3244416201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 msgr2=0x5654b76eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.008+0000 7fcd65662640 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.008+0000 7fcd65662640 1 -- 172.21.15.19:0/3244416201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f7370 msgr2=0x5654b76f9760 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:22:34.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.008+0000 7fcd65662640 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f7370 0x5654b76f9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.008+0000 7fcd65662640 1 -- 172.21.15.19:0/3244416201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b75de3a0 con 0x5654b76edb40 2024-09-12T03:22:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.009+0000 7fcd65662640 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 0x5654b76ec090 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c02a970 tx=0x7fcd5c030500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3d88a872b90cf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.009+0000 7fcd4bfff640 1 -- 172.21.15.19:0/3244416201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c037040 con 0x5654b76edb40 2024-09-12T03:22:34.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.010+0000 7fcd4bfff640 1 -- 172.21.15.19:0/3244416201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c0305b0 con 0x5654b76edb40 2024-09-12T03:22:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd4bfff640 1 -- 172.21.15.19:0/3244416201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c0053b0 con 0x5654b76edb40 2024-09-12T03:22:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3244416201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 msgr2=0x5654b76ec090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 0x5654b76ec090 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c02a970 tx=0x7fcd5c030500 comp rx=0 tx=0).stop 2024-09-12T03:22:34.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3244416201 shutdown_connections 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f7370 0x5654b76f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3244416201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76edb40 0x5654b76ec090 secure :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c02a970 tx=0x7fcd5c030500 comp rx=0 tx=0).stop 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.011+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3244416201 >> 172.21.15.19:0/3244416201 conn(0x5654b76b5310 msgr2=0x5654b76e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.012+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3244416201 shutdown_connections 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.012+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3244416201 wait complete. 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.012+0000 7fcd6a4a0740 1 -- start start 2024-09-12T03:22:34.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.012+0000 7fcd6a4a0740 1 Processor -- start 2024-09-12T03:22:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76ec5d0 0x5654b76f4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76f4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5654b770b070 con 0x5654b76f7370 2024-09-12T03:22:34.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5654b760d060 con 0x5654b76ec5d0 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5654b770b1f0 con 0x5654b76f4c90 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.013+0000 7fcd6a4a0740 10 client.0 _ll_register_callbacks cb 0x5654b73608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48604/0 (socket says 172.21.15.19:48604) 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 -- 172.21.15.19:0/671982371 learned_addr learned my addr 172.21.15.19:0/671982371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:34.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76ec5d0 0x5654b76f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 -- 172.21.15.19:0/671982371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76ec5d0 msgr2=0x5654b76f4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76ec5d0 0x5654b76f4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 -- 172.21.15.19:0/671982371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 msgr2=0x5654b76f4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65e63640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76f4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76f4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.014+0000 7fcd65662640 1 -- 172.21.15.19:0/671982371 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b73c33c0 con 0x5654b76f4c90 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.015+0000 7fcd65e63640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.015+0000 7fcd65662640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c004000 tx=0x7fcd5c0303d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d77f62c07a65ea8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.015+0000 7fcd497fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.015+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c037040 con 0x5654b76f4c90 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.015+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcd5c004030 con 0x5654b76f4c90 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.016+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd5c0305b0 con 0x5654b76f4c90 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.016+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd5c004230 con 0x5654b76f4c90 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.016+0000 7fcd6a4a0740 10 client.25654 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:34.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.016+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5654b770b070 con 0x5654b76f4c90 2024-09-12T03:22:34.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.016+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c0305b0 con 0x5654b76f4c90 2024-09-12T03:22:34.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcd5c054020 con 0x5654b76f4c90 2024-09-12T03:22:34.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 10 client.25654 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:34.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:34.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 20 client.25654 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b77182a0 0x5654b771a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 4 client.25654 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b765f860 con 0x5654b77182a0 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.017+0000 7fcd65e63640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b77182a0 0x5654b771a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.018+0000 7fcd497fa640 10 client.25654 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:34.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.018+0000 7fcd65e63640 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b77182a0 0x5654b771a690 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b3ff04838a6d6b server_cookie=f9f55ad3ce6a8fd0 in_seq=0 out_seq=0 2024-09-12T03:22:34.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd497fa640 1 -- 172.21.15.19:0/671982371 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654b765f860 con 0x5654b77182a0 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd497fa640 10 client.25654 handle_command_reply: tid=0 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 client.25654 shutdown 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 msgr2=0x5654b7714570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c004000 tx=0x7fcd5c0303d0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 shutdown_connections 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b7714570 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76f4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b77182a0 0x5654b771a690 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/671982371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76ec5d0 0x5654b76f4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 >> 172.21.15.19:0/671982371 conn(0x5654b76b5310 msgr2=0x5654b76ee960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.021+0000 7fcd497fa640 0 client.25654 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:34.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.022+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 shutdown_connections 2024-09-12T03:22:34.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.022+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/671982371 wait complete. 2024-09-12T03:22:34.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.022+0000 7fcd6a4a0740 20 client.25654 trim_cache size 0 max 16384 2024-09-12T03:22:34.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 Processor -- start 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 -- start start 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76e8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f4c90 0x5654b76ee7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60118dd0 con 0x5654b76f7370 2024-09-12T03:22:34.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b58a0 con 0x5654b76f4c90 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.053+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b4370 con 0x5654b76ec5d0 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50584/0 (socket says 172.21.15.19:50584) 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 -- 172.21.15.19:0/3039130502 learned_addr learned my addr 172.21.15.19:0/3039130502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65e63640 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f4c90 0x5654b76ee7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76e8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 -- 172.21.15.19:0/3039130502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 msgr2=0x5654b76e8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76e8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 -- 172.21.15.19:0/3039130502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f4c90 msgr2=0x5654b76ee7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f4c90 0x5654b76ee7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.054+0000 7fcd65662640 1 -- 172.21.15.19:0/3039130502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b76e7310 con 0x5654b76f7370 2024-09-12T03:22:34.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.055+0000 7fcd65662640 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0050a0 tx=0x7fcd5c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1631acb3d5bc6073 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.056+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/3039130502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c037070 con 0x5654b76f7370 2024-09-12T03:22:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.057+0000 7fcd4b7fe640 1 -- 172.21.15.19:0/3039130502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5c04c040 con 0x5654b76f7370 2024-09-12T03:22:34.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3039130502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 msgr2=0x5654b76eed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0050a0 tx=0x7fcd5c005a90 comp rx=0 tx=0).stop 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3039130502 shutdown_connections 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76f7370 0x5654b76eed20 secure :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c0050a0 tx=0x7fcd5c005a90 comp rx=0 tx=0).stop 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b76f4c90 0x5654b76ee7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/3039130502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76ec5d0 0x5654b76e8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.058+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3039130502 >> 172.21.15.19:0/3039130502 conn(0x5654b76b5310 msgr2=0x5654b76ee0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.060+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3039130502 shutdown_connections 2024-09-12T03:22:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.062+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/3039130502 wait complete. 2024-09-12T03:22:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.062+0000 7fcd6a4a0740 1 -- start start 2024-09-12T03:22:34.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.063+0000 7fcd6a4a0740 1 Processor -- start 2024-09-12T03:22:34.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b770c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b770cec0 0x5654b7781d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b5110 con 0x5654b76ec5d0 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcd601b4a40 con 0x5654b770cec0 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcd60116db0 con 0x5654b76f4c90 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd6a4a0740 10 client.0 _ll_register_callbacks cb 0x7fcd58009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd65662640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b770cec0 0x5654b7781d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd65662640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd65662640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50598/0 (socket says 172.21.15.19:50598) 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd65662640 1 -- 172.21.15.19:0/1862353278 learned_addr learned my addr 172.21.15.19:0/1862353278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:34.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.064+0000 7fcd65e63640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b770c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 -- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 msgr2=0x5654b770c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b770c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 -- 172.21.15.19:0/1862353278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b770cec0 msgr2=0x5654b7781d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b770cec0 0x5654b7781d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 -- 172.21.15.19:0/1862353278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b76e8510 con 0x5654b76ec5d0 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65e63640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b770c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd65662640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009570 tx=0x7fcd5c02a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f7b00b7ab083ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.066+0000 7fcd497fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c037070 con 0x5654b76ec5d0 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcd5c0046c0 con 0x5654b76ec5d0 2024-09-12T03:22:34.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654b7659d70 con 0x5654b76ec5d0 2024-09-12T03:22:34.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd5c0305b0 con 0x5654b76ec5d0 2024-09-12T03:22:34.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd6a4a0740 10 client.16122 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:34.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.067+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5654b7783530 con 0x5654b76ec5d0 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5654b7659d70 con 0x5654b76ec5d0 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fcd5c047450 con 0x5654b76ec5d0 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 10 client.16122 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 20 client.16122 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b7783940 0x5654b7785d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 4 client.16122 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654b75d6170 con 0x5654b7783940 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.068+0000 7fcd65e63640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b7783940 0x5654b7785d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.069+0000 7fcd65e63640 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b7783940 0x5654b7785d60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7143cd9e6fd6dee server_cookie=785d56d00e694bd5 in_seq=0 out_seq=0 2024-09-12T03:22:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.069+0000 7fcd497fa640 10 client.16122 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd497fa640 1 -- 172.21.15.19:0/1862353278 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654b75d6170 con 0x5654b7783940 2024-09-12T03:22:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd497fa640 10 client.16122 handle_command_reply: tid=0 2024-09-12T03:22:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 client.16122 shutdown 2024-09-12T03:22:34.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 msgr2=0x5654b770c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009570 tx=0x7fcd5c02a380 comp rx=0 tx=0).stop 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 shutdown_connections 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5654b7783940 0x5654b7785d60 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5654b770cec0 0x5654b7781d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5654b76f4c90 0x5654b770c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 --2- 172.21.15.19:0/1862353278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5654b76ec5d0 0x5654b770c440 secure :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c009570 tx=0x7fcd5c02a380 comp rx=0 tx=0).stop 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 >> 172.21.15.19:0/1862353278 conn(0x5654b76b5310 msgr2=0x5654b76e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd497fa640 0 client.16122 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 shutdown_connections 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 1 -- 172.21.15.19:0/1862353278 wait complete. 2024-09-12T03:22:34.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.073+0000 7fcd6a4a0740 20 client.16122 trim_cache size 0 max 16384 2024-09-12T03:22:34.075 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:22:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 msgr2=0x7fcd4c0a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 0x7fcd4c0a5480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd500cf000 tx=0x7fcd500ced10 comp rx=0 tx=0).stop 2024-09-12T03:22:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 msgr2=0x7fcd601abb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 0x7fcd601abb80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcd50005fd0 tx=0x7fcd5000dae0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 shutdown_connections 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fcd4c0a2fc0 0x7fcd4c0a5480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcd6011b140 0x7fcd601abb80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcd60109830 0x7fcd601a77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 --2- 172.21.15.19:0/180679127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcd60109450 0x7fcd601a72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.074+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 >> 172.21.15.19:0/180679127 conn(0x7fcd6006e140 msgr2=0x7fcd6011c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.075+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 shutdown_connections 2024-09-12T03:22:34.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:34.075+0000 7fcd680c7640 1 -- 172.21.15.19:0/180679127 wait complete. 2024-09-12T03:22:35.272 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:35 smithi019 ceph-mon[30953]: pgmap v443: 97 pgs: 97 active+clean; 454 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.5 MiB/s wr, 287 op/s 2024-09-12T03:22:35.293 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:35 smithi135 ceph-mon[35474]: pgmap v443: 97 pgs: 97 active+clean; 454 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.5 MiB/s wr, 287 op/s 2024-09-12T03:22:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:35 smithi121 ceph-mon[34963]: pgmap v443: 97 pgs: 97 active+clean; 454 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 5.5 MiB/s wr, 287 op/s 2024-09-12T03:22:35.696 INFO:tasks.workunit.client.0.smithi019.stdout: 127 953341 713 657663 769 198528 574 2024-09-12T03:22:36.276 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:22:36.277 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:22:36.623 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:37.265 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:37 smithi019 ceph-mon[30953]: pgmap v444: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-12T03:22:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:37 smithi135 ceph-mon[35474]: pgmap v444: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-12T03:22:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:37 smithi121 ceph-mon[34963]: pgmap v444: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 292 op/s 2024-09-12T03:22:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.453+0000 7f4562232640 1 -- 172.21.15.19:0/1222807016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c06ea30 msgr2=0x7f455c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.453+0000 7f4562232640 1 --2- 172.21.15.19:0/1222807016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c06ea30 0x7f455c06ee70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f455801a910 tx=0x7f455803b470 comp rx=0 tx=0).stop 2024-09-12T03:22:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 -- 172.21.15.19:0/1222807016 shutdown_connections 2024-09-12T03:22:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 --2- 172.21.15.19:0/1222807016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c11b140 0x7f455c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 --2- 172.21.15.19:0/1222807016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c06ea30 0x7f455c06ee70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 --2- 172.21.15.19:0/1222807016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c110190 0x7f455c06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 -- 172.21.15.19:0/1222807016 >> 172.21.15.19:0/1222807016 conn(0x7f455c073180 msgr2=0x7f455c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 -- 172.21.15.19:0/1222807016 shutdown_connections 2024-09-12T03:22:37.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.458+0000 7f4562232640 1 -- 172.21.15.19:0/1222807016 wait complete. 2024-09-12T03:22:37.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.460+0000 7f4562232640 1 Processor -- start 2024-09-12T03:22:37.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.460+0000 7f4562232640 1 -- start start 2024-09-12T03:22:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.461+0000 7f4562232640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 0x7f455c11a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.465+0000 7f4562232640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c110190 0x7f455c116960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.465+0000 7f4562232640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.466+0000 7f4562232640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113300 con 0x7f455c11b140 2024-09-12T03:22:37.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.466+0000 7f4561a31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.466+0000 7f4561a31640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50614/0 (socket says 172.21.15.19:50614) 2024-09-12T03:22:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.467+0000 7f4561230640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 0x7f455c11a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.467+0000 7f4561a31640 1 -- 172.21.15.19:0/2233836518 learned_addr learned my addr 172.21.15.19:0/2233836518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.467+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113180 con 0x7f455c110190 2024-09-12T03:22:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.467+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f455c113480 con 0x7f455c06ea30 2024-09-12T03:22:37.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.467+0000 7f4560a2f640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c110190 0x7f455c116960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 -- 172.21.15.19:0/2233836518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 msgr2=0x7f455c11a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 0x7f455c11a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 -- 172.21.15.19:0/2233836518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c110190 msgr2=0x7f455c116960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c110190 0x7f455c116960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 -- 172.21.15.19:0/2233836518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455c1175d0 con 0x7f455c11b140 2024-09-12T03:22:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.469+0000 7f4561a31640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e970 tx=0x7f454c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.470+0000 7f45527fc640 1 -- 172.21.15.19:0/2233836518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c00ccc0 con 0x7f455c11b140 2024-09-12T03:22:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.470+0000 7f45527fc640 1 -- 172.21.15.19:0/2233836518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f454c005500 con 0x7f455c11b140 2024-09-12T03:22:37.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.470+0000 7f45527fc640 1 -- 172.21.15.19:0/2233836518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c010430 con 0x7f455c11b140 2024-09-12T03:22:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.470+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455c1aaf60 con 0x7f455c11b140 2024-09-12T03:22:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.471+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f455c1ab3c0 con 0x7f455c11b140 2024-09-12T03:22:37.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.471+0000 7f4561230640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 0x7f455c11a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:37.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 Processor -- start 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 -- start start 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b8d70 0x55de832b9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b9690 0x55de832c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55de8321e280 con 0x55de832b8d70 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55de831c5b10 con 0x55de832b9690 2024-09-12T03:22:37.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.477+0000 7f456586d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de8321a240 con 0x55de832c3f50 2024-09-12T03:22:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.479+0000 7f4560a2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.479+0000 7f4560a2f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48652/0 (socket says 172.21.15.19:48652) 2024-09-12T03:22:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.479+0000 7f4560a2f640 1 -- 172.21.15.19:0/114448118 learned_addr learned my addr 172.21.15.19:0/114448118 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:37.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.479+0000 7f4561230640 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b9690 0x55de832c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.480+0000 7f4560a2f640 1 -- 172.21.15.19:0/114448118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b9690 msgr2=0x55de832c3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.480+0000 7f4560a2f640 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b9690 0x55de832c3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.480+0000 7f4560a2f640 1 -- 172.21.15.19:0/114448118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b8d70 msgr2=0x55de832b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.480+0000 7f4560a2f640 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b8d70 0x55de832b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.480+0000 7f4560a2f640 1 -- 172.21.15.19:0/114448118 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de831d21f0 con 0x55de832c3f50 2024-09-12T03:22:37.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.481+0000 7f4560a2f640 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f455801a910 tx=0x7f455801a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e272cd67344ecf30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.484+0000 7f45527fc640 1 -- 172.21.15.19:0/2233836518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f454c0105d0 con 0x7f455c11b140 2024-09-12T03:22:37.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.485+0000 7f45527fc640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 0x7f45480a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.486+0000 7f45527fc640 1 -- 172.21.15.19:0/2233836518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f454c0cf720 con 0x7f455c11b140 2024-09-12T03:22:37.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.486+0000 7f4561230640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 0x7f45480a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.488+0000 7f4561230640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 0x7f45480a5250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55de832b6160 tx=0x7f4554009780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.489+0000 7f45477fe640 1 -- 172.21.15.19:0/114448118 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558054070 con 0x55de832c3f50 2024-09-12T03:22:37.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.489+0000 7f45477fe640 1 -- 172.21.15.19:0/114448118 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45580046c0 con 0x55de832c3f50 2024-09-12T03:22:37.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.489+0000 7f45477fe640 1 -- 172.21.15.19:0/114448118 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558050440 con 0x55de832c3f50 2024-09-12T03:22:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.490+0000 7f456586d740 1 -- 172.21.15.19:0/114448118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 msgr2=0x55de832c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.490+0000 7f456586d740 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f455801a910 tx=0x7f455801a510 comp rx=0 tx=0).stop 2024-09-12T03:22:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.492+0000 7f456586d740 1 -- 172.21.15.19:0/114448118 shutdown_connections 2024-09-12T03:22:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.492+0000 7f456586d740 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832c3f50 0x55de832c6340 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f455801a910 tx=0x7f455801a510 comp rx=0 tx=0).stop 2024-09-12T03:22:37.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.492+0000 7f456586d740 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b9690 0x55de832c3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.492+0000 7f456586d740 1 --2- 172.21.15.19:0/114448118 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b8d70 0x55de832b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.492+0000 7f456586d740 1 -- 172.21.15.19:0/114448118 >> 172.21.15.19:0/114448118 conn(0x55de832b5300 msgr2=0x55de832b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.496+0000 7f456586d740 1 -- 172.21.15.19:0/114448118 shutdown_connections 2024-09-12T03:22:37.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.497+0000 7f456586d740 1 -- 172.21.15.19:0/114448118 wait complete. 2024-09-12T03:22:37.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.497+0000 7f456586d740 1 -- start start 2024-09-12T03:22:37.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.497+0000 7f456586d740 1 Processor -- start 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.498+0000 7f456586d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.498+0000 7f456586d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832b9690 0x55de832c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.499+0000 7f456586d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832c1a00 0x55de832d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.499+0000 7f456586d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55de832c6b20 con 0x55de832c1a00 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.499+0000 7f456586d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55de831d8f10 con 0x55de832b8d70 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.499+0000 7f456586d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55de832c6ca0 con 0x55de832b9690 2024-09-12T03:22:37.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.499+0000 7f456586d740 10 client.0 _ll_register_callbacks cb 0x55de831a1a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34650/0 (socket says 172.21.15.19:34650) 2024-09-12T03:22:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 -- 172.21.15.19:0/639904113 learned_addr learned my addr 172.21.15.19:0/639904113 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561a31640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832c1a00 0x55de832d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4560a2f640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832b9690 0x55de832c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 -- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832b9690 msgr2=0x55de832c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832b9690 0x55de832c14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 -- 172.21.15.19:0/639904113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832c1a00 msgr2=0x55de832d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832c1a00 0x55de832d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.501+0000 7f4561230640 1 -- 172.21.15.19:0/639904113 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de82f8f6a0 con 0x55de832b8d70 2024-09-12T03:22:37.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.502+0000 7f4561230640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4554010980 tx=0x7f4554004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcb9d2ce249a7090 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.505+0000 7f4544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:22:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.505+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455401d6d0 con 0x55de832b8d70 2024-09-12T03:22:37.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.505+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f455401d870 con 0x55de832b8d70 2024-09-12T03:22:37.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.505+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45580046c0 con 0x55de832b8d70 2024-09-12T03:22:37.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.508+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f455401da70 con 0x55de832b8d70 2024-09-12T03:22:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.510+0000 7f456586d740 10 client.25640 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:37.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.510+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de832d2da0 con 0x55de832b8d70 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.514+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f45580046c0 con 0x55de832b8d70 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.514+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4554027430 con 0x55de832b8d70 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 20 client.25640 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de832d42a0 0x55de832d6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 4 client.25640 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.516+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de8322b7f0 con 0x55de832d42a0 2024-09-12T03:22:37.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.518+0000 7f4560a2f640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de832d42a0 0x55de832d6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.520+0000 7f4560a2f640 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de832d42a0 0x55de832d6740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48ff92f46214e9a7 server_cookie=18e03f87f155507b in_seq=0 out_seq=0 2024-09-12T03:22:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.522+0000 7f4544ff9640 10 client.25640 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.523+0000 7f4544ff9640 1 -- 172.21.15.19:0/639904113 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55de8322b7f0 con 0x55de832d42a0 2024-09-12T03:22:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.523+0000 7f4544ff9640 10 client.25640 handle_command_reply: tid=0 2024-09-12T03:22:37.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.526+0000 7f456586d740 1 client.25640 shutdown 2024-09-12T03:22:37.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 msgr2=0x55de832c0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4554010980 tx=0x7f4554004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:37.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 shutdown_connections 2024-09-12T03:22:37.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832c1a00 0x55de832d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832b9690 0x55de832c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de832d42a0 0x55de832d6740 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f4544ff9640 0 client.25640 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:37.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 --2- 172.21.15.19:0/639904113 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832b8d70 0x55de832c0f80 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4554010980 tx=0x7f4554004a40 comp rx=0 tx=0).stop 2024-09-12T03:22:37.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.534+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 >> 172.21.15.19:0/639904113 conn(0x55de832b5300 msgr2=0x55de832bb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.537+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 shutdown_connections 2024-09-12T03:22:37.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.539+0000 7f456586d740 1 -- 172.21.15.19:0/639904113 wait complete. 2024-09-12T03:22:37.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.540+0000 7f456586d740 20 client.25640 trim_cache size 0 max 16384 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.582+0000 7f456586d740 1 Processor -- start 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.583+0000 7f456586d740 1 -- start start 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.583+0000 7f456586d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.583+0000 7f456586d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.583+0000 7f456586d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832dc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.584+0000 7f456586d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f454000e510 con 0x55de832b9690 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.584+0000 7f456586d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4540010110 con 0x55de832c1a00 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.584+0000 7f456586d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f454000ebe0 con 0x55de832bff60 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.585+0000 7f4560a2f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.585+0000 7f4560a2f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34662/0 (socket says 172.21.15.19:34662) 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.585+0000 7f4560a2f640 1 -- 172.21.15.19:0/1777812322 learned_addr learned my addr 172.21.15.19:0/1777812322 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.585+0000 7f4561a31640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832dc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4561230640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4560a2f640 1 -- 172.21.15.19:0/1777812322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 msgr2=0x55de832dc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4560a2f640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832dc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4560a2f640 1 -- 172.21.15.19:0/1777812322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 msgr2=0x55de832cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4560a2f640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4560a2f640 1 -- 172.21.15.19:0/1777812322 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de832b5f90 con 0x55de832c1a00 2024-09-12T03:22:37.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.586+0000 7f4561230640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:22:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.590+0000 7f4560a2f640 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4558005ab0 tx=0x7f4558040e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22938a78fe8298be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.591+0000 7f4546ffd640 1 -- 172.21.15.19:0/1777812322 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558054040 con 0x55de832c1a00 2024-09-12T03:22:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.591+0000 7f4546ffd640 1 -- 172.21.15.19:0/1777812322 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4558040a70 con 0x55de832c1a00 2024-09-12T03:22:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.592+0000 7f456586d740 1 -- 172.21.15.19:0/1777812322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 msgr2=0x55de832dc050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.592+0000 7f456586d740 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4558005ab0 tx=0x7f4558040e30 comp rx=0 tx=0).stop 2024-09-12T03:22:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.596+0000 7f456586d740 1 -- 172.21.15.19:0/1777812322 shutdown_connections 2024-09-12T03:22:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.596+0000 7f456586d740 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832dc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.596+0000 7f456586d740 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832dc050 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.596+0000 7f456586d740 1 --2- 172.21.15.19:0/1777812322 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.596+0000 7f456586d740 1 -- 172.21.15.19:0/1777812322 >> 172.21.15.19:0/1777812322 conn(0x55de832b5300 msgr2=0x55de832bf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.600+0000 7f456586d740 1 -- 172.21.15.19:0/1777812322 shutdown_connections 2024-09-12T03:22:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.602+0000 7f456586d740 1 -- 172.21.15.19:0/1777812322 wait complete. 2024-09-12T03:22:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.603+0000 7f456586d740 1 -- start start 2024-09-12T03:22:37.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.603+0000 7f456586d740 1 Processor -- start 2024-09-12T03:22:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f454000f980 con 0x55de832b9690 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f454000f2b0 con 0x55de832c1a00 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f454000c4f0 con 0x55de832bff60 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.604+0000 7f456586d740 10 client.0 _ll_register_callbacks cb 0x7f4554010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.605+0000 7f4561230640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.605+0000 7f4561230640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50636/0 (socket says 172.21.15.19:50636) 2024-09-12T03:22:37.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.605+0000 7f4561230640 1 -- 172.21.15.19:0/1178437296 learned_addr learned my addr 172.21.15.19:0/1178437296 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 -- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 msgr2=0x55de832cc870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832cc870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 -- 172.21.15.19:0/1178437296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 msgr2=0x55de832c9150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832c9150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 -- 172.21.15.19:0/1178437296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de832c9980 con 0x55de832b9690 2024-09-12T03:22:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.606+0000 7f4561230640 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4554006010 tx=0x7f45540276c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87036c67c778a273 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.608+0000 7f4544ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.608+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554009e50 con 0x55de832b9690 2024-09-12T03:22:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.608+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4558040a70 con 0x55de832b9690 2024-09-12T03:22:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.608+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f455401dc10 con 0x55de832b9690 2024-09-12T03:22:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.609+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f455401de10 con 0x55de832b9690 2024-09-12T03:22:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.613+0000 7f456586d740 10 client.16134 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:37.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.613+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55de8333fd60 con 0x55de832b9690 2024-09-12T03:22:37.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.614+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4558040a70 con 0x55de832b9690 2024-09-12T03:22:37.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f45540052f0 con 0x55de832b9690 2024-09-12T03:22:37.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 10 client.16134 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 20 client.16134 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de83340580 0x55de833429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 4 client.16134 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:22:37.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.615+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55de83225d00 con 0x55de83340580 2024-09-12T03:22:37.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.620+0000 7f4560a2f640 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de83340580 0x55de833429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.629+0000 7f4560a2f640 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de83340580 0x55de833429e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87709ee247dde9cc server_cookie=ae24f2883ecc7321 in_seq=0 out_seq=0 2024-09-12T03:22:37.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.629+0000 7f4544ff9640 10 client.16134 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.819+0000 7f4544ff9640 1 -- 172.21.15.19:0/1178437296 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55de83225d00 con 0x55de83340580 2024-09-12T03:22:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.819+0000 7f4544ff9640 10 client.16134 handle_command_reply: tid=0 2024-09-12T03:22:37.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.820+0000 7f456586d740 1 client.16134 shutdown 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 msgr2=0x55de832cc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4554006010 tx=0x7f45540276c0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 shutdown_connections 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55de83340580 0x55de833429e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55de832c1a00 0x55de832c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55de832bff60 0x55de832cc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 --2- 172.21.15.19:0/1178437296 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55de832b9690 0x55de832cc330 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4554006010 tx=0x7f45540276c0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 >> 172.21.15.19:0/1178437296 conn(0x55de832b5300 msgr2=0x55de832b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.822+0000 7f4544ff9640 0 client.16134 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:37.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.823+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 shutdown_connections 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "a8677c0e-d21d-40f7-abe4-e6852d91f219", 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:22:37.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.823+0000 7f456586d740 1 -- 172.21.15.19:0/1178437296 wait complete. 2024-09-12T03:22:37.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.824+0000 7f456586d740 20 client.16134 trim_cache size 0 max 16384 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.824+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 msgr2=0x7f45480a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.824+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 0x7f45480a5250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55de832b6160 tx=0x7f4554009780 comp rx=0 tx=0).stop 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 msgr2=0x7f455c116ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f454c00e970 tx=0x7f454c00ee30 comp rx=0 tx=0).stop 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 shutdown_connections 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f45480a2d90 0x7f45480a5250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f455c11b140 0x7f455c116ea0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f455c110190 0x7f455c116960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 --2- 172.21.15.19:0/2233836518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f455c06ea30 0x7f455c11a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.825+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 >> 172.21.15.19:0/2233836518 conn(0x7f455c073180 msgr2=0x7f455c11b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.826+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 shutdown_connections 2024-09-12T03:22:37.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:37.827+0000 7f4562232640 1 -- 172.21.15.19:0/2233836518 wait complete. 2024-09-12T03:22:38.544 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:22:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:22:38.268+0000 7f754817e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002ac1 [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76/ auth{1=1,2=1,3=1,4=1} fragtree_t(*^3) v116546 ap=2 DIRTYPARENT f(v2 m2024-09-12T03:21:45.752190+0000 101=101+0) n(v2 rc2024-09-12T03:21:49.067158+0000 b2701303 102=101+1) (inest sync->mix g=1,2,3,4) caps={15210=pAsLsXsFs/pAsLsXsFsx@222} | dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55df3208cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x10000002ac1:[<0x10000000399/blog-76 v116546>,<0x10000000208/blogtest_in v1458>,<0x10000000201/src v2234>,<0x10000000006/blogbench-1.0 v1953>,<0x10000000005/tmp v212>,<0x100000001fb/client.0 v202>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v219>,<0x10000000001/sv_1 v236>,<0x10000000000/qa v224>,<0x1/volumes v219>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:45.752190+0000 100=100+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:49.067158+0000 b2701303 102=101+1)","memory_value.dirstat":"f(v2 m2024-09-12T03:21:45.752190+0000 101=101+0)","memory_value.rstat":"n(v2 rc2024-09-12T03:21:49.067158+0000 b2701303 102=101+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-09-12T03:22:38.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:22:39.013 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:22:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:22:38.558+0000 7f754817e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000061e [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-7/ auth{1=1,2=1,3=1,4=1} fragtree_t(*^3) v121251 ap=1 DIRTYPARENT f(v3 m2024-09-12T03:21:38.625258+0000 136=136+0) n(v7 rc2024-09-12T03:21:38.625258+0000 b1811549 137=136+1) caps={15210=pAsLsXsFs/pAsLsXsFsx@506} | ptrwaiter=4 dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyrstat=0 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55df2ee82680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x1000000061e:[<0x10000000399/blog-7 v121251>,<0x10000000208/blogtest_in v1536>,<0x10000000201/src v2234>,<0x10000000006/blogbench-1.0 v1953>,<0x10000000005/tmp v212>,<0x100000001fb/client.0 v202>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v221>,<0x10000000001/sv_1 v239>,<0x10000000000/qa v226>,<0x1/volumes v221>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:33.533306+0000 137=137+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:38.625258+0000 b1811549 137=136+1)","memory_value.dirstat":"f(v3 m2024-09-12T03:21:38.625258+0000 136=136+0)","memory_value.rstat":"n(v7 rc2024-09-12T03:21:38.625258+0000 b1811549 137=136+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-09-12T03:22:39.065 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:22:39.268 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: pgmap v445: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.269 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.270 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.271 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: pgmap v445: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details 2024-09-12T03:22:39.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: pgmap v445: 97 pgs: 97 active+clean; 455 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.5 MiB/s wr, 266 op/s 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:39.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:39.754 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:22:39.441+0000 7f754817e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000016bc [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28/ auth{1=1,2=1,3=1,4=1} fragtree_t(*^3) v126746 ap=2 DIRTYPARENT f(v3 m2024-09-12T03:21:48.869160+0000 113=113+0) n(v6 rc2024-09-12T03:21:48.869160+0000 b2290775 114=113+1) (inest sync->mix g=1,2,3,4) caps={15210=pAsLsXsFs/pAsLsXsFsx@318} | ptrwaiter=0 dirtyscattered=0 request=0 lock=0 dirfrag=8 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55df2fd67700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x100000016bc:[<0x10000000399/blog-28 v126746>,<0x10000000208/blogtest_in v1568>,<0x10000000201/src v2236>,<0x10000000006/blogbench-1.0 v1955>,<0x10000000005/tmp v214>,<0x100000001fb/client.0 v205>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v223>,<0x10000000001/sv_1 v239>,<0x10000000000/qa v226>,<0x1/volumes v221>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:36.813275+0000 112=112+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:48.869160+0000 b2290775 114=113+1)","memory_value.dirstat":"f(v3 m2024-09-12T03:21:48.869160+0000 113=113+0)","memory_value.rstat":"n(v6 rc2024-09-12T03:21:48.869160+0000 b2290775 114=113+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-09-12T03:22:39.754 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:22:39.556+0000 7f754817e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000017cf [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31/ auth{1=1,2=1,3=1,4=1} v116800 ap=2 DIRTYPARENT f(v1 m2024-09-12T03:21:50.257147+0000 54=54+0) n(v1 rc2024-09-12T03:21:50.257147+0000 b1447768 55=54+1) (inest sync->mix g=1,2,3,4) caps={15210=pAsLsXsFs/pAsLsXsFsx@210} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55df2f44e680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x100000017cf:[<0x10000000399/blog-31 v116800>,<0x10000000208/blogtest_in v1576>,<0x10000000201/src v2236>,<0x10000000006/blogbench-1.0 v1955>,<0x10000000005/tmp v214>,<0x100000001fb/client.0 v205>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v223>,<0x10000000001/sv_1 v239>,<0x10000000000/qa v226>,<0x1/volumes v221>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:07.360556+0000 52=52+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:50.257147+0000 b1447768 55=54+1)","memory_value.dirstat":"f(v1 m2024-09-12T03:21:50.257147+0000 54=54+0)","memory_value.rstat":"n(v1 rc2024-09-12T03:21:50.257147+0000 b1447768 55=54+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-12T03:22:39.900 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:22:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: 2024-09-12T03:22:39.610+0000 7fb206f3e640 -1 mds.4.scrubstack _validate_inode_done scrub error on inode [inode 0x100000009b8 [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11/ auth{0=1,1=1,2=1,3=1} fragtree_t(*^3) v121255 ap=2 DIRTYPARENT f(v3 m2024-09-12T03:21:17.153462+0000 193=193+0) n(v5 rc2024-09-12T03:21:17.153462+0000 b4580881 194=193+1) (inest sync->mix g=0,1,2,3) caps={15210=pAsLsXsFs/pAsLsXsFsx@1} | lock=0 importingcaps=0 dirfrag=8 caps=1 stickydirs=0 dirtyparent=1 dirwaiter=0 replicated=1 waiter=0 authpin=1 discoverbase=0 scrubqueue=0 0x561c47542b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x100000009b8:[<0x10000000399/blog-11 v121255>,<0x10000000208/blogtest_in v1534>,<0x10000000201/src v2224>,<0x10000000006/blogbench-1.0 v1928>,<0x10000000005/tmp v195>,<0x100000001fb/client.0 v194>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v221>,<0x10000000001/sv_1 v228>,<0x10000000000/qa v226>,<0x1/volumes v210>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:15.506478+0000 194=194+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:17.153462+0000 b4580881 194=193+1)","memory_value.dirstat":"f(v3 m2024-09-12T03:21:17.153462+0000 193=193+0)","memory_value.rstat":"n(v5 rc2024-09-12T03:21:17.153462+0000 b4580881 194=193+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-61} 2024-09-12T03:22:40.029 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:22:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:22:39.753+0000 7f754817e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000002d09 [...2,head] /volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-83/ auth{1=1,2=1,3=1,4=1} v121293 ap=2 DIRTYPARENT f(v1 m2024-09-12T03:21:48.793161+0000 69=69+0) n(v2 rc2024-09-12T03:21:48.793161+0000 b1349918 70=69+1) (inest sync->mix g=1,2,3,4) caps={15210=pAsLsXsFs/pAsLsXsFsx@183} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=1 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55df310ad180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(2)0x10000002d09:[<0x10000000399/blog-83 v121293>,<0x10000000208/blogtest_in v1592>,<0x10000000201/src v2236>,<0x10000000006/blogbench-1.0 v1955>,<0x10000000005/tmp v214>,<0x100000001fb/client.0 v205>,<0x100000001fa/68e383da-dc2e-4231-8b0e-a1f655e088d6 v223>,<0x10000000001/sv_1 v239>,<0x10000000000/qa v226>,<0x1/volumes v221>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-12T03:21:42.924217+0000 67=67+0)","ondisk_value.rstat":"n(v0 rc2024-09-12T03:21:48.793161+0000 b1349918 70=69+1)","memory_value.dirstat":"f(v1 m2024-09-12T03:21:48.793161+0000 69=69+0)","memory_value.rstat":"n(v2 rc2024-09-12T03:21:48.793161+0000 b1349918 70=69+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-12T03:22:40.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.956+0000 7f4aa2aba640 1 -- 172.21.15.19:0/2940522050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109450 msgr2=0x7f4a9c110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.956+0000 7f4aa2aba640 1 --2- 172.21.15.19:0/2940522050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109450 0x7f4a9c110250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4a90009900 tx=0x7f4a9002a440 comp rx=0 tx=0).stop 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.957+0000 7f4aa2aba640 1 -- 172.21.15.19:0/2940522050 shutdown_connections 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.957+0000 7f4aa2aba640 1 --2- 172.21.15.19:0/2940522050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c11b140 0x7f4a9c11d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.957+0000 7f4aa2aba640 1 --2- 172.21.15.19:0/2940522050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c109830 0x7f4a9c109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.957+0000 7f4aa2aba640 1 --2- 172.21.15.19:0/2940522050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109450 0x7f4a9c110250 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.957+0000 7f4aa2aba640 1 -- 172.21.15.19:0/2940522050 >> 172.21.15.19:0/2940522050 conn(0x7f4a9c06e140 msgr2=0x7f4a9c06e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.958+0000 7f4aa2aba640 1 -- 172.21.15.19:0/2940522050 shutdown_connections 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.958+0000 7f4aa2aba640 1 -- 172.21.15.19:0/2940522050 wait complete. 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.959+0000 7f4aa2aba640 1 Processor -- start 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.959+0000 7f4aa2aba640 1 -- start start 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109830 0x7f4a9c114760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c11b140 0x7f4a9c114ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1111e0 con 0x7f4a9c109830 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c111060 con 0x7f4a9c109450 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa2aba640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c111360 con 0x7f4a9c11b140 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34686/0 (socket says 172.21.15.19:34686) 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4aa0856640 1 -- 172.21.15.19:0/1718837669 learned_addr learned my addr 172.21.15.19:0/1718837669 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.960+0000 7f4a9bfff640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109830 0x7f4a9c114760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa1057640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c11b140 0x7f4a9c114ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 -- 172.21.15.19:0/1718837669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c11b140 msgr2=0x7f4a9c114ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c11b140 0x7f4a9c114ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 -- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109830 msgr2=0x7f4a9c114760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109830 0x7f4a9c114760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 -- 172.21.15.19:0/1718837669 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c115260 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.961+0000 7f4aa0856640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005ab0 tx=0x7f4a90005230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.962+0000 7f4a99ffb640 1 -- 172.21.15.19:0/1718837669 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90040030 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.962+0000 7f4a99ffb640 1 -- 172.21.15.19:0/1718837669 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a90004a30 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.962+0000 7f4a99ffb640 1 -- 172.21.15.19:0/1718837669 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9003c7c0 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.963+0000 7f4aa2aba640 1 -- 172.21.15.19:0/1718837669 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c1154f0 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.963+0000 7f4aa2aba640 1 -- 172.21.15.19:0/1718837669 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c126730 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.968+0000 7f4aa4e93740 1 Processor -- start 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.969+0000 7f4aa4e93740 1 -- start start 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.971+0000 7f4a99ffb640 1 -- 172.21.15.19:0/1718837669 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a90043030 con 0x7f4a9c109450 2024-09-12T03:22:40.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.973+0000 7f4a99ffb640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 0x7f4a840a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.974+0000 7f4a99ffb640 1 -- 172.21.15.19:0/1718837669 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4a900ef990 con 0x7f4a9c109450 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 0x559a289fc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 0x559a289fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a07430 0x559a28a09820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a289622e0 con 0x559a28a07430 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a28909ce0 con 0x559a289fcad0 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a2895e2a0 con 0x559a289fddc0 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.975+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 0x559a289fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.976+0000 7f4a9bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 0x559a289fcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34694/0 (socket says 172.21.15.19:34694) 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.976+0000 7f4a9bfff640 1 -- 172.21.15.19:0/4069605765 learned_addr learned my addr 172.21.15.19:0/4069605765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.976+0000 7f4aa1057640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 0x7f4a840a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.976+0000 7f4aa1057640 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a07430 0x559a28a09820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.977+0000 7f4aa0856640 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 0x559a289fc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 -- 172.21.15.19:0/4069605765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 msgr2=0x559a289fcf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 0x559a289fcf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 -- 172.21.15.19:0/4069605765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a07430 msgr2=0x559a28a09820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a07430 0x559a28a09820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 -- 172.21.15.19:0/4069605765 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a288ee440 con 0x559a289fddc0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.978+0000 7f4aa0856640 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 0x559a289fc590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a900bbea0 tx=0x7f4a900bc8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e4018e28b5c17f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.979+0000 7f4aa2aba640 1 -- 172.21.15.19:0/4069605765 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900c0550 con 0x559a289fddc0 2024-09-12T03:22:40.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.979+0000 7f4aa1057640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 0x7f4a840a5180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c012510 tx=0x7f4a8c009a00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.979+0000 7f4aa2aba640 1 -- 172.21.15.19:0/4069605765 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a900c4e10 con 0x559a289fddc0 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.981+0000 7f4aa4e93740 1 -- 172.21.15.19:0/4069605765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 msgr2=0x559a289fc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.981+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 0x559a289fc590 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4a900bbea0 tx=0x7f4a900bc8a0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 -- 172.21.15.19:0/4069605765 shutdown_connections 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a07430 0x559a28a09820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fcad0 0x559a289fcf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/4069605765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a289fddc0 0x559a289fc590 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 -- 172.21.15.19:0/4069605765 >> 172.21.15.19:0/4069605765 conn(0x559a289f8d20 msgr2=0x559a289f9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 -- 172.21.15.19:0/4069605765 shutdown_connections 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.982+0000 7f4aa4e93740 1 -- 172.21.15.19:0/4069605765 wait complete. 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.983+0000 7f4aa4e93740 1 -- start start 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.983+0000 7f4aa4e93740 1 Processor -- start 2024-09-12T03:22:40.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a06d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a04010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a04550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559a2895e2a0 con 0x559a289fcad0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559a2891d050 con 0x559a289fddc0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559a28a0a060 con 0x559a28a07430 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.984+0000 7f4aa4e93740 10 client.0 _ll_register_callbacks cb 0x559a2895d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.985+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a06d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.985+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a06d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50668/0 (socket says 172.21.15.19:50668) 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.985+0000 7f4aa0856640 1 -- 172.21.15.19:0/434491369 learned_addr learned my addr 172.21.15.19:0/434491369 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.985+0000 7f4aa1057640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a04550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.985+0000 7f4a9bfff640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a04010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.986+0000 7f4aa1057640 1 -- 172.21.15.19:0/434491369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 msgr2=0x559a28a04010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.986+0000 7f4aa1057640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a04010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.986+0000 7f4aa1057640 1 -- 172.21.15.19:0/434491369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 msgr2=0x559a28a06d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.986+0000 7f4aa1057640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a06d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.986+0000 7f4aa1057640 1 -- 172.21.15.19:0/434491369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a286d33c0 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.988+0000 7f4aa1057640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a04550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c012da0 tx=0x7f4a8c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f291d8acc00cf3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.988+0000 7f4a7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.989+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c02a480 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.989+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a900c4e10 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.989+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4a8c02a620 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.989+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c02a820 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.990+0000 7f4aa4e93740 10 client.25681 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.990+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a28a158e0 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.990+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a900c4e10 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4a8c01e070 con 0x559a28a07430 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 10 client.25681 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 10 client.25681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:40.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 20 client.25681 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a17220 0x559a28a19680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 4 client.25681 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.992+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a2896f850 con 0x559a28a17220 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.993+0000 7f4aa0856640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a17220 0x559a28a19680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.993+0000 7f4aa0856640 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a17220 0x559a28a19680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e87f68fc679462f server_cookie=d5f1428acccdc91f in_seq=0 out_seq=0 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.994+0000 7f4a7bfff640 10 client.25681 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.999+0000 7f4a7bfff640 1 -- 172.21.15.19:0/434491369 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a2896f850 con 0x559a28a17220 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.999+0000 7f4a7bfff640 10 client.25681 handle_command_reply: tid=0 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:39.999+0000 7f4aa4e93740 1 client.25681 shutdown 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 msgr2=0x559a28a04550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a04550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c012da0 tx=0x7f4a8c01a000 comp rx=0 tx=0).stop 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 shutdown_connections 2024-09-12T03:22:40.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a04550 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a17220 0x559a28a19680 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a04010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/434491369 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a06d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 >> 172.21.15.19:0/434491369 conn(0x559a289f8d20 msgr2=0x559a289fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.000+0000 7f4a7bfff640 0 client.25681 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.001+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 shutdown_connections 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.001+0000 7f4aa4e93740 1 -- 172.21.15.19:0/434491369 wait complete. 2024-09-12T03:22:40.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.001+0000 7f4aa4e93740 20 client.25681 trim_cache size 0 max 16384 2024-09-12T03:22:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.082+0000 7f4aa4e93740 1 Processor -- start 2024-09-12T03:22:40.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.084+0000 7f4aa4e93740 1 -- start start 2024-09-12T03:22:40.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.084+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.085+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a289ff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.085+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a289f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.085+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c12edc0 con 0x559a289fcad0 2024-09-12T03:22:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.085+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c1309c0 con 0x559a289fddc0 2024-09-12T03:22:40.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.085+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c12f490 con 0x559a28a07430 2024-09-12T03:22:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50674/0 (socket says 172.21.15.19:50674) 2024-09-12T03:22:40.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 -- 172.21.15.19:0/2905535570 learned_addr learned my addr 172.21.15.19:0/2905535570 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 -- 172.21.15.19:0/2905535570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 msgr2=0x559a289f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a289f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 -- 172.21.15.19:0/2905535570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 msgr2=0x559a289ff380 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a289ff380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.086+0000 7f4aa0856640 1 -- 172.21.15.19:0/2905535570 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a28a02fc0 con 0x559a289fcad0 2024-09-12T03:22:40.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.087+0000 7f4aa0856640 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a900bca50 tx=0x7f4a90004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac13301d8f697ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.087+0000 7f4a827fc640 1 -- 172.21.15.19:0/2905535570 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a900da070 con 0x559a289fcad0 2024-09-12T03:22:40.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.088+0000 7f4a827fc640 1 -- 172.21.15.19:0/2905535570 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a900c0550 con 0x559a289fcad0 2024-09-12T03:22:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 -- 172.21.15.19:0/2905535570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 msgr2=0x559a28a13190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f4a900bca50 tx=0x7f4a90004220 comp rx=0 tx=0).stop 2024-09-12T03:22:40.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 -- 172.21.15.19:0/2905535570 shutdown_connections 2024-09-12T03:22:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a289f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a289ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/2905535570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a289fcad0 0x559a28a13190 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.089+0000 7f4aa4e93740 1 -- 172.21.15.19:0/2905535570 >> 172.21.15.19:0/2905535570 conn(0x559a289f8d20 msgr2=0x559a289f76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.092+0000 7f4aa4e93740 1 -- 172.21.15.19:0/2905535570 shutdown_connections 2024-09-12T03:22:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.092+0000 7f4aa4e93740 1 -- 172.21.15.19:0/2905535570 wait complete. 2024-09-12T03:22:40.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.092+0000 7f4aa4e93740 1 -- start start 2024-09-12T03:22:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.092+0000 7f4aa4e93740 1 Processor -- start 2024-09-12T03:22:40.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a0fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a00e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c130230 con 0x559a28a013f0 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c12fb60 con 0x559a289fddc0 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c12ce50 con 0x559a28a07430 2024-09-12T03:22:40.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.093+0000 7f4aa4e93740 10 client.0 _ll_register_callbacks cb 0x7f4a9c110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:22:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.094+0000 7f4aa1057640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.094+0000 7f4aa1057640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50690/0 (socket says 172.21.15.19:50690) 2024-09-12T03:22:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.094+0000 7f4aa1057640 1 -- 172.21.15.19:0/3951764763 learned_addr learned my addr 172.21.15.19:0/3951764763 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:22:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.095+0000 7f4a9bfff640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a00e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.095+0000 7f4aa0856640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a0fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.095+0000 7f4aa1057640 1 -- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 msgr2=0x559a28a00e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.096+0000 7f4aa1057640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a00e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.096+0000 7f4aa1057640 1 -- 172.21.15.19:0/3951764763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 msgr2=0x559a28a0fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:22:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.096+0000 7f4aa1057640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a0fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.096+0000 7f4aa1057640 1 -- 172.21.15.19:0/3951764763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a28a811b0 con 0x559a28a013f0 2024-09-12T03:22:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.096+0000 7f4aa1057640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01fbd0 tx=0x7f4a8c005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c5314a7f3aab26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:22:40.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.097+0000 7f4a7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:22:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.098+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c009e10 con 0x559a28a013f0 2024-09-12T03:22:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.098+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a28969d60 con 0x559a28a013f0 2024-09-12T03:22:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.098+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4a8c01f400 con 0x559a28a013f0 2024-09-12T03:22:40.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.098+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c01f600 con 0x559a28a013f0 2024-09-12T03:22:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.101+0000 7f4aa4e93740 10 client.16152 fetch_fsmap learned FSMap version 30 2024-09-12T03:22:40.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.101+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559a28a82550 con 0x559a28a013f0 2024-09-12T03:22:40.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559a28969d60 con 0x559a28a013f0 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f4a8c01e050 con 0x559a28a013f0 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 10 client.16152 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 10 client.16152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 20 client.16152 populate_metadata read hostname 'smithi019' 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a82d90 0x559a28a851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 4 client.16152 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.102+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a288e6210 con 0x559a28a82d90 2024-09-12T03:22:40.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.103+0000 7f4aa0856640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a82d90 0x559a28a851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:22:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.106+0000 7f4aa0856640 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a82d90 0x559a28a851b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e284734942d0f1 server_cookie=68d21dce51febb4f in_seq=0 out_seq=0 2024-09-12T03:22:40.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.106+0000 7f4a7bfff640 10 client.16152 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.109+0000 7f4a7bfff640 1 -- 172.21.15.19:0/3951764763 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x559a288e6210 con 0x559a28a82d90 2024-09-12T03:22:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.109+0000 7f4a7bfff640 10 client.16152 handle_command_reply: tid=0 2024-09-12T03:22:40.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.109+0000 7f4aa4e93740 1 client.16152 shutdown 2024-09-12T03:22:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 msgr2=0x559a28a01850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c01fbd0 tx=0x7f4a8c005670 comp rx=0 tx=0).stop 2024-09-12T03:22:40.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 shutdown_connections 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559a28a013f0 0x559a28a01850 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559a28a07430 0x559a28a00e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x559a28a82d90 0x559a28a851b0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 --2- 172.21.15.19:0/3951764763 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559a289fddc0 0x559a28a0fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 >> 172.21.15.19:0/3951764763 conn(0x559a289f8d20 msgr2=0x559a289fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4a7bfff640 0 client.16152 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 shutdown_connections 2024-09-12T03:22:40.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.118+0000 7f4aa4e93740 1 -- 172.21.15.19:0/3951764763 wait complete. 2024-09-12T03:22:40.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.122+0000 7f4aa4e93740 20 client.16152 trim_cache size 0 max 16384 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (3033 inodes in the stack)", 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "a8677c0e-d21d-40f7-abe4-e6852d91f219": { 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "a8677c0e-d21d-40f7-abe4-e6852d91f219", 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:40.126 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:22:40.127 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:22:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.131+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 msgr2=0x7f4a840a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.131+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 0x7f4a840a5180 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c012510 tx=0x7f4a8c009a00 comp rx=0 tx=0).stop 2024-09-12T03:22:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.131+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 msgr2=0x7f4a9c11a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:22:40.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.131+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4a90005ab0 tx=0x7f4a90005230 comp rx=0 tx=0).stop 2024-09-12T03:22:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 shutdown_connections 2024-09-12T03:22:40.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4a9c11b140 0x7f4a9c114ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4a840a2cc0 0x7f4a840a5180 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4a9c109830 0x7f4a9c114760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 --2- 172.21.15.19:0/1718837669 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a9c109450 0x7f4a9c11a6e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:22:40.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.134+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 >> 172.21.15.19:0/1718837669 conn(0x7f4a9c06e140 msgr2=0x7f4a9c0722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:22:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.138+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 shutdown_connections 2024-09-12T03:22:40.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:22:40.138+0000 7f4a837fe640 1 -- 172.21.15.19:0/1718837669 wait complete. 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: Scrub error on inode 0x1000000061e (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-7) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: Scrub error on inode 0x100000016bc (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:40 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: Scrub error on inode 0x1000000061e (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-7) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: Scrub error on inode 0x100000016bc (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:40 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:40.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: Scrub error on inode 0x1000000061e (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-7) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: Scrub error on inode 0x100000016bc (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-28) see mds.i log and `damage ls` output for details 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:40.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:40 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: pgmap v446: 97 pgs: 97 active+clean; 458 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: Scrub error on inode 0x100000017cf (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: Scrub error on inode 0x100000009b8 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.k log and `damage ls` output for details 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: Scrub error on inode 0x10000002d09 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-83) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:41.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: pgmap v446: 97 pgs: 97 active+clean; 458 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: Scrub error on inode 0x100000017cf (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: Scrub error on inode 0x100000009b8 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.k log and `damage ls` output for details 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: Scrub error on inode 0x10000002d09 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-83) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:41.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: pgmap v446: 97 pgs: 97 active+clean; 458 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.5 MiB/s wr, 238 op/s 2024-09-12T03:22:41.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: Scrub error on inode 0x100000017cf (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-31) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: Scrub error on inode 0x100000009b8 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-11) see mds.k log and `damage ls` output for details 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: Scrub error on inode 0x10000002d09 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-83) see mds.i log and `damage ls` output for details 2024-09-12T03:22:41.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:41.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:41.488 INFO:tasks.cephfs.filesystem:scrub status for tag:a8677c0e-d21d-40f7-abe4-e6852d91f219 - {'path': '/', 'tag': 'a8677c0e-d21d-40f7-abe4-e6852d91f219', 'options': 'recursive,force'} 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.296 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.297 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.298 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.299 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.300 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.301 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.302 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.303 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.304 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.305 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.306 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.307 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.308 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.309 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:22:42.310 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.310 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.403 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.404 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.405 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.406 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.407 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.408 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.409 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.410 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.411 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.418 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.420 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.421 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.422 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.424 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.425 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.426 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.429 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:42.430 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:43 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:43 smithi121 ceph-mon[34963]: pgmap v447: 97 pgs: 97 active+clean; 463 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-12T03:22:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:43 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:43 smithi019 ceph-mon[30953]: pgmap v447: 97 pgs: 97 active+clean; 463 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-12T03:22:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:43 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:22:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:43 smithi135 ceph-mon[35474]: pgmap v447: 97 pgs: 97 active+clean; 463 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 237 op/s 2024-09-12T03:22:45.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:45 smithi121 ceph-mon[34963]: pgmap v448: 97 pgs: 97 active+clean; 469 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-12T03:22:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:45 smithi019 ceph-mon[30953]: pgmap v448: 97 pgs: 97 active+clean; 469 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-12T03:22:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:45 smithi135 ceph-mon[35474]: pgmap v448: 97 pgs: 97 active+clean; 469 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 189 op/s 2024-09-12T03:22:45.696 INFO:tasks.workunit.client.0.smithi019.stdout: 162 1840155 1638 1286216 2165 360650 281 2024-09-12T03:22:47.345 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:47 smithi135 ceph-mon[35474]: pgmap v449: 97 pgs: 97 active+clean; 490 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-12T03:22:47.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:47 smithi121 ceph-mon[34963]: pgmap v449: 97 pgs: 97 active+clean; 490 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-12T03:22:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:47 smithi019 ceph-mon[30953]: pgmap v449: 97 pgs: 97 active+clean; 490 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 225 op/s 2024-09-12T03:22:49.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:49 smithi121 ceph-mon[34963]: pgmap v450: 97 pgs: 97 active+clean; 536 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-09-12T03:22:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:49 smithi019 ceph-mon[30953]: pgmap v450: 97 pgs: 97 active+clean; 536 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-09-12T03:22:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:49 smithi135 ceph-mon[35474]: pgmap v450: 97 pgs: 97 active+clean; 536 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.3 MiB/s wr, 368 op/s 2024-09-12T03:22:51.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:51 smithi121 ceph-mon[34963]: pgmap v451: 97 pgs: 97 active+clean; 559 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 449 op/s 2024-09-12T03:22:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:51 smithi019 ceph-mon[30953]: pgmap v451: 97 pgs: 97 active+clean; 559 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 449 op/s 2024-09-12T03:22:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:51 smithi135 ceph-mon[35474]: pgmap v451: 97 pgs: 97 active+clean; 559 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.3 MiB/s wr, 449 op/s 2024-09-12T03:22:53.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:53 smithi121 ceph-mon[34963]: pgmap v452: 97 pgs: 97 active+clean; 595 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 553 op/s 2024-09-12T03:22:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:53 smithi019 ceph-mon[30953]: pgmap v452: 97 pgs: 97 active+clean; 595 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 553 op/s 2024-09-12T03:22:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:53 smithi135 ceph-mon[35474]: pgmap v452: 97 pgs: 97 active+clean; 595 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 553 op/s 2024-09-12T03:22:54.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:54.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:54.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:55 smithi121 ceph-mon[34963]: pgmap v453: 97 pgs: 97 active+clean; 627 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 628 op/s 2024-09-12T03:22:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:55 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:55.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:55 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:22:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:55 smithi019 ceph-mon[30953]: pgmap v453: 97 pgs: 97 active+clean; 627 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 628 op/s 2024-09-12T03:22:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:55 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:55 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:22:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:55 smithi135 ceph-mon[35474]: pgmap v453: 97 pgs: 97 active+clean; 627 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 14 MiB/s wr, 628 op/s 2024-09-12T03:22:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:55 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:55 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:22:55.696 INFO:tasks.workunit.client.0.smithi019.stdout: 202 1802443 2025 1293652 2052 322833 824 2024-09-12T03:22:57.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:57 smithi121 ceph-mon[34963]: pgmap v454: 97 pgs: 97 active+clean; 642 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-12T03:22:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:57 smithi019 ceph-mon[30953]: pgmap v454: 97 pgs: 97 active+clean; 642 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-12T03:22:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:57 smithi135 ceph-mon[35474]: pgmap v454: 97 pgs: 97 active+clean; 642 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 667 op/s 2024-09-12T03:22:59.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:22:59 smithi121 ceph-mon[34963]: pgmap v455: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 620 op/s 2024-09-12T03:22:59.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:22:59 smithi019 ceph-mon[30953]: pgmap v455: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 620 op/s 2024-09-12T03:22:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:22:59 smithi135 ceph-mon[35474]: pgmap v455: 97 pgs: 97 active+clean; 653 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 14 MiB/s wr, 620 op/s 2024-09-12T03:23:01.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:01 smithi121 ceph-mon[34963]: pgmap v456: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 465 op/s 2024-09-12T03:23:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:01 smithi019 ceph-mon[30953]: pgmap v456: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 465 op/s 2024-09-12T03:23:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:01 smithi135 ceph-mon[35474]: pgmap v456: 97 pgs: 97 active+clean; 656 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 465 op/s 2024-09-12T03:23:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:03 smithi121 ceph-mon[34963]: pgmap v457: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-12T03:23:03.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:03 smithi019 ceph-mon[30953]: pgmap v457: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-12T03:23:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:03 smithi135 ceph-mon[35474]: pgmap v457: 97 pgs: 97 active+clean; 661 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 401 op/s 2024-09-12T03:23:05.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:05 smithi121 ceph-mon[34963]: pgmap v458: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 288 op/s 2024-09-12T03:23:05.419 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:05 smithi019 ceph-mon[30953]: pgmap v458: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 288 op/s 2024-09-12T03:23:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:05 smithi135 ceph-mon[35474]: pgmap v458: 97 pgs: 97 active+clean; 667 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.7 MiB/s wr, 288 op/s 2024-09-12T03:23:05.699 INFO:tasks.workunit.client.0.smithi019.stdout: 254 2257764 2745 1613254 2784 383301 501 2024-09-12T03:23:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:07 smithi121 ceph-mon[34963]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-12T03:23:07.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:07 smithi019 ceph-mon[30953]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-12T03:23:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:07 smithi135 ceph-mon[35474]: pgmap v459: 97 pgs: 97 active+clean; 676 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.8 MiB/s wr, 223 op/s 2024-09-12T03:23:09.128 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:09 smithi019 ceph-mon[30953]: pgmap v460: 97 pgs: 97 active+clean; 732 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.2 MiB/s wr, 373 op/s 2024-09-12T03:23:09.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:09 smithi121 ceph-mon[34963]: pgmap v460: 97 pgs: 97 active+clean; 732 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.2 MiB/s wr, 373 op/s 2024-09-12T03:23:09.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:09 smithi135 ceph-mon[35474]: pgmap v460: 97 pgs: 97 active+clean; 732 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 8.2 MiB/s wr, 373 op/s 2024-09-12T03:23:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:11 smithi121 ceph-mon[34963]: pgmap v461: 97 pgs: 97 active+clean; 762 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 463 op/s 2024-09-12T03:23:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:11 smithi019 ceph-mon[30953]: pgmap v461: 97 pgs: 97 active+clean; 762 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 463 op/s 2024-09-12T03:23:11.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:23:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:11 smithi135 ceph-mon[35474]: pgmap v461: 97 pgs: 97 active+clean; 762 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.9 MiB/s wr, 463 op/s 2024-09-12T03:23:13.241 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:23:13.241 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:23:13.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:13.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:13 smithi019 ceph-mon[30953]: pgmap v462: 97 pgs: 97 active+clean; 798 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-12T03:23:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:13 smithi135 ceph-mon[35474]: pgmap v462: 97 pgs: 97 active+clean; 798 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-12T03:23:13.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:13 smithi121 ceph-mon[34963]: pgmap v462: 97 pgs: 97 active+clean; 798 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 13 MiB/s wr, 571 op/s 2024-09-12T03:23:15.394 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:15 smithi019 ceph-mon[30953]: pgmap v463: 97 pgs: 97 active+clean; 911 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 22 MiB/s wr, 931 op/s 2024-09-12T03:23:15.476 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:15 smithi135 ceph-mon[35474]: pgmap v463: 97 pgs: 97 active+clean; 911 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 22 MiB/s wr, 931 op/s 2024-09-12T03:23:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:15 smithi121 ceph-mon[34963]: pgmap v463: 97 pgs: 97 active+clean; 911 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 22 MiB/s wr, 931 op/s 2024-09-12T03:23:15.699 INFO:tasks.workunit.client.0.smithi019.stdout: 280 1865358 1072 1326219 1228 277542 573 2024-09-12T03:23:16.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.836+0000 7fa178cc2640 1 -- 172.21.15.19:0/2872557810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa17410e4e0 msgr2=0x7fa1741108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.836+0000 7fa178cc2640 1 --2- 172.21.15.19:0/2872557810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa17410e4e0 0x7fa1741108d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009be0 tx=0x7fa16c02a6c0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 -- 172.21.15.19:0/2872557810 shutdown_connections 2024-09-12T03:23:16.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 --2- 172.21.15.19:0/2872557810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa17410e4e0 0x7fa1741108d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 --2- 172.21.15.19:0/2872557810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa1741012e0 0x7fa17410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 --2- 172.21.15.19:0/2872557810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa174100930 0x7fa174100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 -- 172.21.15.19:0/2872557810 >> 172.21.15.19:0/2872557810 conn(0x7fa1740fc7c0 msgr2=0x7fa1740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 -- 172.21.15.19:0/2872557810 shutdown_connections 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.837+0000 7fa178cc2640 1 -- 172.21.15.19:0/2872557810 wait complete. 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.839+0000 7fa178cc2640 1 Processor -- start 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.839+0000 7fa178cc2640 1 -- start start 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa174100930 0x7fa17419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa17410e4e0 0x7fa17419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa1741140f0 con 0x7fa17410e4e0 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa174113f70 con 0x7fa1741012e0 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa178cc2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa174114270 con 0x7fa174100930 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39808/0 (socket says 172.21.15.19:39808) 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 -- 172.21.15.19:0/986175785 learned_addr learned my addr 172.21.15.19:0/986175785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa17259c640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa174100930 0x7fa17419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 -- 172.21.15.19:0/986175785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa174100930 msgr2=0x7fa17419a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa174100930 0x7fa17419a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 -- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa17410e4e0 msgr2=0x7fa17419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa17410e4e0 0x7fa17419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.840+0000 7fa171d9b640 1 -- 172.21.15.19:0/986175785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa17419f5e0 con 0x7fa1741012e0 2024-09-12T03:23:16.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.841+0000 7fa171d9b640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa16800e9c0 tx=0x7fa16800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.841+0000 7fa1637fe640 1 -- 172.21.15.19:0/986175785 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16800cde0 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.841+0000 7fa1637fe640 1 -- 172.21.15.19:0/986175785 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1680035f0 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.841+0000 7fa1637fe640 1 -- 172.21.15.19:0/986175785 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168010690 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.842+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa17419f8d0 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.842+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1741a71b0 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.859+0000 7fa1637fe640 1 -- 172.21.15.19:0/986175785 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa168010830 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.860+0000 7fa1637fe640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 0x7fa15c0a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.861+0000 7fa1637fe640 1 -- 172.21.15.19:0/986175785 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1680d0300 con 0x7fa1741012e0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.862+0000 7fa17259c640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 0x7fa15c0a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.863+0000 7fa17259c640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 0x7fa15c0a5210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa164009bd0 tx=0x7fa164005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.864+0000 7fa17b09b740 1 Processor -- start 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.866+0000 7fa17b09b740 1 -- start start 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.866+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.867+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.867+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa172d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa172d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58682/0 (socket says 172.21.15.19:58682) 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39818/0 (socket says 172.21.15.19:39818) 2024-09-12T03:23:16.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa17259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.868+0000 7fa17259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60520/0 (socket says 172.21.15.19:60520) 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.869+0000 7fa172d9d640 1 -- 172.21.15.19:0/1289056238 learned_addr learned my addr 172.21.15.19:0/1289056238 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.869+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5555f16bf2e0 con 0x5555f1775900 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.869+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5555f1666ce0 con 0x5555f1762490 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.872+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555f16bb2a0 con 0x5555f1761b70 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 -- 172.21.15.19:0/1289056238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 msgr2=0x5555f1761f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 -- 172.21.15.19:0/1289056238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 msgr2=0x5555f175d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 -- 172.21.15.19:0/1289056238 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f164b440 con 0x5555f1775900 2024-09-12T03:23:16.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa16c004af0 tx=0x7fa16c005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8828f7b7529c971b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.873+0000 7fa17259c640 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.874+0000 7fa160ff9640 1 -- 172.21.15.19:0/1289056238 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c041020 con 0x5555f1775900 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.874+0000 7fa160ff9640 1 -- 172.21.15.19:0/1289056238 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c0303a0 con 0x5555f1775900 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.874+0000 7fa160ff9640 1 -- 172.21.15.19:0/1289056238 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c036030 con 0x5555f1775900 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.879+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 msgr2=0x5555f1777cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.879+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa16c004af0 tx=0x7fa16c005c60 comp rx=0 tx=0).stop 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.880+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 shutdown_connections 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.880+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f1777cf0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.880+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.880+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1289056238 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f1761f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.880+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 >> 172.21.15.19:0/1289056238 conn(0x5555f175e240 msgr2=0x5555f175e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.884+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 shutdown_connections 2024-09-12T03:23:16.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.888+0000 7fa17b09b740 1 -- 172.21.15.19:0/1289056238 wait complete. 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.888+0000 7fa17b09b740 1 -- start start 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 Processor -- start 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1775900 0x5555f175ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5555f1778230 con 0x5555f1761b70 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5555f167a050 con 0x5555f1762490 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5555f17783b0 con 0x5555f1775900 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.889+0000 7fa17b09b740 10 client.0 _ll_register_callbacks cb 0x5555f16ba160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.890+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.891+0000 7fa17259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.891+0000 7fa17259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58694/0 (socket says 172.21.15.19:58694) 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.891+0000 7fa17259c640 1 -- 172.21.15.19:0/2713304417 learned_addr learned my addr 172.21.15.19:0/2713304417 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.893+0000 7fa171d9b640 1 -- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1775900 msgr2=0x5555f175ac30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.893+0000 7fa171d9b640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1775900 0x5555f175ac30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.893+0000 7fa171d9b640 1 -- 172.21.15.19:0/2713304417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 msgr2=0x5555f175d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.893+0000 7fa171d9b640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.893+0000 7fa171d9b640 1 -- 172.21.15.19:0/2713304417 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f14303c0 con 0x5555f1762490 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.894+0000 7fa17259c640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.895+0000 7fa171d9b640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175a640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa16809caa0 tx=0x7fa16809cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba265316eb9b95c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.895+0000 7fa159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.899+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1680ba060 con 0x5555f1762490 2024-09-12T03:23:16.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.899+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa16c0303a0 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.899+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1680037a0 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.904+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1680b4070 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.904+0000 7fa17b09b740 10 client.25673 fetch_fsmap learned FSMap version 30 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.904+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555f1794c50 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.905+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c0303a0 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa16809bf80 con 0x5555f1762490 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 10 client.25673 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 20 client.25673 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1796500 0x5555f1798960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 4 client.25673 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.906+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f16cc850 con 0x5555f1796500 2024-09-12T03:23:16.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.908+0000 7fa17259c640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1796500 0x5555f1798960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.912+0000 7fa17259c640 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1796500 0x5555f1798960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cb97d07789ac2d9 server_cookie=1562dc0b452dd56a in_seq=0 out_seq=0 2024-09-12T03:23:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.912+0000 7fa159ffb640 10 client.25673 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.913+0000 7fa159ffb640 1 -- 172.21.15.19:0/2713304417 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5555f16cc850 con 0x5555f1796500 2024-09-12T03:23:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.913+0000 7fa159ffb640 10 client.25673 handle_command_reply: tid=0 2024-09-12T03:23:16.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.913+0000 7fa17b09b740 1 client.25673 shutdown 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.913+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 msgr2=0x5555f175a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.913+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175a640 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa16809caa0 tx=0x7fa16809cb10 comp rx=0 tx=0).stop 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 shutdown_connections 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1775900 0x5555f175ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1796500 0x5555f1798960 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f175a640 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 --2- 172.21.15.19:0/2713304417 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1761b70 0x5555f175d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 >> 172.21.15.19:0/2713304417 conn(0x5555f175e240 msgr2=0x5555f1763fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.914+0000 7fa159ffb640 0 client.25673 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.922+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 shutdown_connections 2024-09-12T03:23:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.922+0000 7fa17b09b740 1 -- 172.21.15.19:0/2713304417 wait complete. 2024-09-12T03:23:16.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.922+0000 7fa17b09b740 20 client.25673 trim_cache size 0 max 16384 2024-09-12T03:23:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.971+0000 7fa17b09b740 1 Processor -- start 2024-09-12T03:23:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.972+0000 7fa17b09b740 1 -- start start 2024-09-12T03:23:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.972+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 0x5555f1754a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1762490 0x5555f179e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 0x5555f1754a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 0x5555f1754a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39834/0 (socket says 172.21.15.19:39834) 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17259c640 1 -- 172.21.15.19:0/1951704537 learned_addr learned my addr 172.21.15.19:0/1951704537 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f179e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa17406fb00 con 0x5555f1775900 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa174071700 con 0x5555f1761b70 2024-09-12T03:23:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.973+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740701d0 con 0x5555f1762490 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f179e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa171d9b640 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1762490 0x5555f179e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 -- 172.21.15.19:0/1951704537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1762490 msgr2=0x5555f179e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1762490 0x5555f179e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 -- 172.21.15.19:0/1951704537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 msgr2=0x5555f1754a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 0x5555f1754a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.974+0000 7fa172d9d640 1 -- 172.21.15.19:0/1951704537 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f175afa0 con 0x5555f1775900 2024-09-12T03:23:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.975+0000 7fa172d9d640 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f179e7b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa16c02abe0 tx=0x7fa16c02ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5259b845b772ced server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.979+0000 7fa15bfff640 1 -- 172.21.15.19:0/1951704537 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c04b070 con 0x5555f1775900 2024-09-12T03:23:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.979+0000 7fa15bfff640 1 -- 172.21.15.19:0/1951704537 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c030d30 con 0x5555f1775900 2024-09-12T03:23:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.979+0000 7fa15bfff640 1 -- 172.21.15.19:0/1951704537 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c036040 con 0x5555f1775900 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 msgr2=0x5555f179e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f179e7b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa16c02abe0 tx=0x7fa16c02ae60 comp rx=0 tx=0).stop 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 shutdown_connections 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f179e7b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1762490 0x5555f179e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.980+0000 7fa17b09b740 1 --2- 172.21.15.19:0/1951704537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1761b70 0x5555f1754a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.981+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 >> 172.21.15.19:0/1951704537 conn(0x5555f175e240 msgr2=0x5555f1754350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.981+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 shutdown_connections 2024-09-12T03:23:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.981+0000 7fa17b09b740 1 -- 172.21.15.19:0/1951704537 wait complete. 2024-09-12T03:23:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.982+0000 7fa17b09b740 1 -- start start 2024-09-12T03:23:16.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.982+0000 7fa17b09b740 1 Processor -- start 2024-09-12T03:23:16.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.983+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.984+0000 7fa17b09b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f17570b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.984+0000 7fa17259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.984+0000 7fa171d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f17570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.984+0000 7fa17259c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:60552/0 (socket says 172.21.15.19:60552) 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.984+0000 7fa17259c640 1 -- 172.21.15.19:0/4291064258 learned_addr learned my addr 172.21.15.19:0/4291064258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.985+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f17575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.985+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa174070f70 con 0x5555f1775900 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.985+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa1740708a0 con 0x5555f1762490 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.985+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa17406db90 con 0x5555f1761b70 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.985+0000 7fa17b09b740 10 client.0 _ll_register_callbacks cb 0x7fa1680095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.986+0000 7fa17259c640 1 -- 172.21.15.19:0/4291064258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 msgr2=0x5555f17570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:16.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.986+0000 7fa17259c640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f17570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.986+0000 7fa17259c640 1 -- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 msgr2=0x5555f17575f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.987+0000 7fa172d9d640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f17575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.987+0000 7fa17259c640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f17575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.987+0000 7fa17259c640 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f1757b30 con 0x5555f1761b70 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.988+0000 7fa172d9d640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f17575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.988+0000 7fa17259c640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa16401d7b0 tx=0x7fa164004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ef750a8a1c7d4d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:16.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.989+0000 7fa159ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:23:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.990+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa164004450 con 0x5555f1761b70 2024-09-12T03:23:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.990+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1640035f0 con 0x5555f1761b70 2024-09-12T03:23:16.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.990+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa16c030d30 con 0x5555f1761b70 2024-09-12T03:23:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.992+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1640037f0 con 0x5555f1761b70 2024-09-12T03:23:16.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.992+0000 7fa17b09b740 10 client.25702 fetch_fsmap learned FSMap version 30 2024-09-12T03:23:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.992+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555f1801000 con 0x5555f1761b70 2024-09-12T03:23:16.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa16c030d30 con 0x5555f1761b70 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa16402f030 con 0x5555f1761b70 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 10 client.25702 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 20 client.25702 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1801850 0x5555f1803c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 4 client.25702 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.993+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555f16c6d60 con 0x5555f1801850 2024-09-12T03:23:16.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.994+0000 7fa171d9b640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1801850 0x5555f1803c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.003+0000 7fa171d9b640 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1801850 0x5555f1803c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45e1e5dccc2adf8 server_cookie=b6b5e225634f57f2 in_seq=0 out_seq=0 2024-09-12T03:23:17.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.003+0000 7fa159ffb640 10 client.25702 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:17.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.007+0000 7fa159ffb640 1 -- 172.21.15.19:0/4291064258 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5555f16c6d60 con 0x5555f1801850 2024-09-12T03:23:17.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.007+0000 7fa159ffb640 10 client.25702 handle_command_reply: tid=0 2024-09-12T03:23:17.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.007+0000 7fa17b09b740 1 client.25702 shutdown 2024-09-12T03:23:17.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 msgr2=0x5555f177dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa16401d7b0 tx=0x7fa164004730 comp rx=0 tx=0).stop 2024-09-12T03:23:17.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 shutdown_connections 2024-09-12T03:23:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5555f1775900 0x5555f17575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5555f1801850 0x5555f1803c70 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5555f1762490 0x5555f17570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 --2- 172.21.15.19:0/4291064258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5555f1761b70 0x5555f177dac0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.012+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 >> 172.21.15.19:0/4291064258 conn(0x5555f175e240 msgr2=0x5555f175bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.013+0000 7fa159ffb640 0 client.25702 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:17.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.013+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 shutdown_connections 2024-09-12T03:23:17.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.013+0000 7fa17b09b740 1 -- 172.21.15.19:0/4291064258 wait complete. 2024-09-12T03:23:17.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.014+0000 7fa17b09b740 20 client.25702 trim_cache size 0 max 16384 2024-09-12T03:23:17.018 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:17.018 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:23:17.018 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:23:17.018 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:17.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.020+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 msgr2=0x7fa15c0a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.020+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 0x7fa15c0a5210 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa164009bd0 tx=0x7fa164005f90 comp rx=0 tx=0).stop 2024-09-12T03:23:17.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.020+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 msgr2=0x7fa17419ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.020+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa16800e9c0 tx=0x7fa16800ee80 comp rx=0 tx=0).stop 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 shutdown_connections 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa15c0a2d50 0x7fa15c0a5210 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa17410e4e0 0x7fa17419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa1741012e0 0x7fa17419ab70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 --2- 172.21.15.19:0/986175785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa174100930 0x7fa17419a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 >> 172.21.15.19:0/986175785 conn(0x7fa1740fc7c0 msgr2=0x7fa17410f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.023+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 shutdown_connections 2024-09-12T03:23:17.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.024+0000 7fa178cc2640 1 -- 172.21.15.19:0/986175785 wait complete. 2024-09-12T03:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 -- 172.21.15.19:0/439212414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0100e20 msgr2=0x7f77d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 --2- 172.21.15.19:0/439212414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0100e20 0x7f77d0101200 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002a480 comp rx=0 tx=0).stop 2024-09-12T03:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 -- 172.21.15.19:0/439212414 shutdown_connections 2024-09-12T03:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 --2- 172.21.15.19:0/439212414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 0x7f77d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 --2- 172.21.15.19:0/439212414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0102570 0x7f77d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 --2- 172.21.15.19:0/439212414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0100e20 0x7f77d0101200 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 -- 172.21.15.19:0/439212414 >> 172.21.15.19:0/439212414 conn(0x7f77d00fc7d0 msgr2=0x7f77d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 -- 172.21.15.19:0/439212414 shutdown_connections 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.762+0000 7f77d4ff5640 1 -- 172.21.15.19:0/439212414 wait complete. 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 Processor -- start 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 -- start start 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0100e20 0x7f77d0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 0x7f77d019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110fa0 con 0x7f77d0102570 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110e20 con 0x7f77d0100e20 2024-09-12T03:23:17.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.763+0000 7f77d4ff5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111120 con 0x7f77d0102ef0 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 0x7f77d019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58662/0 (socket says 172.21.15.19:58662) 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 -- 172.21.15.19:0/1675195670 learned_addr learned my addr 172.21.15.19:0/1675195670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77ce59c640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0100e20 0x7f77d0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 -- 172.21.15.19:0/1675195670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 msgr2=0x7f77d019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 0x7f77d019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 -- 172.21.15.19:0/1675195670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0100e20 msgr2=0x7f77d0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0100e20 0x7f77d0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 -- 172.21.15.19:0/1675195670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019d4c0 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77cdd9b640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f77b800c920 tx=0x7f77b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8007c70 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77d4ff5640 1 -- 172.21.15.19:0/1675195670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019d7b0 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b8007e10 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8018890 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.764+0000 7f77d4ff5640 1 -- 172.21.15.19:0/1675195670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a5090 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.765+0000 7f77d4ff5640 1 -- 172.21.15.19:0/1675195670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d0101200 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.767+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77b8018a50 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.767+0000 7f77c77fe640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 0x7f77bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.767+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f77b80cf360 con 0x7f77d0102570 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.767+0000 7f77ce59c640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 0x7f77bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.768+0000 7f77ce59c640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 0x7f77bc0a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:17.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:16.771+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77b8014030 con 0x7f77d0102570 2024-09-12T03:23:17.234 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:17 smithi019 ceph-mon[30953]: pgmap v464: 97 pgs: 97 active+clean; 934 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 976 op/s 2024-09-12T03:23:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.232+0000 7f77d4ff5640 1 -- 172.21.15.19:0/1675195670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77d0199870 con 0x7f77d0102570 2024-09-12T03:23:17.236 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:23:17.236 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":30,"btime":"2024-09-12T03:20:59:419518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:20:58.474782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:23:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 30 2024-09-12T03:23:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.235+0000 7f77c77fe640 1 -- 172.21.15.19:0/1675195670 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11981 (secure 0 0 0) 0x7f77d0199870 con 0x7f77d0102570 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.240+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 msgr2=0x7f77bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 0x7f77bc0a5040 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0005b50 comp rx=0 tx=0).stop 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 msgr2=0x7f77d0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f77b800c920 tx=0x7f77b800cde0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 shutdown_connections 2024-09-12T03:23:17.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77d0102ef0 0x7f77d019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77bc0a2b80 0x7f77bc0a5040 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77d0102570 0x7f77d0198a50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 --2- 172.21.15.19:0/1675195670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77d0100e20 0x7f77d0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 >> 172.21.15.19:0/1675195670 conn(0x7f77d00fc7d0 msgr2=0x7f77d006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.241+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 shutdown_connections 2024-09-12T03:23:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:17.242+0000 7f77c57fa640 1 -- 172.21.15.19:0/1675195670 wait complete. 2024-09-12T03:23:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:17 smithi135 ceph-mon[35474]: pgmap v464: 97 pgs: 97 active+clean; 934 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 976 op/s 2024-09-12T03:23:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:17 smithi121 ceph-mon[34963]: pgmap v464: 97 pgs: 97 active+clean; 934 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 976 op/s 2024-09-12T03:23:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:18 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1675195670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:18.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:18 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1675195670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:18.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:18 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1675195670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:18.696 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:23:18.697 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:19.097 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:23:19.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:23:19.316 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:19 smithi019 ceph-mon[30953]: pgmap v465: 97 pgs: 97 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 24 MiB/s wr, 1.00k op/s 2024-09-12T03:23:19.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:19 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:19 smithi135 ceph-mon[35474]: pgmap v465: 97 pgs: 97 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 24 MiB/s wr, 1.00k op/s 2024-09-12T03:23:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:19 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:19 smithi121 ceph-mon[34963]: pgmap v465: 97 pgs: 97 active+clean; 951 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 24 MiB/s wr, 1.00k op/s 2024-09-12T03:23:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:19 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:20.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 -- 172.21.15.19:0/1110996173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc101de0 msgr2=0x7f72fc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 --2- 172.21.15.19:0/1110996173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc101de0 0x7f72fc1021c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009900 tx=0x7f72f802a410 comp rx=0 tx=0).stop 2024-09-12T03:23:20.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 -- 172.21.15.19:0/1110996173 shutdown_connections 2024-09-12T03:23:20.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 --2- 172.21.15.19:0/1110996173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc106f10 0x7f72fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 --2- 172.21.15.19:0/1110996173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 --2- 172.21.15.19:0/1110996173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc101de0 0x7f72fc1021c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 -- 172.21.15.19:0/1110996173 >> 172.21.15.19:0/1110996173 conn(0x7f72fc0fd8c0 msgr2=0x7f72fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 -- 172.21.15.19:0/1110996173 shutdown_connections 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.020+0000 7f73037b5640 1 -- 172.21.15.19:0/1110996173 wait complete. 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.021+0000 7f73037b5640 1 Processor -- start 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.021+0000 7f73037b5640 1 -- start start 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 0x7f72fc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111f70 con 0x7f72fc106f10 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc111df0 con 0x7f72fc101de0 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f73037b5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc1120f0 con 0x7f72fc102790 2024-09-12T03:23:20.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7301551640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7301d52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 0x7f72fc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7301551640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39866/0 (socket says 172.21.15.19:39866) 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7301551640 1 -- 172.21.15.19:0/3176315719 learned_addr learned my addr 172.21.15.19:0/3176315719 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7300d50640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.022+0000 7f7301d52640 1 -- 172.21.15.19:0/3176315719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 msgr2=0x7f72fc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301d52640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301d52640 1 -- 172.21.15.19:0/3176315719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 msgr2=0x7f72fc198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301d52640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301d52640 1 -- 172.21.15.19:0/3176315719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc19d3c0 con 0x7f72fc106f10 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301551640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7300d50640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:20.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f7301d52640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 0x7f72fc19cce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f72f400b540 tx=0x7f72f400ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f4004b20 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72f4004cc0 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc19d6b0 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f401dad0 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.023+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc074420 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.024+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc1021c0 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.027+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72f4024020 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.028+0000 7f72f27fc640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 0x7f72e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.028+0000 7f7301551640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 0x7f72e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.028+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f72f40d77a0 con 0x7f72fc106f10 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.028+0000 7f7301551640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 0x7f72e80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72f80096c0 tx=0x7f72f8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:20.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.035+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f72f402b070 con 0x7f72fc106f10 2024-09-12T03:23:20.096 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.289+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72fc10a0f0 con 0x7f72fc106f10 2024-09-12T03:23:20.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.290+0000 7f72f27fc640 1 -- 172.21.15.19:0/3176315719 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11982 (secure 0 0 0) 0x7f72f40a4710 con 0x7f72fc106f10 2024-09-12T03:23:20.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:23:20.291 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":31,"btime":"2024-09-12T03:23:18:176010+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:23:18.152905+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:23:20.291 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 31 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 msgr2=0x7f72e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 0x7f72e80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f72f80096c0 tx=0x7f72f8005a20 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 msgr2=0x7f72fc19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 0x7f72fc19cce0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f72f400b540 tx=0x7f72f400ba00 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 shutdown_connections 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f72e80a2c00 0x7f72e80a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f72fc106f10 0x7f72fc19cce0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f72fc102790 0x7f72fc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.292+0000 7f73037b5640 1 --2- 172.21.15.19:0/3176315719 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f72fc101de0 0x7f72fc198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.293+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 >> 172.21.15.19:0/3176315719 conn(0x7f72fc0fd8c0 msgr2=0x7f72fc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:20.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.293+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 shutdown_connections 2024-09-12T03:23:20.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:20.293+0000 7f73037b5640 1 -- 172.21.15.19:0/3176315719 wait complete. 2024-09-12T03:23:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:21 smithi135 ceph-mon[35474]: pgmap v466: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 891 op/s 2024-09-12T03:23:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:21 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3176315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:21.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:21 smithi121 ceph-mon[34963]: pgmap v466: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 891 op/s 2024-09-12T03:23:21.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:21 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3176315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:21.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:21 smithi019 ceph-mon[30953]: pgmap v466: 97 pgs: 97 active+clean; 975 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 21 MiB/s wr, 891 op/s 2024-09-12T03:23:21.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:21 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3176315719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 -- 172.21.15.19:0/2595817896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80100e20 msgr2=0x7f0f80101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 --2- 172.21.15.19:0/2595817896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80100e20 0x7f0f80101200 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009900 tx=0x7f0f7002a480 comp rx=0 tx=0).stop 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 -- 172.21.15.19:0/2595817896 shutdown_connections 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 --2- 172.21.15.19:0/2595817896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102ef0 0x7f0f8010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 --2- 172.21.15.19:0/2595817896 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80102570 0x7f0f801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 --2- 172.21.15.19:0/2595817896 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80100e20 0x7f0f80101200 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 -- 172.21.15.19:0/2595817896 >> 172.21.15.19:0/2595817896 conn(0x7f0f800fc7d0 msgr2=0x7f0f800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.770+0000 7f0f86396640 1 -- 172.21.15.19:0/2595817896 shutdown_connections 2024-09-12T03:23:21.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.771+0000 7f0f86396640 1 -- 172.21.15.19:0/2595817896 wait complete. 2024-09-12T03:23:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.771+0000 7f0f86396640 1 Processor -- start 2024-09-12T03:23:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.771+0000 7f0f86396640 1 -- start start 2024-09-12T03:23:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80100e20 0x7f0f800fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80110f80 con 0x7f0f80102ef0 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80110e00 con 0x7f0f80102570 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f86396640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f80111100 con 0x7f0f80100e20 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f84933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42688/0 (socket says 172.21.15.19:42688) 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/333939316 learned_addr learned my addr 172.21.15.19:0/333939316 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f84933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37722/0 (socket says 172.21.15.19:37722) 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80100e20 0x7f0f800fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/333939316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80100e20 msgr2=0x7f0f800fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80100e20 0x7f0f800fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 msgr2=0x7f0f800fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/333939316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f801a69a0 con 0x7f0f80102570 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f84933640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.772+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d770 tx=0x7f0f7400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.773+0000 7f0f7d7fa640 1 -- 172.21.15.19:0/333939316 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74017070 con 0x7f0f80102570 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.773+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f801a6c90 con 0x7f0f80102570 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.773+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f801a7180 con 0x7f0f80102570 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.773+0000 7f0f7d7fa640 1 -- 172.21.15.19:0/333939316 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f74009c30 con 0x7f0f80102570 2024-09-12T03:23:21.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.773+0000 7f0f7d7fa640 1 -- 172.21.15.19:0/333939316 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74020060 con 0x7f0f80102570 2024-09-12T03:23:21.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 Processor -- start 2024-09-12T03:23:21.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 -- start start 2024-09-12T03:23:21.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f7d7fa640 1 -- 172.21.15.19:0/333939316 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0f74010460 con 0x7f0f80102570 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 0x5646c52dfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646c5237160 con 0x5646c52d3170 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646c51de720 con 0x5646c52d2850 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f8876f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646c5233120 con 0x5646c52dda30 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f84933640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f7ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f84933640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42696/0 (socket says 172.21.15.19:42696) 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f84933640 1 -- 172.21.15.19:0/3936231701 learned_addr learned my addr 172.21.15.19:0/3936231701 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.776+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 0x5646c52dfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 -- 172.21.15.19:0/3936231701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 msgr2=0x5646c52dfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 0x5646c52dfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 -- 172.21.15.19:0/3936231701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 msgr2=0x5646c52dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52dd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 -- 172.21.15.19:0/3936231701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c4fa8ba0 con 0x5646c52d2850 2024-09-12T03:23:21.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 0x5646c52dfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f84933640 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011790 tx=0x7f0f78011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168e100b3746de9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f7d7fa640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 0x7f0f680a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f667fc640 1 -- 172.21.15.19:0/3936231701 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78003c30 con 0x5646c52d2850 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f667fc640 1 -- 172.21.15.19:0/3936231701 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7800fe80 con 0x5646c52d2850 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 0x7f0f680a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f667fc640 1 -- 172.21.15.19:0/3936231701 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f780193e0 con 0x5646c52d2850 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.777+0000 7f0f7d7fa640 1 -- 172.21.15.19:0/333939316 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0f740ce8e0 con 0x7f0f80102570 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- 172.21.15.19:0/3936231701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 msgr2=0x5646c52d2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 0x7f0f680a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009230 tx=0x7f0f700058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011790 tx=0x7f0f78011c50 comp rx=0 tx=0).stop 2024-09-12T03:23:21.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- 172.21.15.19:0/3936231701 shutdown_connections 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52dda30 0x5646c52dfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 --2- 172.21.15.19:0/3936231701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52d2c30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- 172.21.15.19:0/3936231701 >> 172.21.15.19:0/3936231701 conn(0x5646c52cede0 msgr2=0x5646c52cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- 172.21.15.19:0/3936231701 shutdown_connections 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- 172.21.15.19:0/3936231701 wait complete. 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.778+0000 7f0f8876f740 1 -- start start 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 Processor -- start 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52da970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d3170 0x5646c52daeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646c52e0600 con 0x5646c52dda30 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646c51bf900 con 0x5646c52d2850 2024-09-12T03:23:21.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646c52e0780 con 0x5646c52d3170 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f8876f740 10 client.0 _ll_register_callbacks cb 0x5646c51bd780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f7ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52da970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f84933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f7ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52da970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42698/0 (socket says 172.21.15.19:42698) 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f7ffff640 1 -- 172.21.15.19:0/814264468 learned_addr learned my addr 172.21.15.19:0/814264468 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.779+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d3170 0x5646c52daeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 -- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d3170 msgr2=0x5646c52daeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d3170 0x5646c52daeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 -- 172.21.15.19:0/814264468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 msgr2=0x5646c52da970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52da970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 -- 172.21.15.19:0/814264468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c51ffc50 con 0x5646c52dda30 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f84933640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52db3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011790 tx=0x7f0f78004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddf0a1624a382d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78028020 con 0x5646c52dda30 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646c5244890 con 0x5646c52dda30 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0f7800fba0 con 0x5646c52dda30 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f7801f040 con 0x5646c52dda30 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f8876f740 10 client.16188 fetch_fsmap learned FSMap version 31 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5646c52f4bd0 con 0x5646c52dda30 2024-09-12T03:23:21.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.780+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5646c5244890 con 0x5646c52dda30 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0f78018070 con 0x5646c52dda30 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 10 client.16188 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 10 client.16188 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 20 client.16188 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c52f5fe0 0x5646c52f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 4 client.16188 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c523ebe0 con 0x5646c52f5fe0 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c52f5fe0 0x5646c52f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f5f7fe640 10 client.16188 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.781+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c52f5fe0 0x5646c52f83d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da78ce707892ae9b server_cookie=be519b46460ec64a in_seq=0 out_seq=0 2024-09-12T03:23:21.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/814264468 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646c523ebe0 con 0x5646c52f5fe0 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f5f7fe640 10 client.16188 handle_command_reply: tid=0 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f8876f740 1 client.16188 shutdown 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 msgr2=0x5646c52db3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52db3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011790 tx=0x7f0f78004990 comp rx=0 tx=0).stop 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.782+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 shutdown_connections 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52db3f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d3170 0x5646c52daeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c52f5fe0 0x5646c52f83d0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 --2- 172.21.15.19:0/814264468 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d2850 0x5646c52da970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 >> 172.21.15.19:0/814264468 conn(0x5646c52cede0 msgr2=0x5646c52d4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f5f7fe640 0 client.16188 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 shutdown_connections 2024-09-12T03:23:21.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 1 -- 172.21.15.19:0/814264468 wait complete. 2024-09-12T03:23:21.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.783+0000 7f0f8876f740 20 client.16188 trim_cache size 0 max 16384 2024-09-12T03:23:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 Processor -- start 2024-09-12T03:23:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 -- start start 2024-09-12T03:23:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52d49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d3170 0x5646c52fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00e3f0 con 0x5646c52dda30 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00fff0 con 0x5646c52d3170 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f8876f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00eac0 con 0x5646c52d2850 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f84933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f84933640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37746/0 (socket says 172.21.15.19:37746) 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f84933640 1 -- 172.21.15.19:0/1969415842 learned_addr learned my addr 172.21.15.19:0/1969415842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d3170 0x5646c52fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.804+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52d49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 -- 172.21.15.19:0/1969415842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 msgr2=0x5646c52d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52d49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 -- 172.21.15.19:0/1969415842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d3170 msgr2=0x5646c52fdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d3170 0x5646c52fdbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 -- 172.21.15.19:0/1969415842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c52d9920 con 0x5646c52dda30 2024-09-12T03:23:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f84933640 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011c90 tx=0x7f0f78005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fc0dadbfdf7bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f65ffb640 1 -- 172.21.15.19:0/1969415842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78028040 con 0x5646c52dda30 2024-09-12T03:23:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f65ffb640 1 -- 172.21.15.19:0/1969415842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78009e00 con 0x5646c52dda30 2024-09-12T03:23:21.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.805+0000 7f0f65ffb640 1 -- 172.21.15.19:0/1969415842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800f840 con 0x5646c52dda30 2024-09-12T03:23:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- 172.21.15.19:0/1969415842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 msgr2=0x5646c52fe0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0f78011c90 tx=0x7f0f78005d10 comp rx=0 tx=0).stop 2024-09-12T03:23:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- 172.21.15.19:0/1969415842 shutdown_connections 2024-09-12T03:23:21.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52dda30 0x5646c52fe0f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52d3170 0x5646c52fdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 --2- 172.21.15.19:0/1969415842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52d49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- 172.21.15.19:0/1969415842 >> 172.21.15.19:0/1969415842 conn(0x5646c52cede0 msgr2=0x5646c52d42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- 172.21.15.19:0/1969415842 shutdown_connections 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- 172.21.15.19:0/1969415842 wait complete. 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 -- start start 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.806+0000 7f0f8876f740 1 Processor -- start 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52dda30 0x5646c52eac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00f860 con 0x5646c52d3170 2024-09-12T03:23:21.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00f190 con 0x5646c52dda30 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6c00c3d0 con 0x5646c52d2850 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f8876f740 10 client.0 _ll_register_callbacks cb 0x7f0f70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37756/0 (socket says 172.21.15.19:37756) 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/280093657 learned_addr learned my addr 172.21.15.19:0/280093657 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f84933640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52dda30 0x5646c52eac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 msgr2=0x5646c52ea1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52ea1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/280093657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52dda30 msgr2=0x5646c52eac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52dda30 0x5646c52eac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 -- 172.21.15.19:0/280093657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c52f8740 con 0x5646c52d3170 2024-09-12T03:23:21.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f7f7fe640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f7409b190 tx=0x7f0f7409ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d36232905b874b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.807+0000 7f0f5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f74097870 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0f740bb0b0 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f78009e00 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f740d0e60 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 10 client.16194 fetch_fsmap learned FSMap version 31 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5646c5360e20 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78009e00 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7f0f7409a650 con 0x5646c52d3170 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 10 client.16194 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 10 client.16194 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 20 client.16194 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:21.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c5361750 0x5646c5363b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 4 client.16194 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:23:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.808+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5646c52ca410 con 0x5646c5361750 2024-09-12T03:23:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.809+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c5361750 0x5646c5363b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.809+0000 7f0f5f7fe640 10 client.16194 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:21.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:21.809+0000 7f0f7ffff640 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c5361750 0x5646c5363b70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff8a91f4c6c26814 server_cookie=641ae3e9b2680cec in_seq=0 out_seq=0 2024-09-12T03:23:21.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:23:22.237 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:22.362 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:22 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:22 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:22.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:22 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 -- 172.21.15.19:0/3389965034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0105590 msgr2=0x7fdbe010d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 --2- 172.21.15.19:0/3389965034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0105590 0x7fdbe010d110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800aa10 tx=0x7fdbd802b720 comp rx=0 tx=0).stop 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 -- 172.21.15.19:0/3389965034 shutdown_connections 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 --2- 172.21.15.19:0/3389965034 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0105590 0x7fdbe010d110 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 --2- 172.21.15.19:0/3389965034 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 0x7fdbe0104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 --2- 172.21.15.19:0/3389965034 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0100460 0x7fdbe0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.877+0000 7fdbe838b640 1 -- 172.21.15.19:0/3389965034 >> 172.21.15.19:0/3389965034 conn(0x7fdbe00fbee0 msgr2=0x7fdbe00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.878+0000 7fdbe838b640 1 -- 172.21.15.19:0/3389965034 shutdown_connections 2024-09-12T03:23:22.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.878+0000 7fdbe838b640 1 -- 172.21.15.19:0/3389965034 wait complete. 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.878+0000 7fdbe838b640 1 Processor -- start 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.878+0000 7fdbe838b640 1 -- start start 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 0x7fdbe019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0105590 0x7fdbe019ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fdc0 con 0x7fdbe0100460 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010fc40 con 0x7fdbe0105590 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe010ff40 con 0x7fdbe0100e10 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37772/0 (socket says 172.21.15.19:37772) 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 -- 172.21.15.19:0/1027894784 learned_addr learned my addr 172.21.15.19:0/1027894784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:22.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0105590 0x7fdbe019ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe5926640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 0x7fdbe019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 -- 172.21.15.19:0/1027894784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 msgr2=0x7fdbe019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 0x7fdbe019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 -- 172.21.15.19:0/1027894784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0105590 msgr2=0x7fdbe019ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0105590 0x7fdbe019ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 -- 172.21.15.19:0/1027894784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbe019f570 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00ebc0 tx=0x7fdbdc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbd77fe640 1 -- 172.21.15.19:0/1027894784 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc010630 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbe019f860 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.879+0000 7fdbd77fe640 1 -- 172.21.15.19:0/1027894784 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc010cb0 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.880+0000 7fdbd77fe640 1 -- 172.21.15.19:0/1027894784 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc018bd0 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.880+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbe01a7140 con 0x7fdbe0100460 2024-09-12T03:23:22.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.882+0000 7fdbd77fe640 1 -- 172.21.15.19:0/1027894784 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdbdc0107d0 con 0x7fdbe0100460 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.882+0000 7fdbea764740 1 Processor -- start 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 -- start start 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 0x555564d2a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2a5d0 0x555564d2aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555564c902f0 con 0x555564d35370 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555564c37cf0 con 0x555564d2bb40 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbea764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555564c8c2b0 con 0x555564d2a5d0 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbd77fe640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 0x7fdbcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 0x555564d2a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe5926640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe5926640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37786/0 (socket says 172.21.15.19:37786) 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 0x555564d2a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42750/0 (socket says 172.21.15.19:42750) 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe5926640 1 -- 172.21.15.19:0/2426365943 learned_addr learned my addr 172.21.15.19:0/2426365943 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe6127640 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2a5d0 0x555564d2aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbd77fe640 1 -- 172.21.15.19:0/1027894784 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdbdc0cf650 con 0x7fdbe0100460 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.883+0000 7fdbe5926640 1 -- 172.21.15.19:0/2426365943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2a5d0 msgr2=0x555564d2aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe5926640 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2a5d0 0x555564d2aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe5926640 1 -- 172.21.15.19:0/2426365943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 msgr2=0x555564d2a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe5926640 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 0x555564d2a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe5926640 1 -- 172.21.15.19:0/2426365943 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555564c1c3a0 con 0x555564d35370 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 0x7fdbcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbe5926640 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bc10 tx=0x7fdbd00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e72c5100808a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbd4dea640 1 -- 172.21.15.19:0/2426365943 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0005020 con 0x555564d35370 2024-09-12T03:23:22.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbd4dea640 1 -- 172.21.15.19:0/2426365943 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd00047a0 con 0x555564d35370 2024-09-12T03:23:22.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.884+0000 7fdbd4dea640 1 -- 172.21.15.19:0/2426365943 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd0011760 con 0x555564d35370 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- 172.21.15.19:0/2426365943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 msgr2=0x555564d37760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 0x7fdbcc0a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8030660 tx=0x7fdbd800a370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdbd000bc10 tx=0x7fdbd00107a0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- 172.21.15.19:0/2426365943 shutdown_connections 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d37760 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2a5d0 0x555564d2aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 --2- 172.21.15.19:0/2426365943 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2bb40 0x555564d2a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- 172.21.15.19:0/2426365943 >> 172.21.15.19:0/2426365943 conn(0x555564cf3310 msgr2=0x555564d26c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- 172.21.15.19:0/2426365943 shutdown_connections 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- 172.21.15.19:0/2426365943 wait complete. 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.885+0000 7fdbea764740 1 -- start start 2024-09-12T03:23:22.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 Processor -- start 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d34d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d32010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555564d38070 con 0x555564d35370 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555564c4b060 con 0x555564d2a5d0 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555564d381f0 con 0x555564d2bb40 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbea764740 10 client.0 _ll_register_callbacks cb 0x55556499e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:22.929 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6127640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d34d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37798/0 (socket says 172.21.15.19:37798) 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 -- 172.21.15.19:0/3891051568 learned_addr learned my addr 172.21.15.19:0/3891051568 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe5926640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d32010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 -- 172.21.15.19:0/3891051568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 msgr2=0x555564d32010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d32010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 -- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 msgr2=0x555564d34d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d34d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6928640 1 -- 172.21.15.19:0/3891051568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555564a013c0 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.886+0000 7fdbe6127640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d34d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbe6928640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800aa10 tx=0x7fdbd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1730328b5dcc5843 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8036040 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbd00047a0 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdbd80037f0 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbd8059050 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbea764740 10 client.16206 fetch_fsmap learned FSMap version 31 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555564d43b60 con 0x555564d35370 2024-09-12T03:23:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.887+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbd00047a0 con 0x555564d35370 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdbd803b040 con 0x555564d35370 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 10 client.16206 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 10 client.16206 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 20 client.16206 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564d45020 0x555564d474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 4 client.16206 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555564c9d860 con 0x555564d45020 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbe6127640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564d45020 0x555564d474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbe6127640 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564d45020 0x555564d474c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3a3a8c19ca9e8c8b server_cookie=33020d664b740ac0 in_seq=0 out_seq=0 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.888+0000 7fdbc9ffb640 10 client.16206 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/3891051568 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555564c9d860 con 0x555564d45020 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbc9ffb640 10 client.16206 handle_command_reply: tid=0 2024-09-12T03:23:22.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 client.16206 shutdown 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 msgr2=0x555564d32600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800aa10 tx=0x7fdbd80046c0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 shutdown_connections 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d32600 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564d45020 0x555564d474c0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d32010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 --2- 172.21.15.19:0/3891051568 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d34d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 >> 172.21.15.19:0/3891051568 conn(0x555564cf3310 msgr2=0x555564d2c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.891+0000 7fdbc9ffb640 0 client.16206 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.892+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 shutdown_connections 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.892+0000 7fdbea764740 1 -- 172.21.15.19:0/3891051568 wait complete. 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.892+0000 7fdbea764740 20 client.16206 trim_cache size 0 max 16384 2024-09-12T03:23:22.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 Processor -- start 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 -- start start 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d26e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d35370 0x555564d41310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01af6b0 con 0x555564d2a5d0 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01b12b0 con 0x555564d35370 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbea764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01afd80 con 0x555564d2bb40 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37808/0 (socket says 172.21.15.19:37808) 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 -- 172.21.15.19:0/2587542992 learned_addr learned my addr 172.21.15.19:0/2587542992 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6928640 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d35370 0x555564d41310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 -- 172.21.15.19:0/2587542992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 msgr2=0x555564d26e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d26e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 -- 172.21.15.19:0/2587542992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d35370 msgr2=0x555564d41310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d35370 0x555564d41310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 -- 172.21.15.19:0/2587542992 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555564d30fc0 con 0x555564d2a5d0 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.908+0000 7fdbe6127640 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00cba0 tx=0x7fdbdc0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226e60026f096d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.909+0000 7fdbcbfff640 1 -- 172.21.15.19:0/2587542992 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc010e60 con 0x555564d2a5d0 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.909+0000 7fdbcbfff640 1 -- 172.21.15.19:0/2587542992 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc0a0270 con 0x555564d2a5d0 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.909+0000 7fdbcbfff640 1 -- 172.21.15.19:0/2587542992 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc09b3c0 con 0x555564d2a5d0 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.909+0000 7fdbea764740 1 -- 172.21.15.19:0/2587542992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 msgr2=0x555564d21c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.909+0000 7fdbea764740 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00cba0 tx=0x7fdbdc0cf040 comp rx=0 tx=0).stop 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 -- 172.21.15.19:0/2587542992 shutdown_connections 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d35370 0x555564d41310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d26e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 --2- 172.21.15.19:0/2587542992 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d2a5d0 0x555564d21c60 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 -- 172.21.15.19:0/2587542992 >> 172.21.15.19:0/2587542992 conn(0x555564cf3310 msgr2=0x555564d26700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 -- 172.21.15.19:0/2587542992 shutdown_connections 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 -- 172.21.15.19:0/2587542992 wait complete. 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 -- start start 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.910+0000 7fdbea764740 1 Processor -- start 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d3da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d2eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01b0b20 con 0x555564d35370 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01b0450 con 0x555564d2a5d0 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdbe01ad690 con 0x555564d2bb40 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 10 client.0 _ll_register_callbacks cb 0x7fdbd8004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37824/0 (socket says 172.21.15.19:37824) 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 -- 172.21.15.19:0/1829659810 learned_addr learned my addr 172.21.15.19:0/1829659810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d3da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe5926640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d2eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 -- 172.21.15.19:0/1829659810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 msgr2=0x555564d2eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d2eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 -- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 msgr2=0x555564d3da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d3da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 -- 172.21.15.19:0/1829659810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555564d2f6a0 con 0x555564d35370 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbe6928640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004770 tx=0x7fdbd803d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0e7e4df0d52b40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8036070 con 0x555564d35370 2024-09-12T03:23:22.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdbd804a630 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.911+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbdc0a0270 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fdbd800d210 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 10 client.16212 fetch_fsmap learned FSMap version 31 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555564dafc00 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdbdc0a0270 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 31) ==== 8382+0+0 (secure 0 0 0) 0x7fdbd800b040 con 0x555564d35370 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 10 client.16212 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 10 client.16212 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 20 client.16212 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564db02c0 0x555564db26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 4 client.16212 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555564c97d70 con 0x555564db02c0 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.912+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564db02c0 0x555564db26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:22.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.913+0000 7fdbe6127640 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564db02c0 0x555564db26b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bf998d5975ba7a18 server_cookie=e9462292d1fab743 in_seq=0 out_seq=0 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.913+0000 7fdbc9ffb640 10 client.16212 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbc9ffb640 1 -- 172.21.15.19:0/1829659810 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555564c97d70 con 0x555564db02c0 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbc9ffb640 10 client.16212 handle_command_reply: tid=0 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 client.16212 shutdown 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 msgr2=0x555564d2f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8004770 tx=0x7fdbd803d460 comp rx=0 tx=0).stop 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 shutdown_connections 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555564d35370 0x555564d2f060 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x555564db02c0 0x555564db26b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555564d2bb40 0x555564d2eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 --2- 172.21.15.19:0/1829659810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555564d2a5d0 0x555564d3da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbc9ffb640 0 client.16212 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 >> 172.21.15.19:0/1829659810 conn(0x555564cf3310 msgr2=0x555564d254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 shutdown_connections 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 1 -- 172.21.15.19:0/1829659810 wait complete. 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.916+0000 7fdbea764740 20 client.16212 trim_cache size 0 max 16384 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 msgr2=0x7fdbcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 0x7fdbcc0a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8030660 tx=0x7fdbd800a370 comp rx=0 tx=0).stop 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 msgr2=0x7fdbe019a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00ebc0 tx=0x7fdbdc00c6a0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 shutdown_connections 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdbe0105590 0x7fdbe019ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdbe0100e10 0x7fdbe019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdbcc0a2b80 0x7fdbcc0a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 --2- 172.21.15.19:0/1027894784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdbe0100460 0x7fdbe019a5c0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 >> 172.21.15.19:0/1027894784 conn(0x7fdbe00fbee0 msgr2=0x7fdbe00fcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 shutdown_connections 2024-09-12T03:23:22.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:22.917+0000 7fdbe838b640 1 -- 172.21.15.19:0/1027894784 wait complete. 2024-09-12T03:23:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:23 smithi135 ceph-mon[35474]: pgmap v467: 97 pgs: 97 active+clean; 983 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 19 MiB/s wr, 804 op/s 2024-09-12T03:23:23.550 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:23 smithi121 ceph-mon[34963]: pgmap v467: 97 pgs: 97 active+clean; 983 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 19 MiB/s wr, 804 op/s 2024-09-12T03:23:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:23 smithi019 ceph-mon[30953]: pgmap v467: 97 pgs: 97 active+clean; 983 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 19 MiB/s wr, 804 op/s 2024-09-12T03:23:23.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:23:24.136 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:24 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:24 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:23:24.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:24 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:24.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:24.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:24.554 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:24 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:23:24.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:24 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:23:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:23:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:23:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:24 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:23:24.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 -- 172.21.15.19:0/3243206241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 msgr2=0x7fe71c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 --2- 172.21.15.19:0/3243206241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c10d950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe714009cd0 tx=0x7fe71402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 -- 172.21.15.19:0/3243206241 shutdown_connections 2024-09-12T03:23:24.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 --2- 172.21.15.19:0/3243206241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c10d950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 --2- 172.21.15.19:0/3243206241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c100a80 0x7fe71c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 --2- 172.21.15.19:0/3243206241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c106a80 0x7fe71c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 -- 172.21.15.19:0/3243206241 >> 172.21.15.19:0/3243206241 conn(0x7fe71c0fc7c0 msgr2=0x7fe71c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 -- 172.21.15.19:0/3243206241 shutdown_connections 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.651+0000 7fe720857640 1 -- 172.21.15.19:0/3243206241 wait complete. 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.652+0000 7fe720857640 1 Processor -- start 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.652+0000 7fe720857640 1 -- start start 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c106a80 0x7fe71c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c110e80 con 0x7fe71c101420 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c110d00 con 0x7fe71c100a80 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe720857640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c111000 con 0x7fe71c106a80 2024-09-12T03:23:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42806/0 (socket says 172.21.15.19:42806) 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 -- 172.21.15.19:0/3406043132 learned_addr learned my addr 172.21.15.19:0/3406043132 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe719d9b640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c106a80 0x7fe71c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 -- 172.21.15.19:0/3406043132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c106a80 msgr2=0x7fe71c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c106a80 0x7fe71c19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 -- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 msgr2=0x7fe71c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe71a59c640 1 -- 172.21.15.19:0/3406043132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c19d3e0 con 0x7fe71c100a80 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.653+0000 7fe719d9b640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe71a59c640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9f0 tx=0x7fe70c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe70b7fe640 1 -- 172.21.15.19:0/3406043132 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c00cd80 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c19d6d0 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c1a4fb0 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe70b7fe640 1 -- 172.21.15.19:0/3406043132 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c010910 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.654+0000 7fe70b7fe640 1 -- 172.21.15.19:0/3406043132 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0188f0 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.657+0000 7fe70b7fe640 1 -- 172.21.15.19:0/3406043132 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe70c018a90 con 0x7fe71c100a80 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.657+0000 7fe722c30740 1 Processor -- start 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.657+0000 7fe722c30740 1 -- start start 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d9b40 0x55a4206d8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a42063e2f0 con 0x55a4206d85d0 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a4205e5cf0 con 0x55a4206e7780 2024-09-12T03:23:24.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe722c30740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a42063a2b0 con 0x55a4206d9b40 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42814/0 (socket says 172.21.15.19:42814) 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 -- 172.21.15.19:0/1484032040 learned_addr learned my addr 172.21.15.19:0/1484032040 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d9b40 0x55a4206d8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe70b7fe640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 0x7fe7040a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 -- 172.21.15.19:0/1484032040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d9b40 msgr2=0x55a4206d8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d9b40 0x55a4206d8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 -- 172.21.15.19:0/1484032040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 msgr2=0x55a4206d8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206d8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.658+0000 7fe719d9b640 1 -- 172.21.15.19:0/1484032040 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4205ca3a0 con 0x55a4206e7780 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe70b7fe640 1 -- 172.21.15.19:0/3406043132 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe70c0cf130 con 0x7fe71c100a80 2024-09-12T03:23:24.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 0x7fe7040a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7100114f0 tx=0x7fe7100119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fd98e0a88951fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe708ff9640 1 -- 172.21.15.19:0/1484032040 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710004020 con 0x55a4206e7780 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe708ff9640 1 -- 172.21.15.19:0/1484032040 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7100041c0 con 0x55a4206e7780 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.659+0000 7fe708ff9640 1 -- 172.21.15.19:0/1484032040 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe71000f950 con 0x55a4206e7780 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 -- 172.21.15.19:0/1484032040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 msgr2=0x55a4206e9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 0x7fe7040a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe714004ab0 tx=0x7fe71402a8e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7100114f0 tx=0x7fe7100119b0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 -- 172.21.15.19:0/1484032040 shutdown_connections 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e9b70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206d8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 --2- 172.21.15.19:0/1484032040 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d9b40 0x55a4206d8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 -- 172.21.15.19:0/1484032040 >> 172.21.15.19:0/1484032040 conn(0x55a4206a1310 msgr2=0x55a4206d4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 -- 172.21.15.19:0/1484032040 shutdown_connections 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.660+0000 7fe722c30740 1 -- 172.21.15.19:0/1484032040 wait complete. 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 -- start start 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 Processor -- start 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d85d0 0x55a4206e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 0x55a4206e4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a4206ea480 con 0x55a4206e7780 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a4205f9060 con 0x55a4206d9b40 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a4206ea600 con 0x55a4206d85d0 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe722c30740 10 client.0 _ll_register_callbacks cb 0x55a42034c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe719d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe719d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42830/0 (socket says 172.21.15.19:42830) 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.661+0000 7fe719d9b640 1 -- 172.21.15.19:0/425890051 learned_addr learned my addr 172.21.15.19:0/425890051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe71a59c640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d85d0 0x55a4206e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 -- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d85d0 msgr2=0x55a4206e7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d85d0 0x55a4206e7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 -- 172.21.15.19:0/425890051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 msgr2=0x55a4206e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 0x55a4206e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 0x55a4206e4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 -- 172.21.15.19:0/425890051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4203af3c0 con 0x55a4206d9b40 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 0x55a4206e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe719d9b640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7100114c0 tx=0x7fe710009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72467acf19392a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.662+0000 7fe701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710028020 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7100041c0 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe710016040 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe710003a20 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe722c30740 10 client.25733 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a4206f5ec0 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.663+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7100041c0 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fe710004cc0 con 0x55a4206d9b40 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 10 client.25733 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 10 client.25733 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 20 client.25733 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4206f72d0 0x55a4206f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 4 client.25733 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.664+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a42064b860 con 0x55a4206f72d0 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.665+0000 7fe71a59c640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4206f72d0 0x55a4206f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.666+0000 7fe71a59c640 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4206f72d0 0x55a4206f96c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10716b1eca9a267b server_cookie=53fdd96eada5c231 in_seq=0 out_seq=0 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.666+0000 7fe701ffb640 10 client.25733 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe701ffb640 1 -- 172.21.15.19:0/425890051 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a42064b860 con 0x55a4206f72d0 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe701ffb640 10 client.25733 handle_command_reply: tid=0 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 client.25733 shutdown 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 msgr2=0x55a4206e4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7100114c0 tx=0x7fe710009d20 comp rx=0 tx=0).stop 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 shutdown_connections 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4206f72d0 0x55a4206f96c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206e7780 0x55a4206e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e4330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 --2- 172.21.15.19:0/425890051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206d85d0 0x55a4206e7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 >> 172.21.15.19:0/425890051 conn(0x55a4206a1310 msgr2=0x55a4206da930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe701ffb640 0 client.25733 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 shutdown_connections 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.667+0000 7fe722c30740 1 -- 172.21.15.19:0/425890051 wait complete. 2024-09-12T03:23:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.668+0000 7fe722c30740 20 client.25733 trim_cache size 0 max 16384 2024-09-12T03:23:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.684+0000 7fe722c30740 1 Processor -- start 2024-09-12T03:23:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.684+0000 7fe722c30740 1 -- start start 2024-09-12T03:23:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.684+0000 7fe722c30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.684+0000 7fe722c30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe722c30740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e7780 0x55a4206e3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe722c30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00e3f0 con 0x55a4206d85d0 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe722c30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00fff0 con 0x55a4206d9b40 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe722c30740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00eac0 con 0x55a4206e7780 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe719d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71ad9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e7780 0x55a4206e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37878/0 (socket says 172.21.15.19:37878) 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 -- 172.21.15.19:0/1394706693 learned_addr learned my addr 172.21.15.19:0/1394706693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 -- 172.21.15.19:0/1394706693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e7780 msgr2=0x55a4206e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e7780 0x55a4206e3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 -- 172.21.15.19:0/1394706693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 msgr2=0x55a4206e2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 -- 172.21.15.19:0/1394706693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4206db120 con 0x55a4206d85d0 2024-09-12T03:23:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.685+0000 7fe71a59c640 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70c018500 tx=0x7fe70c0a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a384e85c49728e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.686+0000 7fe703fff640 1 -- 172.21.15.19:0/1394706693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c09b5e0 con 0x55a4206d85d0 2024-09-12T03:23:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.686+0000 7fe703fff640 1 -- 172.21.15.19:0/1394706693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c0a4610 con 0x55a4206d85d0 2024-09-12T03:23:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.686+0000 7fe703fff640 1 -- 172.21.15.19:0/1394706693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70c09b0d0 con 0x55a4206d85d0 2024-09-12T03:23:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- 172.21.15.19:0/1394706693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 msgr2=0x55a4206f37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe70c018500 tx=0x7fe70c0a1040 comp rx=0 tx=0).stop 2024-09-12T03:23:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- 172.21.15.19:0/1394706693 shutdown_connections 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e7780 0x55a4206e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206d9b40 0x55a4206e2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 --2- 172.21.15.19:0/1394706693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d85d0 0x55a4206f37b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- 172.21.15.19:0/1394706693 >> 172.21.15.19:0/1394706693 conn(0x55a4206a1310 msgr2=0x55a4206f92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- 172.21.15.19:0/1394706693 shutdown_connections 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- 172.21.15.19:0/1394706693 wait complete. 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.687+0000 7fe722c30740 1 -- start start 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 Processor -- start 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 0x55a4206efd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e1330 0x55a4206e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00f860 con 0x55a4206d9b40 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00f190 con 0x55a4206e7780 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc00c3d0 con 0x55a4206e1330 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe722c30740 10 client.0 _ll_register_callbacks cb 0x7fe70c0d2ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 0x55a4206efd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe71a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 0x55a4206efd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37894/0 (socket says 172.21.15.19:37894) 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe71a59c640 1 -- 172.21.15.19:0/1809463374 learned_addr learned my addr 172.21.15.19:0/1809463374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.688+0000 7fe71ad9d640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e1330 0x55a4206e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 -- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e1330 msgr2=0x55a4206e17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e1330 0x55a4206e17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 -- 172.21.15.19:0/1809463374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 msgr2=0x55a4206efd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 0x55a4206efd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 -- 172.21.15.19:0/1809463374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a420760ee0 con 0x55a4206e7780 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe701ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe719d9b640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e0df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe71000f750 tx=0x7fe710005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc7b91d1f75a0f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe710028070 con 0x55a4206e7780 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe70c0a4610 con 0x55a4206e7780 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe710011230 con 0x55a4206e7780 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.689+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe710003bc0 con 0x55a4206e7780 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 10 client.25739 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a420762040 con 0x55a4206e7780 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe70c0a4610 con 0x55a4206e7780 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fe710004a60 con 0x55a4206e7780 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 10 client.25739 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 10 client.25739 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 20 client.25739 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4207626d0 0x55a420764af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 4 client.25739 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.690+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a420645d70 con 0x55a4207626d0 2024-09-12T03:23:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.691+0000 7fe71a59c640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4207626d0 0x55a420764af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.692+0000 7fe701ffb640 10 client.25739 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:23:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.692+0000 7fe71a59c640 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4207626d0 0x55a420764af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bce7b508aca0be90 server_cookie=df274fc29f314cc2 in_seq=0 out_seq=0 2024-09-12T03:23:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.695+0000 7fe701ffb640 1 -- 172.21.15.19:0/1809463374 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a420645d70 con 0x55a4207626d0 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.695+0000 7fe701ffb640 10 client.25739 handle_command_reply: tid=0 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.695+0000 7fe722c30740 1 client.25739 shutdown 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.695+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 msgr2=0x55a4206e0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.695+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e0df0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe71000f750 tx=0x7fe710005270 comp rx=0 tx=0).stop 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 shutdown_connections 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55a4207626d0 0x55a420764af0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a4206e1330 0x55a4206e17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a4206e7780 0x55a4206e0df0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 --2- 172.21.15.19:0/1809463374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a4206d9b40 0x55a4206efd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe701ffb640 0 client.25739 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 >> 172.21.15.19:0/1809463374 conn(0x55a4206a1310 msgr2=0x55a4206e5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 shutdown_connections 2024-09-12T03:23:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 1 -- 172.21.15.19:0/1809463374 wait complete. 2024-09-12T03:23:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.696+0000 7fe722c30740 20 client.25739 trim_cache size 0 max 16384 2024-09-12T03:23:24.698 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 msgr2=0x7fe7040a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 0x7fe7040a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe714004ab0 tx=0x7fe71402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 msgr2=0x7fe71c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00e9f0 tx=0x7fe70c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 shutdown_connections 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe71c106a80 0x7fe71c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe7040a2c00 0x7fe7040a50c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe71c101420 0x7fe71c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 --2- 172.21.15.19:0/3406043132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe71c100a80 0x7fe71c198430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:24.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 >> 172.21.15.19:0/3406043132 conn(0x7fe71c0fc7c0 msgr2=0x7fe71c1048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:24.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 shutdown_connections 2024-09-12T03:23:24.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:24.697+0000 7fe720857640 1 -- 172.21.15.19:0/3406043132 wait complete. 2024-09-12T03:23:25.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:23:25.460 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:25 smithi019 ceph-mon[30953]: pgmap v468: 97 pgs: 97 active+clean; 987 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 718 op/s 2024-09-12T03:23:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:25 smithi135 ceph-mon[35474]: pgmap v468: 97 pgs: 97 active+clean; 987 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 718 op/s 2024-09-12T03:23:25.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:25 smithi121 ceph-mon[34963]: pgmap v468: 97 pgs: 97 active+clean; 987 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 17 MiB/s wr, 718 op/s 2024-09-12T03:23:25.699 INFO:tasks.workunit.client.0.smithi019.stdout: 300 906274 1113 639894 1015 148417 1053 2024-09-12T03:23:25.753 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 -- 172.21.15.19:0/1334946570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 msgr2=0x7fc290106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 --2- 172.21.15.19:0/1334946570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290106810 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc278009900 tx=0x7fc27802a3f0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 -- 172.21.15.19:0/1334946570 shutdown_connections 2024-09-12T03:23:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 --2- 172.21.15.19:0/1334946570 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290106f40 0x7fc29010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 --2- 172.21.15.19:0/1334946570 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290106810 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 --2- 172.21.15.19:0/1334946570 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290101e10 0x7fc2901021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 -- 172.21.15.19:0/1334946570 >> 172.21.15.19:0/1334946570 conn(0x7fc2900fd8d0 msgr2=0x7fc2900ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 -- 172.21.15.19:0/1334946570 shutdown_connections 2024-09-12T03:23:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.287+0000 7fc2958b8640 1 -- 172.21.15.19:0/1334946570 wait complete. 2024-09-12T03:23:26.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.288+0000 7fc2958b8640 1 Processor -- start 2024-09-12T03:23:26.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.288+0000 7fc2958b8640 1 -- start start 2024-09-12T03:23:26.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290101e10 0x7fc2901107c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 0x7fc290111240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc290075d20 con 0x7fc2901027c0 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc290075ba0 con 0x7fc290101e10 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc2958b8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc290075ea0 con 0x7fc290106f40 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc28e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc28e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37914/0 (socket says 172.21.15.19:37914) 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc28e7fc640 1 -- 172.21.15.19:0/2872380885 learned_addr learned my addr 172.21.15.19:0/2872380885 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 0x7fc290111240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.289+0000 7fc28effd640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290101e10 0x7fc2901107c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 -- 172.21.15.19:0/2872380885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 msgr2=0x7fc290111240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 0x7fc290111240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 -- 172.21.15.19:0/2872380885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290101e10 msgr2=0x7fc2901107c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290101e10 0x7fc2901107c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 -- 172.21.15.19:0/2872380885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2901aada0 con 0x7fc2901027c0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 0x7fc290111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc27802a9e0 tx=0x7fc278003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc2948b6640 1 -- 172.21.15.19:0/2872380885 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27803f070 con 0x7fc2901027c0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc2948b6640 1 -- 172.21.15.19:0/2872380885 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc278031c90 con 0x7fc2901027c0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2901ab030 con 0x7fc2901027c0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc2948b6640 1 -- 172.21.15.19:0/2872380885 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278035030 con 0x7fc2901027c0 2024-09-12T03:23:26.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.290+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2901ab510 con 0x7fc2901027c0 2024-09-12T03:23:26.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.294+0000 7fc2948b6640 1 -- 172.21.15.19:0/2872380885 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc278042070 con 0x7fc2901027c0 2024-09-12T03:23:26.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.294+0000 7fc2948b6640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 0x7fc27c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.294+0000 7fc297c91740 1 Processor -- start 2024-09-12T03:23:26.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 -- start start 2024-09-12T03:23:26.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc2948b6640 1 -- 172.21.15.19:0/2872380885 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc2780ef7d0 con 0x7fc2901027c0 2024-09-12T03:23:26.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28effd640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 0x7fc27c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56504fa832f0 con 0x56504fb1eb40 2024-09-12T03:23:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56504fa2acf0 con 0x56504fb1d5d0 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc297c91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56504fa7f2b0 con 0x56504fb28370 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55424/0 (socket says 172.21.15.19:55424) 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28f7fe640 1 -- 172.21.15.19:0/111519371 learned_addr learned my addr 172.21.15.19:0/111519371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28effd640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.295+0000 7fc28effd640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 0x7fc27c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280010740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28f7fe640 1 -- 172.21.15.19:0/111519371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 msgr2=0x56504fb1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb1da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28f7fe640 1 -- 172.21.15.19:0/111519371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 msgr2=0x56504fb1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28f7fe640 1 -- 172.21.15.19:0/111519371 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504fa0f3a0 con 0x56504fb28370 2024-09-12T03:23:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28effd640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.296+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.297+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc28800bfb0 tx=0x7fc288010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa4b384c57913e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.297+0000 7fc285ffb640 1 -- 172.21.15.19:0/111519371 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288011700 con 0x56504fb28370 2024-09-12T03:23:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.298+0000 7fc285ffb640 1 -- 172.21.15.19:0/111519371 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc288011d80 con 0x56504fb28370 2024-09-12T03:23:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.298+0000 7fc285ffb640 1 -- 172.21.15.19:0/111519371 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28801adb0 con 0x56504fb28370 2024-09-12T03:23:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.298+0000 7fc297c91740 1 -- 172.21.15.19:0/111519371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 msgr2=0x56504fb2a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.298+0000 7fc297c91740 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc28800bfb0 tx=0x7fc288010860 comp rx=0 tx=0).stop 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 -- 172.21.15.19:0/111519371 shutdown_connections 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb28370 0x56504fb2a760 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 --2- 172.21.15.19:0/111519371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 -- 172.21.15.19:0/111519371 >> 172.21.15.19:0/111519371 conn(0x56504fae6310 msgr2=0x56504fb19c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 -- 172.21.15.19:0/111519371 shutdown_connections 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 -- 172.21.15.19:0/111519371 wait complete. 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 -- start start 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.299+0000 7fc297c91740 1 Processor -- start 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb24fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb25520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56504fb2b0c0 con 0x56504fb1d5d0 2024-09-12T03:23:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56504fa3e060 con 0x56504fb28370 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56504fb2b240 con 0x56504fb1eb40 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 10 client.0 _ll_register_callbacks cb 0x56504f7918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37936/0 (socket says 172.21.15.19:37936) 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 -- 172.21.15.19:0/600519529 learned_addr learned my addr 172.21.15.19:0/600519529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb25520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb24fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 -- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 msgr2=0x56504fb24fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb24fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 -- 172.21.15.19:0/600519529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 msgr2=0x56504fb25520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb25520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 -- 172.21.15.19:0/600519529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504f7f43c0 con 0x56504fb1d5d0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28effd640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc280013ac0 tx=0x7fc280004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d454c44740f4bd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc26effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb24fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28001eab0 con 0x56504fb1d5d0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb25520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.300+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc288011d80 con 0x56504fb1d5d0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc28001ec50 con 0x56504fb1d5d0 2024-09-12T03:23:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc2800283f0 con 0x56504fb1d5d0 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 10 client.16239 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56504fb36b40 con 0x56504fb1d5d0 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc288011d80 con 0x56504fb1d5d0 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fc28000f040 con 0x56504fb1d5d0 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 10 client.16239 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 10 client.16239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 20 client.16239 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fb37f50 0x56504fb3a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 4 client.16239 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.301+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504fa90860 con 0x56504fb37f50 2024-09-12T03:23:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.302+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fb37f50 0x56504fb3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.303+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fb37f50 0x56504fb3a360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5651da36d654980a server_cookie=25ddab3360b14d2 in_seq=0 out_seq=0 2024-09-12T03:23:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.303+0000 7fc26effd640 10 client.16239 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:23:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.338+0000 7fc26effd640 1 -- 172.21.15.19:0/600519529 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56504fa90860 con 0x56504fb37f50 2024-09-12T03:23:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.338+0000 7fc26effd640 10 client.16239 handle_command_reply: tid=0 2024-09-12T03:23:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.338+0000 7fc297c91740 1 client.16239 shutdown 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.338+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 msgr2=0x56504fb27cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.338+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc280013ac0 tx=0x7fc280004ab0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 shutdown_connections 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fb37f50 0x56504fb3a360 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb25520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb24fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 --2- 172.21.15.19:0/600519529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1d5d0 0x56504fb27cf0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 >> 172.21.15.19:0/600519529 conn(0x56504fae6310 msgr2=0x56504fb1f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc26effd640 0 client.16239 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 shutdown_connections 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 1 -- 172.21.15.19:0/600519529 wait complete. 2024-09-12T03:23:26.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.339+0000 7fc297c91740 20 client.16239 trim_cache size 0 max 16384 2024-09-12T03:23:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 Processor -- start 2024-09-12T03:23:26.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 -- start start 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1d5d0 0x56504fb34390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb3fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b3bf0 con 0x56504fb1eb40 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b57f0 con 0x56504fb28370 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc297c91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b42c0 con 0x56504fb1d5d0 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc28e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc28e7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37948/0 (socket says 172.21.15.19:37948) 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.356+0000 7fc28e7fc640 1 -- 172.21.15.19:0/846122959 learned_addr learned my addr 172.21.15.19:0/846122959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28effd640 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1d5d0 0x56504fb34390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb3fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 -- 172.21.15.19:0/846122959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1d5d0 msgr2=0x56504fb34390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1d5d0 0x56504fb34390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 -- 172.21.15.19:0/846122959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 msgr2=0x56504fb3fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb3fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 -- 172.21.15.19:0/846122959 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504fb25890 con 0x56504fb1eb40 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc2780b84a0 tx=0x7fc2780dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0038d9e7282712d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc2857fa640 1 -- 172.21.15.19:0/846122959 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278049040 con 0x56504fb1eb40 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc2857fa640 1 -- 172.21.15.19:0/846122959 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2780c4cb0 con 0x56504fb1eb40 2024-09-12T03:23:26.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.357+0000 7fc2857fa640 1 -- 172.21.15.19:0/846122959 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2780c4820 con 0x56504fb1eb40 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 -- 172.21.15.19:0/846122959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 msgr2=0x56504fb24860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fc2780b84a0 tx=0x7fc2780dc040 comp rx=0 tx=0).stop 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 -- 172.21.15.19:0/846122959 shutdown_connections 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb28370 0x56504fb3fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb1eb40 0x56504fb24860 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.358+0000 7fc297c91740 1 --2- 172.21.15.19:0/846122959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1d5d0 0x56504fb34390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.359+0000 7fc297c91740 1 -- 172.21.15.19:0/846122959 >> 172.21.15.19:0/846122959 conn(0x56504fae6310 msgr2=0x56504fb23870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.359+0000 7fc297c91740 1 -- 172.21.15.19:0/846122959 shutdown_connections 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.359+0000 7fc297c91740 1 -- 172.21.15.19:0/846122959 wait complete. 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.359+0000 7fc297c91740 1 -- start start 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.359+0000 7fc297c91740 1 Processor -- start 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb2bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b5060 con 0x56504fb28370 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b4990 con 0x56504fb1d5d0 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc2901b1bd0 con 0x56504fb1eb40 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc297c91740 10 client.0 _ll_register_callbacks cb 0x7fc280013460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37952/0 (socket says 172.21.15.19:37952) 2024-09-12T03:23:26.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 -- 172.21.15.19:0/1016479998 learned_addr learned my addr 172.21.15.19:0/1016479998 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28e7fc640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb2bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28effd640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 -- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 msgr2=0x56504fb2bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb2bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 -- 172.21.15.19:0/1016479998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 msgr2=0x56504fb309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.360+0000 7fc28f7fe640 1 -- 172.21.15.19:0/1016479998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504fb2c7a0 con 0x56504fb28370 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc28f7fe640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x56504fb1e070 tx=0x7fc288005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e6a3162223286b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc26effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc288005570 con 0x56504fb28370 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2780c4cb0 con 0x56504fb28370 2024-09-12T03:23:26.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc288004020 con 0x56504fb28370 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc288004220 con 0x56504fb28370 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc297c91740 10 client.16245 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.361+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56504fba2960 con 0x56504fb28370 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2780c4cb0 con 0x56504fb28370 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fc288003680 con 0x56504fb28370 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 10 client.16245 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 10 client.16245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 20 client.16245 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fba2d50 0x56504fba5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 4 client.16245 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56504fa8ad70 con 0x56504fba2d50 2024-09-12T03:23:26.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.362+0000 7fc28effd640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fba2d50 0x56504fba5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.363+0000 7fc28effd640 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fba2d50 0x56504fba5140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37ab6a98cf19d7a1 server_cookie=3899458d4ae640c2 in_seq=0 out_seq=0 2024-09-12T03:23:26.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.363+0000 7fc26effd640 10 client.16245 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:23:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.368+0000 7fc26effd640 1 -- 172.21.15.19:0/1016479998 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56504fa8ad70 con 0x56504fba2d50 2024-09-12T03:23:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.368+0000 7fc26effd640 10 client.16245 handle_command_reply: tid=0 2024-09-12T03:23:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.368+0000 7fc297c91740 1 client.16245 shutdown 2024-09-12T03:23:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 msgr2=0x56504fb2c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x56504fb1e070 tx=0x7fc288005270 comp rx=0 tx=0).stop 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 shutdown_connections 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56504fba2d50 0x56504fba5140 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56504fb28370 0x56504fb2c230 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56504fb1eb40 0x56504fb2bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 --2- 172.21.15.19:0/1016479998 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56504fb1d5d0 0x56504fb309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 >> 172.21.15.19:0/1016479998 conn(0x56504fae6310 msgr2=0x56504fb184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc26effd640 0 client.16245 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 shutdown_connections 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 1 -- 172.21.15.19:0/1016479998 wait complete. 2024-09-12T03:23:26.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.369+0000 7fc297c91740 20 client.16245 trim_cache size 0 max 16384 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 msgr2=0x7fc27c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 0x7fc27c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc280005fd0 tx=0x7fc280010740 comp rx=0 tx=0).stop 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 msgr2=0x7fc290110d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:26.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc27802a9e0 tx=0x7fc278003c10 comp rx=0 tx=0).stop 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 shutdown_connections 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc290106f40 0x7fc290111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc27c0a2b80 0x7fc27c0a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc2901027c0 0x7fc290110d00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 --2- 172.21.15.19:0/2872380885 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc290101e10 0x7fc2901107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 >> 172.21.15.19:0/2872380885 conn(0x7fc2900fd8d0 msgr2=0x7fc2900fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 shutdown_connections 2024-09-12T03:23:26.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:26.370+0000 7fc2958b8640 1 -- 172.21.15.19:0/2872380885 wait complete. 2024-09-12T03:23:27.203 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:23:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:27 smithi019 ceph-mon[30953]: pgmap v469: 97 pgs: 97 active+clean; 988 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-09-12T03:23:27.520 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:27 smithi135 ceph-mon[35474]: pgmap v469: 97 pgs: 97 active+clean; 988 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-09-12T03:23:27.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:27 smithi121 ceph-mon[34963]: pgmap v469: 97 pgs: 97 active+clean; 988 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 340 op/s 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 -- 172.21.15.19:0/4001419328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 msgr2=0x7f672c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 --2- 172.21.15.19:0/4001419328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c101200 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6728009900 tx=0x7f672802a480 comp rx=0 tx=0).stop 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 -- 172.21.15.19:0/4001419328 shutdown_connections 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 --2- 172.21.15.19:0/4001419328 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 --2- 172.21.15.19:0/4001419328 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f672c102570 0x7f672c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 --2- 172.21.15.19:0/4001419328 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c101200 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 -- 172.21.15.19:0/4001419328 >> 172.21.15.19:0/4001419328 conn(0x7f672c0fc7d0 msgr2=0x7f672c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 -- 172.21.15.19:0/4001419328 shutdown_connections 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.185+0000 7f673372a640 1 -- 172.21.15.19:0/4001419328 wait complete. 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 Processor -- start 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 -- start start 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f672c102570 0x7f672c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f672c10fe80 con 0x7f672c100e20 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f672c10fd00 con 0x7f672c102ef0 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f673372a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f672c110000 con 0x7f672c102570 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f67314c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f67314c6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37978/0 (socket says 172.21.15.19:37978) 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f67314c6640 1 -- 172.21.15.19:0/882204571 learned_addr learned my addr 172.21.15.19:0/882204571 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:28.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f6731cc7640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f67314c6640 1 -- 172.21.15.19:0/882204571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f672c102570 msgr2=0x7f672c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.186+0000 7f67314c6640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f672c102570 0x7f672c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f67314c6640 1 -- 172.21.15.19:0/882204571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 msgr2=0x7f672c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f67314c6640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f67314c6640 1 -- 172.21.15.19:0/882204571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672c19f700 con 0x7f672c100e20 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f6731cc7640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f67314c6640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6728006010 tx=0x7f6728003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f671e7fc640 1 -- 172.21.15.19:0/882204571 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672803f070 con 0x7f672c100e20 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f671e7fc640 1 -- 172.21.15.19:0/882204571 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728031c30 con 0x7f672c100e20 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672c19f990 con 0x7f672c100e20 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f671e7fc640 1 -- 172.21.15.19:0/882204571 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728035030 con 0x7f672c100e20 2024-09-12T03:23:28.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.187+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672c1a72d0 con 0x7f672c100e20 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.189+0000 7f671e7fc640 1 -- 172.21.15.19:0/882204571 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6728042070 con 0x7f672c100e20 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 Processor -- start 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 -- start start 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55beb83a82f0 con 0x55beb8443b40 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55beb834fcf0 con 0x55beb84425d0 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6735b03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55beb83a42b0 con 0x55beb844d370 2024-09-12T03:23:28.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6731cc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6731cc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37980/0 (socket says 172.21.15.19:37980) 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42946/0 (socket says 172.21.15.19:42946) 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f67314c6640 1 -- 172.21.15.19:0/2982823852 learned_addr learned my addr 172.21.15.19:0/2982823852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f671e7fc640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 0x7f67180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6730cc5640 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f6731cc7640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 0x7f67180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.190+0000 7f671e7fc640 1 -- 172.21.15.19:0/882204571 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67280ef850 con 0x7f672c100e20 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 -- 172.21.15.19:0/2982823852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 msgr2=0x55beb844f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 -- 172.21.15.19:0/2982823852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 msgr2=0x55beb8442a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 -- 172.21.15.19:0/2982823852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb83343a0 con 0x55beb8443b40 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f67314c6640 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6724005fd0 tx=0x7f672400ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a9e6631e05005b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f67177fe640 1 -- 172.21.15.19:0/2982823852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724017da0 con 0x55beb8443b40 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f67177fe640 1 -- 172.21.15.19:0/2982823852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672401a8d0 con 0x55beb8443b40 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f67177fe640 1 -- 172.21.15.19:0/2982823852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6724019350 con 0x55beb8443b40 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.191+0000 7f6731cc7640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 0x7f67180a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6724021000 tx=0x7f672401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- 172.21.15.19:0/2982823852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 msgr2=0x55beb8442090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6724005fd0 tx=0x7f672400ac10 comp rx=0 tx=0).stop 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- 172.21.15.19:0/2982823852 shutdown_connections 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8442a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 --2- 172.21.15.19:0/2982823852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8442090 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- 172.21.15.19:0/2982823852 >> 172.21.15.19:0/2982823852 conn(0x55beb840b310 msgr2=0x55beb843ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- 172.21.15.19:0/2982823852 shutdown_connections 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- 172.21.15.19:0/2982823852 wait complete. 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.192+0000 7f6735b03740 1 -- start start 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 Processor -- start 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb844cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55beb8450070 con 0x55beb8443b40 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55beb8363060 con 0x55beb84425d0 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55beb84501f0 con 0x55beb844d370 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6735b03740 10 client.0 _ll_register_callbacks cb 0x55beb80b68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6730cc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6730cc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37986/0 (socket says 172.21.15.19:37986) 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6730cc5640 1 -- 172.21.15.19:0/3128294193 learned_addr learned my addr 172.21.15.19:0/3128294193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f67314c6640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb844cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.193+0000 7f6731cc7640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 -- 172.21.15.19:0/3128294193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 msgr2=0x55beb844a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 -- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 msgr2=0x55beb844cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb844cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 -- 172.21.15.19:0/3128294193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb81193c0 con 0x55beb8443b40 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f67314c6640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb844cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6730cc5640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f67200116f0 tx=0x7f6720011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4085300ccfb9d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000f920 con 0x55beb8443b40 2024-09-12T03:23:28.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f672000fac0 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672401a8d0 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f672000fcc0 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6735b03740 10 client.16257 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55beb845bba0 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.194+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672401a8d0 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f6720018070 con 0x55beb8443b40 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 10 client.16257 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 10 client.16257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 20 client.16257 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb845cfb0 0x55beb845f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 4 client.16257 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beb83b5860 con 0x55beb845cfb0 2024-09-12T03:23:28.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f67314c6640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb845cfb0 0x55beb845f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f67314c6640 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb845cfb0 0x55beb845f430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a48859875556d42 server_cookie=eac24888c74a93e5 in_seq=0 out_seq=0 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.195+0000 7f6714ff9640 10 client.16257 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.197+0000 7f6714ff9640 1 -- 172.21.15.19:0/3128294193 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55beb83b5860 con 0x55beb845cfb0 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.197+0000 7f6714ff9640 10 client.16257 handle_command_reply: tid=0 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 client.16257 shutdown 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 msgr2=0x55beb844a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f67200116f0 tx=0x7f6720011bb0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 shutdown_connections 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb845cfb0 0x55beb845f430 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb844a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb844a030 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6714ff9640 0 client.16257 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:23:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 --2- 172.21.15.19:0/3128294193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb844cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 >> 172.21.15.19:0/3128294193 conn(0x55beb840b310 msgr2=0x55beb8444930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 shutdown_connections 2024-09-12T03:23:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 1 -- 172.21.15.19:0/3128294193 wait complete. 2024-09-12T03:23:28.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.198+0000 7f6735b03740 20 client.16257 trim_cache size 0 max 16384 2024-09-12T03:23:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.213+0000 7f6735b03740 1 Processor -- start 2024-09-12T03:23:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.213+0000 7f6735b03740 1 -- start start 2024-09-12T03:23:28.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.213+0000 7f6735b03740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8445120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.213+0000 7f6735b03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8445660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.213+0000 7f6735b03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb8464d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6735b03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1af730 con 0x55beb8443b40 2024-09-12T03:23:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6735b03740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b1330 con 0x55beb84425d0 2024-09-12T03:23:28.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6735b03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1afe00 con 0x55beb844d370 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8445120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8445660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8445120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42972/0 (socket says 172.21.15.19:42972) 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f67314c6640 1 -- 172.21.15.19:0/2767111338 learned_addr learned my addr 172.21.15.19:0/2767111338 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6731cc7640 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb8464d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 -- 172.21.15.19:0/2767111338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 msgr2=0x55beb8464d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb8464d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 -- 172.21.15.19:0/2767111338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 msgr2=0x55beb8445120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8445120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 -- 172.21.15.19:0/2767111338 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beb84422d0 con 0x55beb8443b40 2024-09-12T03:23:28.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6730cc5640 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8445660 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55beb844b5e0 tx=0x7f67200058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55df47699416ffad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.214+0000 7f6716ffd640 1 -- 172.21.15.19:0/2767111338 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720032070 con 0x55beb8443b40 2024-09-12T03:23:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6716ffd640 1 -- 172.21.15.19:0/2767111338 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6720020030 con 0x55beb8443b40 2024-09-12T03:23:28.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6716ffd640 1 -- 172.21.15.19:0/2767111338 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720003ad0 con 0x55beb8443b40 2024-09-12T03:23:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 -- 172.21.15.19:0/2767111338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 msgr2=0x55beb8445660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8445660 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55beb844b5e0 tx=0x7f67200058d0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 -- 172.21.15.19:0/2767111338 shutdown_connections 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb8464d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8445660 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 --2- 172.21.15.19:0/2767111338 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb8445120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 -- 172.21.15.19:0/2767111338 >> 172.21.15.19:0/2767111338 conn(0x55beb840b310 msgr2=0x55beb844c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 -- 172.21.15.19:0/2767111338 shutdown_connections 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.215+0000 7f6735b03740 1 -- 172.21.15.19:0/2767111338 wait complete. 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 -- start start 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 Processor -- start 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8451b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb84c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b0ba0 con 0x55beb8443b40 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1b04d0 con 0x55beb84425d0 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f672c1ad710 con 0x55beb844d370 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f6735b03740 10 client.0 _ll_register_callbacks cb 0x7f6720009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.216+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42982/0 (socket says 172.21.15.19:42982) 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 -- 172.21.15.19:0/1738994117 learned_addr learned my addr 172.21.15.19:0/1738994117 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6730cc5640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8451b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6731cc7640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb84c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 -- 172.21.15.19:0/1738994117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 msgr2=0x55beb84c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb84c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 -- 172.21.15.19:0/1738994117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 msgr2=0x55beb8451b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8451b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 -- 172.21.15.19:0/1738994117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beb84c7210 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6730cc5640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8451b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f67314c6640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67280b84c0 tx=0x7f672808d910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9ab56a0d8eb4d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6714ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728049040 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6720020030 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.217+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f67280ed410 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6728031de0 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 10 client.25772 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55beb84c83b0 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6720020030 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f67280b86d0 con 0x55beb84425d0 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 10 client.25772 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 10 client.25772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:23:28.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 20 client.25772 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb84c8b00 0x55beb84caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 4 client.25772 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55beb83afd70 con 0x55beb84c8b00 2024-09-12T03:23:28.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.218+0000 7f6730cc5640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb84c8b00 0x55beb84caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.220+0000 7f6714ff9640 10 client.25772 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.220+0000 7f6730cc5640 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb84c8b00 0x55beb84caf20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ba93c7d40dfc0bda server_cookie=d6537d89ad63543a in_seq=0 out_seq=0 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6714ff9640 1 -- 172.21.15.19:0/1738994117 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55beb83afd70 con 0x55beb84c8b00 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6714ff9640 10 client.25772 handle_command_reply: tid=0 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 client.25772 shutdown 2024-09-12T03:23:28.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 msgr2=0x55beb84515e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f67280b84c0 tx=0x7f672808d910 comp rx=0 tx=0).stop 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 shutdown_connections 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55beb84c8b00 0x55beb84caf20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55beb844d370 0x55beb84c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55beb8443b40 0x55beb8451b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6714ff9640 0 client.25772 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 --2- 172.21.15.19:0/1738994117 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55beb84425d0 0x55beb84515e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 >> 172.21.15.19:0/1738994117 conn(0x55beb840b310 msgr2=0x55beb843d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 shutdown_connections 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 1 -- 172.21.15.19:0/1738994117 wait complete. 2024-09-12T03:23:28.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.222+0000 7f6735b03740 20 client.25772 trim_cache size 0 max 16384 2024-09-12T03:23:28.225 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:23:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 msgr2=0x7f67180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 0x7f67180a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6724021000 tx=0x7f672401f000 comp rx=0 tx=0).stop 2024-09-12T03:23:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 msgr2=0x7f672c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:28.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6728006010 tx=0x7f6728003c10 comp rx=0 tx=0).stop 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 shutdown_connections 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f672c102ef0 0x7f672c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f672c102570 0x7f672c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f67180a2b80 0x7f67180a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 --2- 172.21.15.19:0/882204571 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f672c100e20 0x7f672c19a750 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 >> 172.21.15.19:0/882204571 conn(0x7f672c0fc7d0 msgr2=0x7f672c10c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 shutdown_connections 2024-09-12T03:23:28.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:28.223+0000 7f673372a640 1 -- 172.21.15.19:0/882204571 wait complete. 2024-09-12T03:23:29.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:23:29.351 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:29 smithi019 ceph-mon[30953]: pgmap v470: 97 pgs: 97 active+clean; 996 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 380 op/s 2024-09-12T03:23:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:29 smithi135 ceph-mon[35474]: pgmap v470: 97 pgs: 97 active+clean; 996 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 380 op/s 2024-09-12T03:23:29.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:29 smithi121 ceph-mon[34963]: pgmap v470: 97 pgs: 97 active+clean; 996 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.0 MiB/s wr, 380 op/s 2024-09-12T03:23:29.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 -- 172.21.15.19:0/3093535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 msgr2=0x7f13601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 --2- 172.21.15.19:0/3093535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601029b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f1350009900 tx=0x7f135002a480 comp rx=0 tx=0).stop 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 -- 172.21.15.19:0/3093535791 shutdown_connections 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 --2- 172.21.15.19:0/3093535791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 --2- 172.21.15.19:0/3093535791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601029b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 --2- 172.21.15.19:0/3093535791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f1360101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 -- 172.21.15.19:0/3093535791 >> 172.21.15.19:0/3093535791 conn(0x7f13600fc7d0 msgr2=0x7f13600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 -- 172.21.15.19:0/3093535791 shutdown_connections 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.867+0000 7f1368408640 1 -- 172.21.15.19:0/3093535791 wait complete. 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.868+0000 7f1368408640 1 Processor -- start 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.868+0000 7f1368408640 1 -- start start 2024-09-12T03:23:29.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.868+0000 7f1368408640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f13601962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.868+0000 7f1368408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f1368408640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f1368408640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f136010fec0 con 0x7f1360102570 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f1368408640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f136010fd40 con 0x7f1360102ef0 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f1368408640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1360110040 con 0x7f1360100e20 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13661a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f13601962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38026/0 (socket says 172.21.15.19:38026) 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 -- 172.21.15.19:0/1602757546 learned_addr learned my addr 172.21.15.19:0/1602757546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13661a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f13601962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55518/0 (socket says 172.21.15.19:55518) 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13669a5640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 -- 172.21.15.19:0/1602757546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 msgr2=0x7f13601962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f13601962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 -- 172.21.15.19:0/1602757546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 msgr2=0x7f136019ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136019ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 -- 172.21.15.19:0/1602757546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136019b250 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13669a5640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13659a3640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f135002aa70 tx=0x7f1350003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.869+0000 7f13577fe640 1 -- 172.21.15.19:0/1602757546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135003f070 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.870+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136019b4e0 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.870+0000 7f13577fe640 1 -- 172.21.15.19:0/1602757546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1350031c30 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.870+0000 7f13577fe640 1 -- 172.21.15.19:0/1602757546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350035030 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.870+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a7230 con 0x7f1360102570 2024-09-12T03:23:29.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.872+0000 7f13577fe640 1 -- 172.21.15.19:0/1602757546 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1350042070 con 0x7f1360102570 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.872+0000 7f136a7e1740 1 Processor -- start 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 -- start start 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf2ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6be922f0 con 0x55ec6bf2db40 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec6be39cf0 con 0x55ec6bf37370 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f136a7e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ec6be8e2b0 con 0x55ec6bf2c5d0 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f13659a3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f13669a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f13659a3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:43014/0 (socket says 172.21.15.19:43014) 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.873+0000 7f13659a3640 1 -- 172.21.15.19:0/2381884781 learned_addr learned my addr 172.21.15.19:0/2381884781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13661a4640 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf2ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 -- 172.21.15.19:0/2381884781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 msgr2=0x55ec6bf2ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf2ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 -- 172.21.15.19:0/2381884781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 msgr2=0x55ec6bf2c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf2c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 -- 172.21.15.19:0/2381884781 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6be1e3a0 con 0x55ec6bf37370 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13669a5640 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13659a3640 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13500f5c10 tx=0x7f13500f5d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e07056a05d063c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.874+0000 7f13577fe640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 0x7f134c0a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.875+0000 7f13669a5640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 0x7f134c0a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.875+0000 7f1354dea640 1 -- 172.21.15.19:0/2381884781 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500fe070 con 0x55ec6bf37370 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.875+0000 7f1354dea640 1 -- 172.21.15.19:0/2381884781 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13500bc630 con 0x55ec6bf37370 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.875+0000 7f1354dea640 1 -- 172.21.15.19:0/2381884781 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135004cbe0 con 0x55ec6bf37370 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.875+0000 7f13577fe640 1 -- 172.21.15.19:0/1602757546 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13500ef6c0 con 0x7f1360102570 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- 172.21.15.19:0/2381884781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 msgr2=0x55ec6bf39760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f13500f5c10 tx=0x7f13500f5d00 comp rx=0 tx=0).stop 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f13669a5640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 0x7f134c0a5030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1358005fd0 tx=0x7f1358009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- 172.21.15.19:0/2381884781 shutdown_connections 2024-09-12T03:23:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf37370 0x55ec6bf39760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf2ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2381884781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf2c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- 172.21.15.19:0/2381884781 >> 172.21.15.19:0/2381884781 conn(0x55ec6bef5310 msgr2=0x55ec6bf28c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- 172.21.15.19:0/2381884781 shutdown_connections 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- 172.21.15.19:0/2381884781 wait complete. 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 -- start start 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.876+0000 7f136a7e1740 1 Processor -- start 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf36cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf37370 0x55ec6bf344a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bf42640 con 0x55ec6bf37370 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec6be4d060 con 0x55ec6bf2db40 2024-09-12T03:23:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ec6bf427c0 con 0x55ec6bf2c5d0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f136a7e1740 10 client.0 _ll_register_callbacks cb 0x55ec6bba08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f13659a3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f13669a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf37370 0x55ec6bf344a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f13659a3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:43030/0 (socket says 172.21.15.19:43030) 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.877+0000 7f13659a3640 1 -- 172.21.15.19:0/2864976371 learned_addr learned my addr 172.21.15.19:0/2864976371 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13661a4640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf36cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 -- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 msgr2=0x55ec6bf36cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf36cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 -- 172.21.15.19:0/2864976371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf37370 msgr2=0x55ec6bf344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf37370 0x55ec6bf344a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 -- 172.21.15.19:0/2864976371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6bc033c0 con 0x55ec6bf2db40 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f13659a3640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13500bbec0 tx=0x7f135002a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ec55e83846c5661 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f1349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350042880 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13500efd80 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.878+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13500bc630 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1350042aa0 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 10 client.25784 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec6bf42640 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f135003b460 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f13500b85a0 con 0x55ec6bf2db40 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 10 client.25784 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 20 client.25784 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bf4f4d0 0x55ec6bf518f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 4 client.25784 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:29.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.879+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13500bc630 con 0x55ec6bf4f4d0 2024-09-12T03:23:29.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.880+0000 7f13661a4640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bf4f4d0 0x55ec6bf518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.881+0000 7f13661a4640 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bf4f4d0 0x55ec6bf518f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4801a48fa8aadb server_cookie=9ad8537c754341dd in_seq=0 out_seq=0 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.881+0000 7f1349ffb640 10 client.25784 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f1349ffb640 1 -- 172.21.15.19:0/2864976371 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f13500bc630 con 0x55ec6bf4f4d0 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f1349ffb640 10 client.25784 handle_command_reply: tid=0 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 client.25784 shutdown 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 msgr2=0x55ec6bf33f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13500bbec0 tx=0x7f135002a480 comp rx=0 tx=0).stop 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 shutdown_connections 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf37370 0x55ec6bf344a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bf4f4d0 0x55ec6bf518f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f1349ffb640 0 client.25784 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2db40 0x55ec6bf33f60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 --2- 172.21.15.19:0/2864976371 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf36cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.886+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 >> 172.21.15.19:0/2864976371 conn(0x55ec6bef5310 msgr2=0x55ec6bf2e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.887+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 shutdown_connections 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.887+0000 7f136a7e1740 1 -- 172.21.15.19:0/2864976371 wait complete. 2024-09-12T03:23:29.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.887+0000 7f136a7e1740 20 client.25784 trim_cache size 0 max 16384 2024-09-12T03:23:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 Processor -- start 2024-09-12T03:23:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 -- start start 2024-09-12T03:23:29.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ab420 con 0x55ec6bf2db40 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ad020 con 0x55ec6bf2c5d0 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.902+0000 7f136a7e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13601abaf0 con 0x55ec6bf37370 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38056/0 (socket says 172.21.15.19:38056) 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 -- 172.21.15.19:0/3915610413 learned_addr learned my addr 172.21.15.19:0/3915610413 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13661a4640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13669a5640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 -- 172.21.15.19:0/3915610413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 msgr2=0x55ec6bf571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 -- 172.21.15.19:0/3915610413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 msgr2=0x55ec6bf4b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf4b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 -- 172.21.15.19:0/3915610413 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6bf324b0 con 0x55ec6bf2db40 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13669a5640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f13659a3640 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f13500b8340 tx=0x7f13500f5d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14150ff2954009d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f134bfff640 1 -- 172.21.15.19:0/3915610413 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500fe070 con 0x55ec6bf2db40 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f134bfff640 1 -- 172.21.15.19:0/3915610413 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13500f5320 con 0x55ec6bf2db40 2024-09-12T03:23:29.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.903+0000 7f134bfff640 1 -- 172.21.15.19:0/3915610413 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1350042cc0 con 0x55ec6bf2db40 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 -- 172.21.15.19:0/3915610413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 msgr2=0x55ec6bf32f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f13500b8340 tx=0x7f13500f5d00 comp rx=0 tx=0).stop 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 -- 172.21.15.19:0/3915610413 shutdown_connections 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf32f40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 --2- 172.21.15.19:0/3915610413 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf4b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 -- 172.21.15.19:0/3915610413 >> 172.21.15.19:0/3915610413 conn(0x55ec6bef5310 msgr2=0x55ec6bf32830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 -- 172.21.15.19:0/3915610413 shutdown_connections 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.904+0000 7f136a7e1740 1 -- 172.21.15.19:0/3915610413 wait complete. 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 -- start start 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 Processor -- start 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf479f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf42f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ac890 con 0x55ec6bf2db40 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13601ac1c0 con 0x55ec6bf2c5d0 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13601a9420 con 0x55ec6bf37370 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f136a7e1740 10 client.0 _ll_register_callbacks cb 0x7f13580099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:29.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f13659a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38070/0 (socket says 172.21.15.19:38070) 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f13659a3640 1 -- 172.21.15.19:0/893041526 learned_addr learned my addr 172.21.15.19:0/893041526 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.905+0000 7f13661a4640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf479f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13669a5640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf42f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 -- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 msgr2=0x55ec6bf42f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf42f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 -- 172.21.15.19:0/893041526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 msgr2=0x55ec6bf479f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf479f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 -- 172.21.15.19:0/893041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6bf43750 con 0x55ec6bf2db40 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f13659a3640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13500efb40 tx=0x7f135004c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e76740c6ba25f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f1349ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:29.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13500fe050 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f13500efd80 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13500f5320 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f13500a1050 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f136a7e1740 10 client.16281 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.906+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec6bf43750 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13500f5320 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f1350042cc0 con 0x55ec6bf2db40 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 10 client.16281 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 10 client.16281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 20 client.16281 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bfb9f80 0x55ec6bfbc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 4 client.16281 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:23:29.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec6be9f860 con 0x55ec6bfb9f80 2024-09-12T03:23:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.907+0000 7f13661a4640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bfb9f80 0x55ec6bfbc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.908+0000 7f13661a4640 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bfb9f80 0x55ec6bfbc3a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=257fec9f4d106cb9 server_cookie=fef93879bbf498f1 in_seq=0 out_seq=0 2024-09-12T03:23:29.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.908+0000 7f1349ffb640 10 client.16281 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:29.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f1349ffb640 1 -- 172.21.15.19:0/893041526 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec6be9f860 con 0x55ec6bfb9f80 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f1349ffb640 10 client.16281 handle_command_reply: tid=0 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 client.16281 shutdown 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 msgr2=0x55ec6bf47f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f13500efb40 tx=0x7f135004c4f0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 shutdown_connections 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec6bf37370 0x55ec6bf42f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec6bf2db40 0x55ec6bf47f30 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec6bfb9f80 0x55ec6bfbc3a0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 --2- 172.21.15.19:0/893041526 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec6bf2c5d0 0x55ec6bf479f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 >> 172.21.15.19:0/893041526 conn(0x55ec6bef5310 msgr2=0x55ec6bf33050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f1349ffb640 0 client.16281 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.909+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 shutdown_connections 2024-09-12T03:23:29.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f136a7e1740 1 -- 172.21.15.19:0/893041526 wait complete. 2024-09-12T03:23:29.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f136a7e1740 20 client.16281 trim_cache size 0 max 16384 2024-09-12T03:23:29.915 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 msgr2=0x7f134c0a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 0x7f134c0a5030 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1358005fd0 tx=0x7f1358009500 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 msgr2=0x7f13601967e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.910+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f135002aa70 tx=0x7f1350003c10 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 shutdown_connections 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1360102ef0 0x7f136019ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f134c0a2b70 0x7f134c0a5030 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1360102570 0x7f13601967e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 --2- 172.21.15.19:0/1602757546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1360100e20 0x7f13601962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 >> 172.21.15.19:0/1602757546 conn(0x7f13600fc7d0 msgr2=0x7f1360106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:29.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 shutdown_connections 2024-09-12T03:23:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:29.911+0000 7f1368408640 1 -- 172.21.15.19:0/1602757546 wait complete. 2024-09-12T03:23:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:31 smithi135 ceph-mon[35474]: pgmap v471: 97 pgs: 97 active+clean; 1010 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 553 op/s 2024-09-12T03:23:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:31 smithi121 ceph-mon[34963]: pgmap v471: 97 pgs: 97 active+clean; 1010 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 553 op/s 2024-09-12T03:23:31.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:31 smithi019 ceph-mon[30953]: pgmap v471: 97 pgs: 97 active+clean; 1010 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 553 op/s 2024-09-12T03:23:31.772 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:23:31.772 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:23:32.031 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/381235791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102570 msgr2=0x7fd3ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/381235791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102570 0x7fd3ac1029b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009900 tx=0x7fd39c02a480 comp rx=0 tx=0).stop 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/381235791 shutdown_connections 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/381235791 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/381235791 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102570 0x7fd3ac1029b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/381235791 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac100e20 0x7fd3ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/381235791 >> 172.21.15.19:0/381235791 conn(0x7fd3ac0fc7d0 msgr2=0x7fd3ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/381235791 shutdown_connections 2024-09-12T03:23:32.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.501+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/381235791 wait complete. 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 Processor -- start 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 -- start start 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 0x7fd3ac199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac1a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac110f50 con 0x7fd3ac102ef0 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac110dd0 con 0x7fd3ac100e20 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b3e4f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1110d0 con 0x7fd3ac102570 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b23ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac1a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40352/0 (socket says 172.21.15.19:40352) 2024-09-12T03:23:32.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b1beb640 1 -- 172.21.15.19:0/4155870544 learned_addr learned my addr 172.21.15.19:0/4155870544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.502+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 0x7fd3ac199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 -- 172.21.15.19:0/4155870544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 msgr2=0x7fd3ac199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 0x7fd3ac199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 -- 172.21.15.19:0/4155870544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 msgr2=0x7fd3ac199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 -- 172.21.15.19:0/4155870544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac1a4900 con 0x7fd3ac102ef0 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 0x7fd3ac199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b23ec640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac1a43c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3a2ffd640 1 -- 172.21.15.19:0/4155870544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400ce20 con 0x7fd3ac102ef0 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3a2ffd640 1 -- 172.21.15.19:0/4155870544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4005020 con 0x7fd3ac102ef0 2024-09-12T03:23:32.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1a4ad0 con 0x7fd3ac102ef0 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3a2ffd640 1 -- 172.21.15.19:0/4155870544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4010720 con 0x7fd3ac102ef0 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.503+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac1a4f90 con 0x7fd3ac102ef0 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3a2ffd640 1 -- 172.21.15.19:0/4155870544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3a4005540 con 0x7fd3ac102ef0 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 Processor -- start 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 -- start start 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b15b60 0x55c9d8b15f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b20d40 0x55c9d8b23130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8a7b280 con 0x55c9d8b20d40 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8a22b10 con 0x55c9d8b15b60 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b6228740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8a77240 con 0x55c9d8b16480 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b15b60 0x55c9d8b15f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b20d40 0x55c9d8b23130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53758/0 (socket says 172.21.15.19:53758) 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.506+0000 7fd3b1beb640 1 -- 172.21.15.19:0/1431939838 learned_addr learned my addr 172.21.15.19:0/1431939838 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3a2ffd640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 -- 172.21.15.19:0/1431939838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b15b60 msgr2=0x55c9d8b15f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b15b60 0x55c9d8b15f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 -- 172.21.15.19:0/1431939838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b20d40 msgr2=0x55c9d8b23130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b20d40 0x55c9d8b23130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 -- 172.21.15.19:0/1431939838 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8a2f1f0 con 0x55c9d8b16480 2024-09-12T03:23:32.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b23ec640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8011650 tx=0x7fd3a8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63c914623d8fb8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd3a2ffd640 1 -- 172.21.15.19:0/4155870544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd3a40cf6a0 con 0x7fd3ac102ef0 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd397fff640 1 -- 172.21.15.19:0/1431939838 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a801e070 con 0x55c9d8b16480 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd397fff640 1 -- 172.21.15.19:0/1431939838 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a800fb20 con 0x55c9d8b16480 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.507+0000 7fd397fff640 1 -- 172.21.15.19:0/1431939838 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8019c50 con 0x55c9d8b16480 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 -- 172.21.15.19:0/1431939838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 msgr2=0x55c9d8b20800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b23ec640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 0x7fd3980a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40988e0 tx=0x7fd3a4067030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8011650 tx=0x7fd3a8009f90 comp rx=0 tx=0).stop 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 -- 172.21.15.19:0/1431939838 shutdown_connections 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b20d40 0x55c9d8b23130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b20800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1431939838 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b15b60 0x55c9d8b15f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 -- 172.21.15.19:0/1431939838 >> 172.21.15.19:0/1431939838 conn(0x55c9d8b120f0 msgr2=0x55c9d8b124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 -- 172.21.15.19:0/1431939838 shutdown_connections 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.508+0000 7fd3b6228740 1 -- 172.21.15.19:0/1431939838 wait complete. 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 -- start start 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 Processor -- start 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b1e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8b23910 con 0x55c9d8b15b60 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8a35f10 con 0x55c9d8b20d40 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8b23a90 con 0x55c9d8b16480 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b6228740 10 client.0 _ll_register_callbacks cb 0x55c9d89fea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42622/0 (socket says 172.21.15.19:42622) 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 -- 172.21.15.19:0/2410038759 learned_addr learned my addr 172.21.15.19:0/2410038759 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b1e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 -- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 msgr2=0x55c9d8b1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.509+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b13ea640 1 -- 172.21.15.19:0/2410038759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 msgr2=0x55c9d8b1e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b1e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b13ea640 1 -- 172.21.15.19:0/2410038759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d87ec6a0 con 0x55c9d8b15b60 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd39c02a9a0 tx=0x7fd39c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc14b6a59790416 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c035040 con 0x55c9d8b15b60 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd39c031540 con 0x55c9d8b15b60 2024-09-12T03:23:32.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a800fb20 con 0x55c9d8b15b60 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd39c031740 con 0x55c9d8b15b60 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 10 client.16293 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c9d8b2f4c0 con 0x55c9d8b15b60 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a800fb20 con 0x55c9d8b15b60 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fd39c03c830 con 0x55c9d8b15b60 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 10 client.16293 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 20 client.16293 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.510+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b30920 0x55c9d8b32d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.511+0000 7fd3b6228740 4 client.16293 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.511+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d8a887f0 con 0x55c9d8b30920 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.511+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b30920 0x55c9d8b32d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.511+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b30920 0x55c9d8b32d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17ddeb72f5a24260 server_cookie=4703d90fff313e11 in_seq=0 out_seq=0 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.511+0000 7fd3957fa640 10 client.16293 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3957fa640 1 -- 172.21.15.19:0/2410038759 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9d8a887f0 con 0x55c9d8b30920 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3957fa640 10 client.16293 handle_command_reply: tid=0 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 client.16293 shutdown 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 msgr2=0x55c9d8b1dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd39c02a9a0 tx=0x7fd39c02a480 comp rx=0 tx=0).stop 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 shutdown_connections 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b30920 0x55c9d8b32d80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b1e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b16480 0x55c9d8b1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2410038759 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b1dbd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3957fa640 0 client.16293 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 >> 172.21.15.19:0/2410038759 conn(0x55c9d8b120f0 msgr2=0x55c9d8b11270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 shutdown_connections 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 1 -- 172.21.15.19:0/2410038759 wait complete. 2024-09-12T03:23:32.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.514+0000 7fd3b6228740 20 client.16293 trim_cache size 0 max 16384 2024-09-12T03:23:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.531+0000 7fd3b6228740 1 Processor -- start 2024-09-12T03:23:32.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.531+0000 7fd3b6228740 1 -- start start 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b16480 0x55c9d8b38b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b20d40 0x55c9d8b10a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1ad520 con 0x55c9d8b15b60 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1af120 con 0x55c9d8b16480 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b6228740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1adbf0 con 0x55c9d8b20d40 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42634/0 (socket says 172.21.15.19:42634) 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 -- 172.21.15.19:0/1042608459 learned_addr learned my addr 172.21.15.19:0/1042608459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:32.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b16480 0x55c9d8b38b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b20d40 0x55c9d8b10a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 -- 172.21.15.19:0/1042608459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b20d40 msgr2=0x55c9d8b10a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b20d40 0x55c9d8b10a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 -- 172.21.15.19:0/1042608459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b16480 msgr2=0x55c9d8b38b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b16480 0x55c9d8b38b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 -- 172.21.15.19:0/1042608459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8b11b90 con 0x55c9d8b15b60 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd39c001eb0 tx=0x7fd39c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199bda99a7ea5a70 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.532+0000 7fd3977fe640 1 -- 172.21.15.19:0/1042608459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c035070 con 0x55c9d8b15b60 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3977fe640 1 -- 172.21.15.19:0/1042608459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c031540 con 0x55c9d8b15b60 2024-09-12T03:23:32.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3977fe640 1 -- 172.21.15.19:0/1042608459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c03c870 con 0x55c9d8b15b60 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 -- 172.21.15.19:0/1042608459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 msgr2=0x55c9d8b38650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd39c001eb0 tx=0x7fd39c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 -- 172.21.15.19:0/1042608459 shutdown_connections 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b20d40 0x55c9d8b10a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b16480 0x55c9d8b38b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 --2- 172.21.15.19:0/1042608459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b15b60 0x55c9d8b38650 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 -- 172.21.15.19:0/1042608459 >> 172.21.15.19:0/1042608459 conn(0x55c9d8b120f0 msgr2=0x55c9d8b1ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 -- 172.21.15.19:0/1042608459 shutdown_connections 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.533+0000 7fd3b6228740 1 -- 172.21.15.19:0/1042608459 wait complete. 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 -- start start 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 Processor -- start 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b15b60 0x55c9d8b29450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1ae990 con 0x55c9d8b16480 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1ae2c0 con 0x55c9d8b20d40 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac1ab500 con 0x55c9d8b15b60 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b6228740 10 client.0 _ll_register_callbacks cb 0x7fd39c003670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:32.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b1beb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42638/0 (socket says 172.21.15.19:42638) 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b1beb640 1 -- 172.21.15.19:0/2688396687 learned_addr learned my addr 172.21.15.19:0/2688396687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b15b60 0x55c9d8b29450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.534+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 -- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b15b60 msgr2=0x55c9d8b29450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b15b60 0x55c9d8b29450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 -- 172.21.15.19:0/2688396687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 msgr2=0x55c9d8b24cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b24cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 -- 172.21.15.19:0/2688396687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8b25510 con 0x55c9d8b16480 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b1beb640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009cd0 tx=0x7fd3a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10248b4dddd980c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a801e040 con 0x55c9d8b16480 2024-09-12T03:23:32.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd39c031540 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd3a80050b0 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd3a80052b0 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b6228740 10 client.16299 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c9d8b9b690 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.535+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd39c031540 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fd3a800f6d0 con 0x55c9d8b16480 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 10 client.16299 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 20 client.16299 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b9bd00 0x55c9d8b9e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 4 client.16299 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c9d8a82d00 con 0x55c9d8b9bd00 2024-09-12T03:23:32.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b9bd00 0x55c9d8b9e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3b13ea640 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b9bd00 0x55c9d8b9e120 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f9721e8783f7fb server_cookie=69a1e2725fadc132 in_seq=0 out_seq=0 2024-09-12T03:23:32.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.536+0000 7fd3957fa640 10 client.16299 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3957fa640 1 -- 172.21.15.19:0/2688396687 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c9d8a82d00 con 0x55c9d8b9bd00 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3957fa640 10 client.16299 handle_command_reply: tid=0 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 client.16299 shutdown 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 msgr2=0x55c9d8b24780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009cd0 tx=0x7fd3a8009260 comp rx=0 tx=0).stop 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 shutdown_connections 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c9d8b9bd00 0x55c9d8b9e120 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c9d8b20d40 0x55c9d8b24cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c9d8b16480 0x55c9d8b24780 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3957fa640 0 client.16299 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 --2- 172.21.15.19:0/2688396687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c9d8b15b60 0x55c9d8b29450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 >> 172.21.15.19:0/2688396687 conn(0x55c9d8b120f0 msgr2=0x55c9d8b1db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 shutdown_connections 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 1 -- 172.21.15.19:0/2688396687 wait complete. 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.539+0000 7fd3b6228740 20 client.16299 trim_cache size 0 max 16384 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "765c8552-ec58-4dfb-b94c-200bc03610f5", 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:23:32.543 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 msgr2=0x7fd3980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 0x7fd3980a5040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40988e0 tx=0x7fd3a4067030 comp rx=0 tx=0).stop 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 msgr2=0x7fd3ac1a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac1a43c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009fd0 tx=0x7fd3a400ef10 comp rx=0 tx=0).stop 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 shutdown_connections 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd3ac102ef0 0x7fd3ac1a43c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd3ac102570 0x7fd3ac199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 --2- 172.21.15.19:0/4155870544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3ac100e20 0x7fd3ac199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 >> 172.21.15.19:0/4155870544 conn(0x7fd3ac0fc7d0 msgr2=0x7fd3ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 shutdown_connections 2024-09-12T03:23:32.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:32.540+0000 7fd3b3e4f640 1 -- 172.21.15.19:0/4155870544 wait complete. 2024-09-12T03:23:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:33 smithi135 ceph-mon[35474]: pgmap v472: 97 pgs: 97 active+clean; 1020 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-12T03:23:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:33 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:33 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:33.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:33 smithi121 ceph-mon[34963]: pgmap v472: 97 pgs: 97 active+clean; 1020 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-12T03:23:33.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:33 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:33.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:33 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:33 smithi019 ceph-mon[30953]: pgmap v472: 97 pgs: 97 active+clean; 1020 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 601 op/s 2024-09-12T03:23:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:33 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:33 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:33.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:23:34.217 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:34.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:23:34.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.506 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:34 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:34 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:34.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:34 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:34.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- 172.21.15.19:0/1657116463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c102d30 msgr2=0x7faa2c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 --2- 172.21.15.19:0/1657116463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c102d30 0x7faa2c103110 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7faa1c009900 tx=0x7faa1c02a480 comp rx=0 tx=0).stop 2024-09-12T03:23:34.728 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- 172.21.15.19:0/1657116463 shutdown_connections 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 --2- 172.21.15.19:0/1657116463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 --2- 172.21.15.19:0/1657116463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c103650 0x7faa2c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 --2- 172.21.15.19:0/1657116463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c102d30 0x7faa2c103110 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- 172.21.15.19:0/1657116463 >> 172.21.15.19:0/1657116463 conn(0x7faa2c0fc790 msgr2=0x7faa2c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- 172.21.15.19:0/1657116463 shutdown_connections 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- 172.21.15.19:0/1657116463 wait complete. 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 Processor -- start 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.724+0000 7faa317ac640 1 -- start start 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c102d30 0x7faa2c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111f60 con 0x7faa2c103650 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111de0 con 0x7faa2c10c160 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa317ac640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1120e0 con 0x7faa2c102d30 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42670/0 (socket says 172.21.15.19:42670) 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 -- 172.21.15.19:0/3285581049 learned_addr learned my addr 172.21.15.19:0/3285581049 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa2affd640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c102d30 0x7faa2c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 -- 172.21.15.19:0/3285581049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c102d30 msgr2=0x7faa2c198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c102d30 0x7faa2c198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 -- 172.21.15.19:0/3285581049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 msgr2=0x7faa2c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.730 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 -- 172.21.15.19:0/3285581049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c19d3d0 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.725+0000 7faa23fff640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faa1400d740 tx=0x7faa1400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.726+0000 7faa28ff9640 1 -- 172.21.15.19:0/3285581049 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14017070 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.726+0000 7faa28ff9640 1 -- 172.21.15.19:0/3285581049 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa1400de50 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.726+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c19d6c0 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.726+0000 7faa28ff9640 1 -- 172.21.15.19:0/3285581049 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1400f620 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.726+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c1a4fa0 con 0x7faa2c103650 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.728+0000 7faa33b85740 1 Processor -- start 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.728+0000 7faa33b85740 1 -- start start 2024-09-12T03:23:34.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.728+0000 7faa28ff9640 1 -- 172.21.15.19:0/3285581049 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa14009c30 con 0x7faa2c103650 2024-09-12T03:23:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf1fe880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf209140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf20ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599bf1642e0 con 0x5599bf1fe4a0 2024-09-12T03:23:34.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5599bf10bce0 con 0x5599bf209680 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa33b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599bf1602a0 con 0x5599bf1fedc0 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf209140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf1fe880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf209140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:53816/0 (socket says 172.21.15.19:53816) 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2affd640 1 -- 172.21.15.19:0/1906123987 learned_addr learned my addr 172.21.15.19:0/1906123987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa23fff640 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf20ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 -- 172.21.15.19:0/1906123987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 msgr2=0x5599bf209140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf209140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 -- 172.21.15.19:0/1906123987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 msgr2=0x5599bf20ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf20ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa2b7fe640 1 -- 172.21.15.19:0/1906123987 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bf0f0440 con 0x5599bf1fe4a0 2024-09-12T03:23:34.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.729+0000 7faa23fff640 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf20ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf1fe880 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7faa24005fd0 tx=0x7faa24011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2886c1b2ac4793f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa28ff9640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 0x7faa180a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa217fa640 1 -- 172.21.15.19:0/1906123987 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24004d20 con 0x5599bf1fe4a0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa217fa640 1 -- 172.21.15.19:0/1906123987 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2400f9a0 con 0x5599bf1fe4a0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa217fa640 1 -- 172.21.15.19:0/1906123987 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24004550 con 0x5599bf1fe4a0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa2affd640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 0x7faa180a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.730+0000 7faa28ff9640 1 -- 172.21.15.19:0/3285581049 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa140ceb00 con 0x7faa2c103650 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- 172.21.15.19:0/1906123987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 msgr2=0x5599bf1fe880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf1fe880 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7faa24005fd0 tx=0x7faa24011bf0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa2affd640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 0x7faa180a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0096c0 tx=0x7faa1c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- 172.21.15.19:0/1906123987 shutdown_connections 2024-09-12T03:23:34.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf20ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf209140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 --2- 172.21.15.19:0/1906123987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf1fe880 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- 172.21.15.19:0/1906123987 >> 172.21.15.19:0/1906123987 conn(0x5599bf1faa30 msgr2=0x5599bf1fae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- 172.21.15.19:0/1906123987 shutdown_connections 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- 172.21.15.19:0/1906123987 wait complete. 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 -- start start 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.731+0000 7faa33b85740 1 Processor -- start 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf206730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5599bf20c250 con 0x5599bf1fedc0 2024-09-12T03:23:34.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5599bf11f050 con 0x5599bf209680 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5599bf20c3d0 con 0x5599bf1fe4a0 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa33b85740 10 client.0 _ll_register_callbacks cb 0x5599bf15f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf206730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42684/0 (socket says 172.21.15.19:42684) 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa23fff640 1 -- 172.21.15.19:0/3832749588 learned_addr learned my addr 172.21.15.19:0/3832749588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 -- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 msgr2=0x5599bf208f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 -- 172.21.15.19:0/3832749588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 msgr2=0x5599bf2061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 -- 172.21.15.19:0/3832749588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599beed53c0 con 0x5599bf209680 2024-09-12T03:23:34.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa23fff640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf206730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa24003fd0 tx=0x7faa24003d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d80d6e2f535ad20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.732+0000 7faa127fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2401e040 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2400f9a0 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa24011100 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa24011300 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 10 client.25820 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599bf217bf0 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa2400f9a0 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7faa24018070 con 0x5599bf209680 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 10 client.25820 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 10 client.25820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:34.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 20 client.25820 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf219000 0x5599bf21b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 4 client.25820 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599bf171850 con 0x5599bf219000 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.733+0000 7faa2affd640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf219000 0x5599bf21b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.734+0000 7faa2affd640 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf219000 0x5599bf21b3f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6717ec508ad932c3 server_cookie=1fd38f1af562911a in_seq=0 out_seq=0 2024-09-12T03:23:34.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.734+0000 7faa127fc640 10 client.25820 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:34.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa127fc640 1 -- 172.21.15.19:0/3832749588 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599bf171850 con 0x5599bf219000 2024-09-12T03:23:34.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa127fc640 10 client.25820 handle_command_reply: tid=0 2024-09-12T03:23:34.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 client.25820 shutdown 2024-09-12T03:23:34.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 msgr2=0x5599bf206730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf206730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7faa24003fd0 tx=0x7faa24003d60 comp rx=0 tx=0).stop 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 shutdown_connections 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf219000 0x5599bf21b3f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf209680 0x5599bf206730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fedc0 0x5599bf2061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa127fc640 0 client.25820 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 --2- 172.21.15.19:0/3832749588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 >> 172.21.15.19:0/3832749588 conn(0x5599bf1faa30 msgr2=0x5599bf200710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 shutdown_connections 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 1 -- 172.21.15.19:0/3832749588 wait complete. 2024-09-12T03:23:34.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.736+0000 7faa33b85740 20 client.25820 trim_cache size 0 max 16384 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 Processor -- start 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 -- start start 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1faa30 0x5599bf21b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1af790 con 0x5599bf1faa30 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1b1390 con 0x5599bf1fedc0 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.754+0000 7faa33b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1afe60 con 0x5599bf1fe4a0 2024-09-12T03:23:34.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1faa30 0x5599bf21b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40440/0 (socket says 172.21.15.19:40440) 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 -- 172.21.15.19:0/95313073 learned_addr learned my addr 172.21.15.19:0/95313073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa23fff640 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 -- 172.21.15.19:0/95313073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 msgr2=0x5599bf208160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 -- 172.21.15.19:0/95313073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1faa30 msgr2=0x5599bf21b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1faa30 0x5599bf21b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 -- 172.21.15.19:0/95313073 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bf1ff330 con 0x5599bf1fedc0 2024-09-12T03:23:34.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa24003cb0 tx=0x7faa240052a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5b6245dbb466b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa20ff9640 1 -- 172.21.15.19:0/95313073 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2400fe60 con 0x5599bf1fedc0 2024-09-12T03:23:34.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa20ff9640 1 -- 172.21.15.19:0/95313073 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa24011100 con 0x5599bf1fedc0 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.755+0000 7faa20ff9640 1 -- 172.21.15.19:0/95313073 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240056c0 con 0x5599bf1fedc0 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 -- 172.21.15.19:0/95313073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 msgr2=0x5599bf220c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa24003cb0 tx=0x7faa240052a0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 -- 172.21.15.19:0/95313073 shutdown_connections 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1fedc0 0x5599bf220c50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fe4a0 0x5599bf208160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 --2- 172.21.15.19:0/95313073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1faa30 0x5599bf21b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 -- 172.21.15.19:0/95313073 >> 172.21.15.19:0/95313073 conn(0x5599bf206890 msgr2=0x5599bf207a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 -- 172.21.15.19:0/95313073 shutdown_connections 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.756+0000 7faa33b85740 1 -- 172.21.15.19:0/95313073 wait complete. 2024-09-12T03:23:34.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 -- start start 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 Processor -- start 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf27bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1b0c00 con 0x5599bf1fe4a0 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1b0530 con 0x5599bf1faa30 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1ad770 con 0x5599bf1fedc0 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa33b85740 10 client.0 _ll_register_callbacks cb 0x7faa1c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa2affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa23fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42712/0 (socket says 172.21.15.19:42712) 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa2affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40450/0 (socket says 172.21.15.19:40450) 2024-09-12T03:23:34.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.757+0000 7faa23fff640 1 -- 172.21.15.19:0/2432525799 learned_addr learned my addr 172.21.15.19:0/2432525799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2b7fe640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf27bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 -- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 msgr2=0x5599bf27bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf27bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 -- 172.21.15.19:0/2432525799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 msgr2=0x5599bf20f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 -- 172.21.15.19:0/2432525799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bf27c2a0 con 0x5599bf1faa30 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa23fff640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa2affd640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa1c000c00 tx=0x7faa1c03f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da7706f60efb55b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa127fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c035040 con 0x5599bf1faa30 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa24011100 con 0x5599bf1faa30 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faa1c003c00 con 0x5599bf1faa30 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.758+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7faa1c003e00 con 0x5599bf1faa30 2024-09-12T03:23:34.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 10 client.25826 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5599bf283680 con 0x5599bf1faa30 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa24011100 con 0x5599bf1faa30 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7faa1c04b6d0 con 0x5599bf1faa30 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 10 client.25826 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 20 client.25826 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf209680 0x5599bf285dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 4 client.25826 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599bf16bd60 con 0x5599bf209680 2024-09-12T03:23:34.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.759+0000 7faa23fff640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf209680 0x5599bf285dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.760+0000 7faa23fff640 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf209680 0x5599bf285dc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44ce8e9c96b09bea server_cookie=132f9f146a2bcec2 in_seq=0 out_seq=0 2024-09-12T03:23:34.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.760+0000 7faa127fc640 10 client.25826 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.761+0000 7faa127fc640 1 -- 172.21.15.19:0/2432525799 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x5599bf16bd60 con 0x5599bf209680 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.761+0000 7faa127fc640 10 client.25826 handle_command_reply: tid=0 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.761+0000 7faa33b85740 1 client.25826 shutdown 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.761+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 msgr2=0x5599bf20ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.761+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa1c000c00 tx=0x7faa1c03f070 comp rx=0 tx=0).stop 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 shutdown_connections 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5599bf1fedc0 0x5599bf27bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5599bf1fe4a0 0x5599bf20f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5599bf209680 0x5599bf285dc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 --2- 172.21.15.19:0/2432525799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5599bf1faa30 0x5599bf20ec90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa127fc640 0 client.25826 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 >> 172.21.15.19:0/2432525799 conn(0x5599bf206890 msgr2=0x5599bf208270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 shutdown_connections 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 1 -- 172.21.15.19:0/2432525799 wait complete. 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa33b85740 20 client.25826 trim_cache size 0 max 16384 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (6161 inodes in the stack)", 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "765c8552-ec58-4dfb-b94c-200bc03610f5": { 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "765c8552-ec58-4dfb-b94c-200bc03610f5", 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:23:34.766 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 msgr2=0x7faa180a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 0x7faa180a50c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0096c0 tx=0x7faa1c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 msgr2=0x7faa2c198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faa1400d740 tx=0x7faa1400dc00 comp rx=0 tx=0).stop 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 shutdown_connections 2024-09-12T03:23:34.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faa2c10c160 0x7faa2c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faa180a2c00 0x7faa180a50c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faa2c103650 0x7faa2c198960 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 --2- 172.21.15.19:0/3285581049 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faa2c102d30 0x7faa2c198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.762+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 >> 172.21.15.19:0/3285581049 conn(0x7faa2c0fc790 msgr2=0x7faa2c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.763+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 shutdown_connections 2024-09-12T03:23:34.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:34.763+0000 7faa317ac640 1 -- 172.21.15.19:0/3285581049 wait complete. 2024-09-12T03:23:35.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.076+0000 7f0f5f7fe640 1 -- 172.21.15.19:0/280093657 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5646c52ca410 con 0x5646c5361750 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.076+0000 7f0f5f7fe640 10 client.16194 handle_command_reply: tid=0 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.076+0000 7f0f8876f740 1 client.16194 shutdown 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 msgr2=0x5646c52ea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0f7409b190 tx=0x7f0f7409ac20 comp rx=0 tx=0).stop 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 shutdown_connections 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5646c5361750 0x5646c5363b70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646c52dda30 0x5646c52eac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f5f7fe640 0 client.16194 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646c52d3170 0x5646c52ea6e0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 --2- 172.21.15.19:0/280093657 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646c52d2850 0x5646c52ea1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 >> 172.21.15.19:0/280093657 conn(0x5646c52cede0 msgr2=0x5646c52d4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 shutdown_connections 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 1 -- 172.21.15.19:0/280093657 wait complete. 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 14, 2024-09-12T03:23:35.079 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "349877ad": { 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 14, 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.2, 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:23:35.080 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:35.081 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.077+0000 7f0f8876f740 20 client.16194 trim_cache size 0 max 16384 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.078+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 msgr2=0x7f0f680a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.078+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 0x7f0f680a50c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009230 tx=0x7f0f700058d0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.078+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 msgr2=0x7f0f800fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.078+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0f7400d770 tx=0x7f0f7400dc30 comp rx=0 tx=0).stop 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 shutdown_connections 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0f680a2c00 0x7f0f680a50c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0f80102ef0 0x7f0f800fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f80102570 0x7f0f800fd1c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 --2- 172.21.15.19:0/333939316 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0f80100e20 0x7f0f800fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 >> 172.21.15.19:0/333939316 conn(0x7f0f800fc7d0 msgr2=0x7f0f8006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 shutdown_connections 2024-09-12T03:23:35.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:35.079+0000 7f0f86396640 1 -- 172.21.15.19:0/333939316 wait complete. 2024-09-12T03:23:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:35 smithi019 ceph-mon[30953]: pgmap v473: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 660 op/s 2024-09-12T03:23:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:35 smithi135 ceph-mon[35474]: pgmap v473: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 660 op/s 2024-09-12T03:23:35.586 INFO:tasks.cephfs.filesystem:scrub status for tag:765c8552-ec58-4dfb-b94c-200bc03610f5 - {'path': '/', 'tag': '765c8552-ec58-4dfb-b94c-200bc03610f5', 'options': 'recursive,force'} 2024-09-12T03:23:35.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:35 smithi121 ceph-mon[34963]: pgmap v473: 97 pgs: 97 active+clean; 1.0 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.2 MiB/s wr, 660 op/s 2024-09-12T03:23:35.699 INFO:tasks.workunit.client.0.smithi019.stdout: 300 5933 0 3837 0 1197 0 2024-09-12T03:23:36.344 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '349877ad', quiesce took 23.1 seconds. Will release after: -2.950000000000003 2024-09-12T03:23:36.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:36 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:36 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:36.651 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:36.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:36 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 -- 172.21.15.19:0/2737613010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541082a0 msgr2=0x7f385410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 --2- 172.21.15.19:0/2737613010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541082a0 0x7f385410ec40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f384c009cd0 tx=0x7f384c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 -- 172.21.15.19:0/2737613010 shutdown_connections 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 --2- 172.21.15.19:0/2737613010 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541082a0 0x7f385410ec40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 --2- 172.21.15.19:0/2737613010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541036e0 0x7f3854107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 --2- 172.21.15.19:0/2737613010 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 0x7f3854103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 -- 172.21.15.19:0/2737613010 >> 172.21.15.19:0/2737613010 conn(0x7f38540fc7d0 msgr2=0x7f38540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 -- 172.21.15.19:0/2737613010 shutdown_connections 2024-09-12T03:23:37.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.185+0000 7f385add9640 1 -- 172.21.15.19:0/2737613010 wait complete. 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.186+0000 7f385add9640 1 Processor -- start 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.186+0000 7f385add9640 1 -- start start 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 0x7f3854198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541082a0 0x7f385419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f38541120a0 con 0x7f38541036e0 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3854111f20 con 0x7f38541082a0 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f385add9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3854112220 con 0x7f3854102d30 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42730/0 (socket says 172.21.15.19:42730) 2024-09-12T03:23:37.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 -- 172.21.15.19:0/4198661420 learned_addr learned my addr 172.21.15.19:0/4198661420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3858b75640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 0x7f3854198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3859376640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541082a0 0x7f385419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 -- 172.21.15.19:0/4198661420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 msgr2=0x7f3854198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 0x7f3854198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 -- 172.21.15.19:0/4198661420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541082a0 msgr2=0x7f385419ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541082a0 0x7f385419ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 -- 172.21.15.19:0/4198661420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385419d4e0 con 0x7f38541036e0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3853fff640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f384000e970 tx=0x7f384000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.187+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384000cd80 con 0x7f38541036e0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.188+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38400035f0 con 0x7f38541036e0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.188+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385419d7d0 con 0x7f38541036e0 2024-09-12T03:23:37.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.188+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3840010690 con 0x7f38541036e0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.188+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38541a50b0 con 0x7f38541036e0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.189+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3854103110 con 0x7f38541036e0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.190+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f38400037a0 con 0x7f38541036e0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.191+0000 7f3851ffb640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 0x7f38440a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.191+0000 7f3858b75640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 0x7f38440a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.191+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f38400cf1c0 con 0x7f38541036e0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.192+0000 7f3858b75640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 0x7f38440a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f384800bbe0 tx=0x7f384800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:37.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.196+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f384009c130 con 0x7f38541036e0 2024-09-12T03:23:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:37 smithi019 ceph-mon[30953]: pgmap v474: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-09-12T03:23:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:37 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.467+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f385410a2e0 con 0x7f38541036e0 2024-09-12T03:23:37.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.468+0000 7f3851ffb640 1 -- 172.21.15.19:0/4198661420 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f384009ba30 con 0x7f38541036e0 2024-09-12T03:23:37.469 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:23:37.469 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"btime":"2024-09-12T03:23:23:197279+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:23:23.152454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:23:37.469 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 msgr2=0x7f38440a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 0x7f38440a50c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f384800bbe0 tx=0x7f384800b390 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 msgr2=0x7f3854198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f384000e970 tx=0x7f384000ee30 comp rx=0 tx=0).stop 2024-09-12T03:23:37.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 shutdown_connections 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38541082a0 0x7f385419ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f38440a2c00 0x7f38440a50c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f38541036e0 0x7f3854198a70 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 --2- 172.21.15.19:0/4198661420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3854102d30 0x7f3854198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 >> 172.21.15.19:0/4198661420 conn(0x7f38540fc7d0 msgr2=0x7f3854105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 shutdown_connections 2024-09-12T03:23:37.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:37.471+0000 7f385add9640 1 -- 172.21.15.19:0/4198661420 wait complete. 2024-09-12T03:23:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:37 smithi135 ceph-mon[35474]: pgmap v474: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-09-12T03:23:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:37 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:37 smithi121 ceph-mon[34963]: pgmap v474: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 6.9 MiB/s wr, 624 op/s 2024-09-12T03:23:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:37 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4198661420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4198661420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:38.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4198661420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:39 smithi135 ceph-mon[35474]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-09-12T03:23:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:39.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:39 smithi121 ceph-mon[34963]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-09-12T03:23:39.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:39 smithi019 ceph-mon[30953]: pgmap v475: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 7.0 MiB/s wr, 646 op/s 2024-09-12T03:23:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:39.703 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 349877ad --release --await' 2024-09-12T03:23:39.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 349877ad --release --await 2024-09-12T03:23:39.984 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:40.360 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:40.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 -- 172.21.15.19:0/3220708729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0102d90 msgr2=0x7f96f0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3220708729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0102d90 0x7f96f0103170 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009930 tx=0x7f96ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:23:40.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 -- 172.21.15.19:0/3220708729 shutdown_connections 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3220708729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 0x7f96f010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3220708729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0103740 0x7f96f0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3220708729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0102d90 0x7f96f0103170 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 -- 172.21.15.19:0/3220708729 >> 172.21.15.19:0/3220708729 conn(0x7f96f00fc7d0 msgr2=0x7f96f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 -- 172.21.15.19:0/3220708729 shutdown_connections 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.583+0000 7f96f7af2640 1 -- 172.21.15.19:0/3220708729 wait complete. 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 Processor -- start 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 -- start start 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 0x7f96f01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 0x7f96f0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96f0112060 con 0x7f96f0103740 2024-09-12T03:23:40.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96f0111ee0 con 0x7f96f0108290 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f7af2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01121e0 con 0x7f96f0102d90 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f508d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f508d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41054/0 (socket says 172.21.15.19:41054) 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f588e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 0x7f96f01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f508d640 1 -- 172.21.15.19:0/3351668812 learned_addr learned my addr 172.21.15.19:0/3351668812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f608f640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 0x7f96f0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.584+0000 7f96f508d640 1 -- 172.21.15.19:0/3351668812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 msgr2=0x7f96f01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f508d640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 0x7f96f01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f508d640 1 -- 172.21.15.19:0/3351668812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 msgr2=0x7f96f0101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f508d640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 0x7f96f0101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f508d640 1 -- 172.21.15.19:0/3351668812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f01a6a60 con 0x7f96f0103740 2024-09-12T03:23:40.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f588e640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 0x7f96f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f508d640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9c0 tx=0x7f96e000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96e6ffd640 1 -- 172.21.15.19:0/3351668812 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e000cd80 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f01a6d50 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96e6ffd640 1 -- 172.21.15.19:0/3351668812 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e0005500 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96e6ffd640 1 -- 172.21.15.19:0/3351668812 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0010690 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.585+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f01a7250 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.587+0000 7f96e6ffd640 1 -- 172.21.15.19:0/3351668812 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96e00046c0 con 0x7f96f0103740 2024-09-12T03:23:40.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 Processor -- start 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 -- start start 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8ee6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8ee7240 0x557fc8ee9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557fc8e41620 con 0x557fc8edc060 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557fc8de8e50 con 0x557fc8edc980 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557fc8e3d5e0 con 0x557fc8ee7240 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f608f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f608f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41064/0 (socket says 172.21.15.19:41064) 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f608f640 1 -- 172.21.15.19:0/2503871833 learned_addr learned my addr 172.21.15.19:0/2503871833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f508d640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8ee7240 0x557fc8ee9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.588+0000 7f96f588e640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8ee6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96e6ffd640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 0x7f96dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 -- 172.21.15.19:0/2503871833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8ee7240 msgr2=0x557fc8ee9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8ee7240 0x557fc8ee9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 -- 172.21.15.19:0/2503871833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 msgr2=0x557fc8edc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 -- 172.21.15.19:0/2503871833 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc8dd9d40 con 0x557fc8edc980 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f608f640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f608f640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 0x7f96dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96e6ffd640 1 -- 172.21.15.19:0/3351668812 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f96e00cf3a0 con 0x7f96f0103740 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96f588e640 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8ee6d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005940 tx=0x7f96ec005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fdc27b09df4d790 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96dbfff640 1 -- 172.21.15.19:0/2503871833 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec035040 con 0x557fc8edc980 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96dbfff640 1 -- 172.21.15.19:0/2503871833 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ec031870 con 0x557fc8edc980 2024-09-12T03:23:40.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.589+0000 7f96dbfff640 1 -- 172.21.15.19:0/2503871833 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0047a0 con 0x557fc8edc980 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2503871833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 msgr2=0x557fc8ee6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8ee6d00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f96ec005940 tx=0x7f96ec005c00 comp rx=0 tx=0).stop 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f608f640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 0x7f96dc0a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96e80107d0 tx=0x7f96e8004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2503871833 shutdown_connections 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8ee7240 0x557fc8ee9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8ee6d00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2503871833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc060 0x557fc8edc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2503871833 >> 172.21.15.19:0/2503871833 conn(0x557fc8ed85f0 msgr2=0x557fc8ed89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2503871833 shutdown_connections 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.590+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2503871833 wait complete. 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 -- start start 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 Processor -- start 2024-09-12T03:23:40.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 0x557fc8ee4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8ee4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557fc8ee9e10 con 0x557fc8ee7240 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557fc8dc9f10 con 0x557fc8edc060 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557fc8ee9f90 con 0x557fc8edc980 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f9ecb740 10 client.0 _ll_register_callbacks cb 0x557fc8deddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f608f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8ee4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f508d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f588e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 0x557fc8ee4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f508d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55960/0 (socket says 172.21.15.19:55960) 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f588e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 0x557fc8ee4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42610/0 (socket says 172.21.15.19:42610) 2024-09-12T03:23:40.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.591+0000 7f96f508d640 1 -- 172.21.15.19:0/2068672590 learned_addr learned my addr 172.21.15.19:0/2068672590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 -- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 msgr2=0x557fc8ee45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 -- 172.21.15.19:0/2068672590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 msgr2=0x557fc8ee4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 0x557fc8ee4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 -- 172.21.15.19:0/2068672590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc8b50580 con 0x557fc8ee7240 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f508d640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f608f640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8ee4b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f96e800f790 tx=0x7f96e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6a696cf6b8a0a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004d20 con 0x557fc8ee7240 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f96e8005020 con 0x557fc8ee7240 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96ec031870 con 0x557fc8ee7240 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96e8005220 con 0x557fc8ee7240 2024-09-12T03:23:40.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f9ecb740 10 client.16335 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.592+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557fc8c1a960 con 0x557fc8ee7240 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96ec031870 con 0x557fc8ee7240 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f96e8026070 con 0x557fc8ee7240 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 10 client.16335 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 10 client.16335 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 20 client.16335 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8ef6d80 0x557fc8ef91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 4 client.16335 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557fc8e4eb90 con 0x557fc8ef6d80 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f588e640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8ef6d80 0x557fc8ef91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96f588e640 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8ef6d80 0x557fc8ef91e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=132e820657fbc21 server_cookie=e545939660a628ab in_seq=0 out_seq=0 2024-09-12T03:23:40.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.593+0000 7f96d97fa640 10 client.16335 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.595+0000 7f96d97fa640 1 -- 172.21.15.19:0/2068672590 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557fc8e4eb90 con 0x557fc8ef6d80 2024-09-12T03:23:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.595+0000 7f96d97fa640 10 client.16335 handle_command_reply: tid=0 2024-09-12T03:23:40.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 client.16335 shutdown 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 msgr2=0x557fc8ee4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8ee4b00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f96e800f790 tx=0x7f96e8004ab0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 shutdown_connections 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8ee4b00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc980 0x557fc8ee45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8ef6d80 0x557fc8ef91e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/2068672590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc060 0x557fc8ee4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96d97fa640 0 client.16335 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 >> 172.21.15.19:0/2068672590 conn(0x557fc8ed85f0 msgr2=0x557fc8ed7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 shutdown_connections 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 1 -- 172.21.15.19:0/2068672590 wait complete. 2024-09-12T03:23:40.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.596+0000 7f96f9ecb740 20 client.16335 trim_cache size 0 max 16384 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 Processor -- start 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 -- start start 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8ede1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8ee7240 0x557fc8efeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01af7f0 con 0x557fc8edc980 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b13f0 con 0x557fc8ee7240 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01afec0 con 0x557fc8edc060 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f508d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f508d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41080/0 (socket says 172.21.15.19:41080) 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f508d640 1 -- 172.21.15.19:0/914475155 learned_addr learned my addr 172.21.15.19:0/914475155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.614+0000 7f96f588e640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8ede1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f608f640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8ee7240 0x557fc8efeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 -- 172.21.15.19:0/914475155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 msgr2=0x557fc8ede1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8ede1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 -- 172.21.15.19:0/914475155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8ee7240 msgr2=0x557fc8efeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8ee7240 0x557fc8efeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 -- 172.21.15.19:0/914475155 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fc8ee9f30 con 0x557fc8edc980 2024-09-12T03:23:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f588e640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8ede1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96f508d640 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96e00985e0 tx=0x7f96e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebb5cb3829a70a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96db7fe640 1 -- 172.21.15.19:0/914475155 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0080040 con 0x557fc8edc980 2024-09-12T03:23:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96db7fe640 1 -- 172.21.15.19:0/914475155 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e009c310 con 0x557fc8edc980 2024-09-12T03:23:40.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.615+0000 7f96db7fe640 1 -- 172.21.15.19:0/914475155 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e00a43f0 con 0x557fc8edc980 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- 172.21.15.19:0/914475155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 msgr2=0x557fc8efeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f96e00985e0 tx=0x7f96e00057b0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- 172.21.15.19:0/914475155 shutdown_connections 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8ee7240 0x557fc8efeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8edc980 0x557fc8efeab0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/914475155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8ede1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- 172.21.15.19:0/914475155 >> 172.21.15.19:0/914475155 conn(0x557fc8ed85f0 msgr2=0x557fc8eddab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:40.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- 172.21.15.19:0/914475155 shutdown_connections 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- 172.21.15.19:0/914475155 wait complete. 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.616+0000 7f96f9ecb740 1 -- start start 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 Processor -- start 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8eef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8eef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b0c60 con 0x557fc8ee7240 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01b0590 con 0x557fc8edc980 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f96f01ad7d0 con 0x557fc8edc060 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f9ecb740 10 client.0 _ll_register_callbacks cb 0x7f96e800fda0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41088/0 (socket says 172.21.15.19:41088) 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 -- 172.21.15.19:0/1287407904 learned_addr learned my addr 172.21.15.19:0/1287407904 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f588e640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8eef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f508d640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8eef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 -- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 msgr2=0x557fc8eef340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8eef340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 -- 172.21.15.19:0/1287407904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 msgr2=0x557fc8eef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8eef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.617+0000 7f96f608f640 1 -- 172.21.15.19:0/1287407904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fc8eebf90 con 0x557fc8ee7240 2024-09-12T03:23:40.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96f588e640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8eef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:23:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96f608f640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96e800f830 tx=0x7f96e8005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bad521ff121e45b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e8004b20 con 0x557fc8ee7240 2024-09-12T03:23:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96e009c310 con 0x557fc8ee7240 2024-09-12T03:23:40.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f96e8004cc0 con 0x557fc8ee7240 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96e8004ec0 con 0x557fc8ee7240 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96f9ecb740 10 client.16341 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557fc8f62570 con 0x557fc8ee7240 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.618+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f96e009c310 con 0x557fc8ee7240 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f96e8026040 con 0x557fc8ee7240 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 10 client.16341 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 10 client.16341 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 20 client.16341 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8f62d20 0x557fc8f65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 4 client.16341 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "349877ad", "release": true, "await": true}] 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "349877ad", "release": true, "await": true}) -- 0x557fc8e490a0 con 0x557fc8f62d20 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f588e640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8f62d20 0x557fc8f65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:40.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96d97fa640 10 client.16341 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:40.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:40.619+0000 7f96f588e640 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8f62d20 0x557fc8f65140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85f6c2257eafa44c server_cookie=588c4695e0ac8ac9 in_seq=0 out_seq=0 2024-09-12T03:23:40.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96d97fa640 1 -- 172.21.15.19:0/1287407904 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557fc8e490a0 con 0x557fc8f62d20 2024-09-12T03:23:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96d97fa640 10 client.16341 handle_command_reply: tid=0 2024-09-12T03:23:41.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 client.16341 shutdown 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 msgr2=0x557fc8eeb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f96e800f830 tx=0x7f96e8005fa0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 shutdown_connections 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557fc8ee7240 0x557fc8eeb760 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557fc8f62d20 0x557fc8f65140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96d97fa640 0 client.16341 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557fc8edc980 0x557fc8eef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.370+0000 7f96f9ecb740 1 --2- 172.21.15.19:0/1287407904 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557fc8edc060 0x557fc8eef340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.371+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 >> 172.21.15.19:0/1287407904 conn(0x557fc8ed85f0 msgr2=0x557fc8ee4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.371+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 shutdown_connections 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.371+0000 7f96f9ecb740 1 -- 172.21.15.19:0/1287407904 wait complete. 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.371+0000 7f96f9ecb740 20 client.16341 trim_cache size 0 max 16384 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:23:41.372 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 16, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "349877ad": { 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 16, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.2, 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:23:41.373 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:41.374 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 msgr2=0x7f96dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 0x7f96dc0a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f96e80107d0 tx=0x7f96e8004a40 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 msgr2=0x7f96f0100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e9c0 tx=0x7f96e000ee80 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 shutdown_connections 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96f0108290 0x7f96f0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f96dc0a2b80 0x7f96dc0a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f96f0103740 0x7f96f0100f30 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 --2- 172.21.15.19:0/3351668812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f96f0102d90 0x7f96f01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:41.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 >> 172.21.15.19:0/3351668812 conn(0x7f96f00fc7d0 msgr2=0x7f96f0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.372+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 shutdown_connections 2024-09-12T03:23:41.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:41.373+0000 7f96f7af2640 1 -- 172.21.15.19:0/3351668812 wait complete. 2024-09-12T03:23:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:41 smithi135 ceph-mon[35474]: pgmap v476: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-09-12T03:23:41.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:41 smithi121 ceph-mon[34963]: pgmap v476: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-09-12T03:23:41.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:41 smithi019 ceph-mon[30953]: pgmap v476: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 572 op/s 2024-09-12T03:23:42.518 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '349877ad', total seconds elapsed: 29.3 2024-09-12T03:23:42.518 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:23:42.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:23:42.829 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.393+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3449128945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41082a0 msgr2=0x7fd4f410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.393+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3449128945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41082a0 0x7fd4f410ec40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009cd0 tx=0x7fd4ec02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3449128945 shutdown_connections 2024-09-12T03:23:43.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3449128945 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41082a0 0x7fd4f410ec40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3449128945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f41036e0 0x7fd4f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3449128945 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f4102d30 0x7fd4f4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3449128945 >> 172.21.15.19:0/3449128945 conn(0x7fd4f40fc7d0 msgr2=0x7fd4f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3449128945 shutdown_connections 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3449128945 wait complete. 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.394+0000 7fd4fb79b640 1 Processor -- start 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 -- start start 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 0x7fd4f4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f41082a0 0x7fd4f4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f41120d0 con 0x7fd4f41036e0 2024-09-12T03:23:43.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4111f50 con 0x7fd4f4102d30 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4fb79b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4112250 con 0x7fd4f41082a0 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4f9537640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 0x7fd4f4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4f8d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4f8d36640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41116/0 (socket says 172.21.15.19:41116) 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4f8d36640 1 -- 172.21.15.19:0/3199112373 learned_addr learned my addr 172.21.15.19:0/3199112373 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.395+0000 7fd4f9d38640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f41082a0 0x7fd4f4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 -- 172.21.15.19:0/3199112373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f41082a0 msgr2=0x7fd4f4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f41082a0 0x7fd4f4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 -- 172.21.15.19:0/3199112373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 msgr2=0x7fd4f4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 0x7fd4f4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 -- 172.21.15.19:0/3199112373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f41a6a30 con 0x7fd4f41036e0 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f9537640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 0x7fd4f4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:43.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4f8d36640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e970 tx=0x7fd4e400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cd80 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41a6d20 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e40035f0 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4010690 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.396+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f41a7250 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.398+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4103110 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.400+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd4e40037a0 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.400+0000 7fd4ea7fc640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 0x7fd4e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.401+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd4e40cf340 con 0x7fd4f41036e0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.401+0000 7fd4f9537640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 0x7fd4e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.401+0000 7fd4f9537640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 0x7fd4e00a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009fc0 tx=0x7fd4f0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:43.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.404+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4e409c2b0 con 0x7fd4f41036e0 2024-09-12T03:23:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:43 smithi135 ceph-mon[35474]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-09-12T03:23:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:43 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:23:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:43 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:43 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:43 smithi121 ceph-mon[34963]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-09-12T03:23:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:43 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:23:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:43 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:43 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.657+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4f4101ec0 con 0x7fd4f41036e0 2024-09-12T03:23:43.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:43 smithi019 ceph-mon[30953]: pgmap v477: 97 pgs: 97 active+clean; 1.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.2 MiB/s wr, 360 op/s 2024-09-12T03:23:43.657 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:43 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:23:43.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:43 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.658 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:43 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:23:43.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.658+0000 7fd4ea7fc640 1 -- 172.21.15.19:0/3199112373 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7fd4f4101ec0 con 0x7fd4f41036e0 2024-09-12T03:23:43.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:23:43.659 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"btime":"2024-09-12T03:23:23:197279+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:23:23.152454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:23:43.659 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 msgr2=0x7fd4e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 0x7fd4e00a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0009fc0 tx=0x7fd4f0005f90 comp rx=0 tx=0).stop 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 msgr2=0x7fd4f4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e970 tx=0x7fd4e400ee30 comp rx=0 tx=0).stop 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 shutdown_connections 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd4f41082a0 0x7fd4f4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd4e00a2b80 0x7fd4e00a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd4f41036e0 0x7fd4f4100ed0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 --2- 172.21.15.19:0/3199112373 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f4102d30 0x7fd4f4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.660+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 >> 172.21.15.19:0/3199112373 conn(0x7fd4f40fc7d0 msgr2=0x7fd4f4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.661+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 shutdown_connections 2024-09-12T03:23:43.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:43.661+0000 7fd4fb79b640 1 -- 172.21.15.19:0/3199112373 wait complete. 2024-09-12T03:23:44.536 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:44 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3199112373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:44.543 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:44 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3199112373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:44.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:44 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3199112373' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:23:45.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:23:45.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:23:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:45 smithi135 ceph-mon[35474]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 542 op/s 2024-09-12T03:23:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:45 smithi121 ceph-mon[34963]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 542 op/s 2024-09-12T03:23:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:45 smithi019 ceph-mon[30953]: pgmap v478: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 10 MiB/s wr, 542 op/s 2024-09-12T03:23:45.699 INFO:tasks.workunit.client.0.smithi019.stdout: 315 114241 732 80082 616 18670 121 2024-09-12T03:23:45.713 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.366+0000 7f7897955640 1 -- 172.21.15.19:0/3694734086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f789011b140 msgr2=0x7f789011d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.366+0000 7f7897955640 1 --2- 172.21.15.19:0/3694734086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f789011b140 0x7f789011d530 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002a420 comp rx=0 tx=0).stop 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.368+0000 7f7897955640 1 -- 172.21.15.19:0/3694734086 shutdown_connections 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.368+0000 7f7897955640 1 --2- 172.21.15.19:0/3694734086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f789011b140 0x7f789011d530 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.368+0000 7f7897955640 1 --2- 172.21.15.19:0/3694734086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890109830 0x7f7890109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.368+0000 7f7897955640 1 --2- 172.21.15.19:0/3694734086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7890109450 0x7f7890110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.368+0000 7f7897955640 1 -- 172.21.15.19:0/3694734086 >> 172.21.15.19:0/3694734086 conn(0x7f789006e140 msgr2=0x7f789006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.370+0000 7f7897955640 1 -- 172.21.15.19:0/3694734086 shutdown_connections 2024-09-12T03:23:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.372+0000 7f7897955640 1 -- 172.21.15.19:0/3694734086 wait complete. 2024-09-12T03:23:46.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.373+0000 7f7897955640 1 Processor -- start 2024-09-12T03:23:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 -- start start 2024-09-12T03:23:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 0x7f78901a7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890109830 0x7f78901a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7890120de0 con 0x7f7890109450 2024-09-12T03:23:46.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7890120c60 con 0x7f789011b140 2024-09-12T03:23:46.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7897955640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7890120f60 con 0x7f7890109830 2024-09-12T03:23:46.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.375+0000 7f7895ef2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f7895ef2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42650/0 (socket says 172.21.15.19:42650) 2024-09-12T03:23:46.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f7895ef2640 1 -- 172.21.15.19:0/3027388751 learned_addr learned my addr 172.21.15.19:0/3027388751 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:46.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f78956f1640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 0x7f78901a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f7895ef2640 1 -- 172.21.15.19:0/3027388751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890109830 msgr2=0x7f78901a77a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f7895ef2640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890109830 0x7f78901a77a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.376+0000 7f7895ef2640 1 -- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 msgr2=0x7f78901a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.377+0000 7f7895ef2640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 0x7f78901a7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.377+0000 7f7895ef2640 1 -- 172.21.15.19:0/3027388751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78901ac210 con 0x7f789011b140 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.377+0000 7f78956f1640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 0x7f78901a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.377+0000 7f7895ef2640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f788002a990 tx=0x7f7880004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.378+0000 7f78867fc640 1 -- 172.21.15.19:0/3027388751 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788003f030 con 0x7f789011b140 2024-09-12T03:23:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.378+0000 7f78867fc640 1 -- 172.21.15.19:0/3027388751 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7880003ad0 con 0x7f789011b140 2024-09-12T03:23:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.378+0000 7f78867fc640 1 -- 172.21.15.19:0/3027388751 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880035070 con 0x7f789011b140 2024-09-12T03:23:46.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.378+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78901ac4a0 con 0x7f789011b140 2024-09-12T03:23:46.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.378+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78901ac930 con 0x7f789011b140 2024-09-12T03:23:46.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.384+0000 7f7899d2e740 1 Processor -- start 2024-09-12T03:23:46.387 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.387+0000 7f78867fc640 1 -- 172.21.15.19:0/3027388751 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7880042030 con 0x7f789011b140 2024-09-12T03:23:46.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 -- start start 2024-09-12T03:23:46.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162814c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x559162821e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559162779160 con 0x559162814850 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559162720720 con 0x55916281fa30 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.388+0000 7f7899d2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559162775120 con 0x559162815170 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.389+0000 7f78956f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.389+0000 7f78956f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:56008/0 (socket says 172.21.15.19:56008) 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.389+0000 7f78956f1640 1 -- 172.21.15.19:0/267581222 learned_addr learned my addr 172.21.15.19:0/267581222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.390+0000 7f78867fc640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 0x7f787c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.390+0000 7f78867fc640 1 -- 172.21.15.19:0/3027388751 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f78800ef380 con 0x7f789011b140 2024-09-12T03:23:46.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.391+0000 7f7895ef2640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 0x7f787c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.392+0000 7f7895ef2640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 0x7f787c0a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78800b7ff0 tx=0x7f788002a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.392+0000 7f7894ef0640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162814c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.392+0000 7f7894ef0640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x559162821e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.392+0000 7f78956f1640 1 -- 172.21.15.19:0/267581222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 msgr2=0x559162821e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.393+0000 7f78956f1640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x559162821e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.393+0000 7f78956f1640 1 -- 172.21.15.19:0/267581222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 msgr2=0x559162814c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.393+0000 7f78956f1640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162814c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.393+0000 7f78956f1640 1 -- 172.21.15.19:0/267581222 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591624eaba0 con 0x559162815170 2024-09-12T03:23:46.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.393+0000 7f7894ef0640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x559162821e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:23:46.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.395+0000 7f78956f1640 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7888005f40 tx=0x7f7888011d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6566596eb118c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.396+0000 7f787b7fe640 1 -- 172.21.15.19:0/267581222 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888011330 con 0x559162815170 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.396+0000 7f787b7fe640 1 -- 172.21.15.19:0/267581222 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788800fe80 con 0x559162815170 2024-09-12T03:23:46.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.396+0000 7f787b7fe640 1 -- 172.21.15.19:0/267581222 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78880193e0 con 0x559162815170 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.397+0000 7f7899d2e740 1 -- 172.21.15.19:0/267581222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 msgr2=0x55916281f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.397+0000 7f7899d2e740 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7888005f40 tx=0x7f7888011d30 comp rx=0 tx=0).stop 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 -- 172.21.15.19:0/267581222 shutdown_connections 2024-09-12T03:23:46.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x559162821e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x55916281f4f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 --2- 172.21.15.19:0/267581222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162814c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 -- 172.21.15.19:0/267581222 >> 172.21.15.19:0/267581222 conn(0x559162810de0 msgr2=0x5591628111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 -- 172.21.15.19:0/267581222 shutdown_connections 2024-09-12T03:23:46.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.398+0000 7f7899d2e740 1 -- 172.21.15.19:0/267581222 wait complete. 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.399+0000 7f7899d2e740 1 -- start start 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.399+0000 7f7899d2e740 1 Processor -- start 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.399+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162814850 0x55916281c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55916281fa30 0x55916281d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559162822650 con 0x55916281fa30 2024-09-12T03:23:46.400 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559162701900 con 0x559162815170 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5591628227d0 con 0x559162814850 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f7899d2e740 10 client.0 _ll_register_callbacks cb 0x5591626ff780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f78956f1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f78956f1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42674/0 (socket says 172.21.15.19:42674) 2024-09-12T03:23:46.401 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.400+0000 7f78956f1640 1 -- 172.21.15.19:0/1373920372 learned_addr learned my addr 172.21.15.19:0/1373920372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 -- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162814850 msgr2=0x55916281c910 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162814850 0x55916281c910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 -- 172.21.15.19:0/1373920372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55916281fa30 msgr2=0x55916281d390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55916281fa30 0x55916281d390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 -- 172.21.15.19:0/1373920372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559162741c50 con 0x559162815170 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f78956f1640 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7888010ca0 tx=0x7f78880113d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0e3e94da24a624 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f7878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:23:46.402 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888029060 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7888004a40 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.401+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559162786890 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.402+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f788801f040 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.402+0000 7f7899d2e740 10 client.25865 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.402+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55916282e350 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.402+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559162786890 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.402+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f7888018070 con 0x559162815170 2024-09-12T03:23:46.403 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 10 client.25865 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 10 client.25865 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 20 client.25865 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x559162831c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 4 client.25865 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559162780be0 con 0x55916282f810 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.403+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x559162831c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.404+0000 7f7878ff9640 10 client.25865 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:46.404 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.404+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x559162831c00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283c11095e15342b server_cookie=c7e63be3993617cc in_seq=0 out_seq=0 2024-09-12T03:23:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7878ff9640 1 -- 172.21.15.19:0/1373920372 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559162780be0 con 0x55916282f810 2024-09-12T03:23:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7878ff9640 10 client.25865 handle_command_reply: tid=0 2024-09-12T03:23:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 client.25865 shutdown 2024-09-12T03:23:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 msgr2=0x55916281ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.406 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7888010ca0 tx=0x7f78880113d0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 shutdown_connections 2024-09-12T03:23:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55916281fa30 0x55916281d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x559162831c00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7878ff9640 0 client.25865 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162815170 0x55916281ce50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1373920372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162814850 0x55916281c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 >> 172.21.15.19:0/1373920372 conn(0x559162810de0 msgr2=0x559162816b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 shutdown_connections 2024-09-12T03:23:46.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.406+0000 7f7899d2e740 1 -- 172.21.15.19:0/1373920372 wait complete. 2024-09-12T03:23:46.409 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.407+0000 7f7899d2e740 20 client.25865 trim_cache size 0 max 16384 2024-09-12T03:23:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.427+0000 7f7899d2e740 1 Processor -- start 2024-09-12T03:23:46.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.427+0000 7f7899d2e740 1 -- start start 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.427+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.427+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 0x5591628172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.427+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55916281fa30 0x5591628373c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7899d2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78901bc2b0 con 0x559162815170 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7899d2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78901bdeb0 con 0x559162814850 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7899d2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78901bc980 con 0x55916281fa30 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:42680/0 (socket says 172.21.15.19:42680) 2024-09-12T03:23:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 -- 172.21.15.19:0/1716540993 learned_addr learned my addr 172.21.15.19:0/1716540993 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55916281fa30 0x5591628373c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f78956f1640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 0x5591628172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 -- 172.21.15.19:0/1716540993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55916281fa30 msgr2=0x5591628373c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55916281fa30 0x5591628373c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 -- 172.21.15.19:0/1716540993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 msgr2=0x5591628172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 0x5591628172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f7894ef0640 1 -- 172.21.15.19:0/1716540993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591628145d0 con 0x559162814850 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.428+0000 7f78956f1640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 0x5591628172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.429+0000 7f7894ef0640 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f788c004820 tx=0x7f788c006f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e1226be3b137189 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.429+0000 7f787affd640 1 -- 172.21.15.19:0/1716540993 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00f040 con 0x559162814850 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.429+0000 7f787affd640 1 -- 172.21.15.19:0/1716540993 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c004500 con 0x559162814850 2024-09-12T03:23:46.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.429+0000 7f787affd640 1 -- 172.21.15.19:0/1716540993 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c015e00 con 0x559162814850 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 -- 172.21.15.19:0/1716540993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 msgr2=0x55916281b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f788c004820 tx=0x7f788c006f30 comp rx=0 tx=0).stop 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 -- 172.21.15.19:0/1716540993 shutdown_connections 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55916281fa30 0x5591628373c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162815170 0x5591628172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.431 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 --2- 172.21.15.19:0/1716540993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559162814850 0x55916281b850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.430+0000 7f7899d2e740 1 -- 172.21.15.19:0/1716540993 >> 172.21.15.19:0/1716540993 conn(0x559162810de0 msgr2=0x55916281e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.431+0000 7f7899d2e740 1 -- 172.21.15.19:0/1716540993 shutdown_connections 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.431+0000 7f7899d2e740 1 -- 172.21.15.19:0/1716540993 wait complete. 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.431+0000 7f7899d2e740 1 -- start start 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.431+0000 7f7899d2e740 1 Processor -- start 2024-09-12T03:23:46.432 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162823750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x5591628241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78901bd720 con 0x559162814850 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78901bd050 con 0x55916281fa30 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78901ba290 con 0x559162815170 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7899d2e740 10 client.0 _ll_register_callbacks cb 0x7f7888011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:56046/0 (socket says 172.21.15.19:56046) 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 -- 172.21.15.19:0/3313814979 learned_addr learned my addr 172.21.15.19:0/3313814979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:23:46.433 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f7894ef0640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162823750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 -- 172.21.15.19:0/3313814979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 msgr2=0x5591628241d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:23:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x5591628241d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 -- 172.21.15.19:0/3313814979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 msgr2=0x559162823750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:23:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162823750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:46.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.432+0000 7f78956f1640 1 -- 172.21.15.19:0/3313814979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559162899390 con 0x559162815170 2024-09-12T03:23:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f78956f1640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f78880117f0 tx=0x7f7888011130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3101122cb69cbbed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:23:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f7878ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:23:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7888034020 con 0x559162815170 2024-09-12T03:23:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f788c004500 con 0x559162815170 2024-09-12T03:23:46.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f788800fae0 con 0x559162815170 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.433+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7888003970 con 0x559162815170 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 10 client.25882 fetch_fsmap learned FSMap version 32 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55916289a5b0 con 0x559162815170 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c004500 con 0x559162815170 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f7888018070 con 0x559162815170 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 10 client.25882 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 10 client.25882 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 20 client.25882 populate_metadata read hostname 'smithi019' 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x55916289ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:23:46.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 4 client.25882 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:23:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5591626fcf40 con 0x55916282f810 2024-09-12T03:23:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.434+0000 7f7894ef0640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x55916289ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:23:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.435+0000 7f7878ff9640 10 client.25882 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:46.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:46.435+0000 7f7894ef0640 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x55916289ce10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c136b1225676f081 server_cookie=7ca8041a82ac8745 in_seq=0 out_seq=0 2024-09-12T03:23:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:47 smithi135 ceph-mon[35474]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 515 op/s 2024-09-12T03:23:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:47 smithi121 ceph-mon[34963]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 515 op/s 2024-09-12T03:23:47.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:47 smithi019 ceph-mon[30953]: pgmap v479: 97 pgs: 97 active+clean; 1.1 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 9.0 MiB/s wr, 515 op/s 2024-09-12T03:23:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:49 smithi135 ceph-mon[35474]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.8 MiB/s wr, 638 op/s 2024-09-12T03:23:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:49 smithi121 ceph-mon[34963]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.8 MiB/s wr, 638 op/s 2024-09-12T03:23:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:49 smithi019 ceph-mon[30953]: pgmap v480: 97 pgs: 97 active+clean; 1.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 9.8 MiB/s wr, 638 op/s 2024-09-12T03:23:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:51 smithi135 ceph-mon[35474]: pgmap v481: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 644 op/s 2024-09-12T03:23:51.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:51 smithi121 ceph-mon[34963]: pgmap v481: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 644 op/s 2024-09-12T03:23:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:51 smithi019 ceph-mon[30953]: pgmap v481: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 10 MiB/s wr, 644 op/s 2024-09-12T03:23:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:53 smithi135 ceph-mon[35474]: pgmap v482: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-12T03:23:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:53 smithi121 ceph-mon[34963]: pgmap v482: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-12T03:23:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:53 smithi019 ceph-mon[30953]: pgmap v482: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 10 MiB/s wr, 619 op/s 2024-09-12T03:23:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:23:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:23:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:23:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:23:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:54 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:23:54.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:23:54.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:23:54.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:23:54.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:23:54.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:54 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:23:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:23:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:23:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:23:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:23:54.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:54 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7878ff9640 1 -- 172.21.15.19:0/3313814979 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5591626fcf40 con 0x55916282f810 2024-09-12T03:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7878ff9640 10 client.25882 handle_command_reply: tid=0 2024-09-12T03:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 client.25882 shutdown 2024-09-12T03:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 msgr2=0x559162823c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f78880117f0 tx=0x7f7888011130 comp rx=0 tx=0).stop 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 shutdown_connections 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55916282f810 0x55916289ce10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55916281fa30 0x5591628241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559162815170 0x559162823c90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7878ff9640 0 client.25882 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 --2- 172.21.15.19:0/3313814979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559162814850 0x559162823750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 >> 172.21.15.19:0/3313814979 conn(0x559162810de0 msgr2=0x55916282a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 shutdown_connections 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 1 -- 172.21.15.19:0/3313814979 wait complete. 2024-09-12T03:23:55.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7899d2e740 20 client.25882 trim_cache size 0 max 16384 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 18, 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "6252fa75": { 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 18, 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:23:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.2, 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 msgr2=0x7f787c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 0x7f787c0a5250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f78800b7ff0 tx=0x7f788002a420 comp rx=0 tx=0).stop 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 msgr2=0x7f78901abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.370+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f788002a990 tx=0x7f7880004610 comp rx=0 tx=0).stop 2024-09-12T03:23:55.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 shutdown_connections 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f789011b140 0x7f78901abb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7890109830 0x7f78901a77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f787c0a2d90 0x7f787c0a5250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 --2- 172.21.15.19:0/3027388751 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7890109450 0x7f78901a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 >> 172.21.15.19:0/3027388751 conn(0x7f789006e140 msgr2=0x7f789011be40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 shutdown_connections 2024-09-12T03:23:55.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:23:55.371+0000 7f7897955640 1 -- 172.21.15.19:0/3027388751 wait complete. 2024-09-12T03:23:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:55 smithi135 ceph-mon[35474]: pgmap v483: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-12T03:23:55.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:55 smithi121 ceph-mon[34963]: pgmap v483: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-12T03:23:55.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:55 smithi019 ceph-mon[30953]: pgmap v483: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 649 op/s 2024-09-12T03:23:55.699 INFO:tasks.workunit.client.0.smithi019.stdout: 322 83932 284 58968 312 14553 41 2024-09-12T03:23:56.204 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6252fa75', quiesce took 13.7 seconds. Will release after: 6.449999999999999 2024-09-12T03:23:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:57 smithi135 ceph-mon[35474]: pgmap v484: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-09-12T03:23:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:57 smithi121 ceph-mon[34963]: pgmap v484: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-09-12T03:23:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:57 smithi019 ceph-mon[30953]: pgmap v484: 97 pgs: 97 active+clean; 1.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.5 MiB/s wr, 370 op/s 2024-09-12T03:23:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:23:59 smithi135 ceph-mon[35474]: pgmap v485: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-12T03:23:59.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:23:59 smithi121 ceph-mon[34963]: pgmap v485: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-12T03:23:59.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:23:59 smithi019 ceph-mon[30953]: pgmap v485: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 5.2 MiB/s wr, 391 op/s 2024-09-12T03:24:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:01 smithi135 ceph-mon[35474]: pgmap v486: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-12T03:24:01.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:01 smithi121 ceph-mon[34963]: pgmap v486: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-12T03:24:01.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:01 smithi019 ceph-mon[30953]: pgmap v486: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-09-12T03:24:02.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:02.943 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:03.387 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:03 smithi019 ceph-mon[30953]: pgmap v487: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 -- 172.21.15.19:0/2737172990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac1037a0 msgr2=0x7f2cac10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 --2- 172.21.15.19:0/2737172990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac1037a0 0x7f2cac10bc00 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02a410 comp rx=0 tx=0).stop 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 -- 172.21.15.19:0/2737172990 shutdown_connections 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 --2- 172.21.15.19:0/2737172990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac10c2f0 0x7f2cac10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 --2- 172.21.15.19:0/2737172990 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac1037a0 0x7f2cac10bc00 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 --2- 172.21.15.19:0/2737172990 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac102df0 0x7f2cac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 -- 172.21.15.19:0/2737172990 >> 172.21.15.19:0/2737172990 conn(0x7f2cac0fc7d0 msgr2=0x7f2cac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 -- 172.21.15.19:0/2737172990 shutdown_connections 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.491+0000 7f2cb3493640 1 -- 172.21.15.19:0/2737172990 wait complete. 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 Processor -- start 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 -- start start 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 0x7f2cac198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 0x7f2cac198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac10c2f0 0x7f2cac19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1120d0 con 0x7f2cac10c2f0 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac111f50 con 0x7f2cac102df0 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.492+0000 7f2cb3493640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac112250 con 0x7f2cac1037a0 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb1a30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac10c2f0 0x7f2cac19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb0a2e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 0x7f2cac198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 0x7f2cac198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb0a2e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 0x7f2cac198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:32914/0 (socket says 172.21.15.19:32914) 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb0a2e640 1 -- 172.21.15.19:0/1595077334 learned_addr learned my addr 172.21.15.19:0/1595077334 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 -- 172.21.15.19:0/1595077334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 msgr2=0x7f2cac198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 0x7f2cac198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 -- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac10c2f0 msgr2=0x7f2cac19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac10c2f0 0x7f2cac19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 -- 172.21.15.19:0/1595077334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac19d5f0 con 0x7f2cac102df0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb122f640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 0x7f2cac198640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9f0 tx=0x7f2ca800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800ccc0 con 0x7f2cac102df0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ca8005020 con 0x7f2cac102df0 2024-09-12T03:24:03.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac19d8e0 con 0x7f2cac102df0 2024-09-12T03:24:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca80049e0 con 0x7f2cac102df0 2024-09-12T03:24:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.493+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac100980 con 0x7f2cac102df0 2024-09-12T03:24:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.494+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cac06a790 con 0x7f2cac102df0 2024-09-12T03:24:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.496+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ca8005560 con 0x7f2cac102df0 2024-09-12T03:24:03.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.496+0000 7f2ca27fc640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 0x7f2c980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.496+0000 7f2cb0a2e640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 0x7f2c980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.496+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2ca80cefe0 con 0x7f2cac102df0 2024-09-12T03:24:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.497+0000 7f2cb0a2e640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 0x7f2c980a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0fee40 tx=0x7f2c9c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:03.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.500+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ca80051d0 con 0x7f2cac102df0 2024-09-12T03:24:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:03 smithi135 ceph-mon[35474]: pgmap v487: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-12T03:24:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:03 smithi121 ceph-mon[34963]: pgmap v487: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.749+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2cac1031d0 con 0x7f2cac102df0 2024-09-12T03:24:03.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.749+0000 7f2ca27fc640 1 -- 172.21.15.19:0/1595077334 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f2cac1031d0 con 0x7f2cac102df0 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"btime":"2024-09-12T03:23:23:197279+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:23:23.152454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:24:03.750 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 msgr2=0x7f2c980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 0x7f2c980a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2cac0fee40 tx=0x7f2c9c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 msgr2=0x7f2cac198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 0x7f2cac198640 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2ca800e9f0 tx=0x7f2ca800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 shutdown_connections 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c980a2b80 0x7f2c980a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2cac10c2f0 0x7f2cac19cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2cac1037a0 0x7f2cac198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.751+0000 7f2cb3493640 1 --2- 172.21.15.19:0/1595077334 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2cac102df0 0x7f2cac198640 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.752+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 >> 172.21.15.19:0/1595077334 conn(0x7f2cac0fc7d0 msgr2=0x7f2cac0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:03.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.752+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 shutdown_connections 2024-09-12T03:24:03.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:03.752+0000 7f2cb3493640 1 -- 172.21.15.19:0/1595077334 wait complete. 2024-09-12T03:24:04.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:04 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1595077334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:04.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:04 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1595077334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:04.764 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 6252fa75 --release --await' 2024-09-12T03:24:04.764 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 6252fa75 --release --await 2024-09-12T03:24:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:04 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1595077334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:05.071 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:05.423 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:05 smithi019 ceph-mon[30953]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 470 op/s 2024-09-12T03:24:05.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:24:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:05 smithi121 ceph-mon[34963]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 470 op/s 2024-09-12T03:24:05.700 INFO:tasks.workunit.client.0.smithi019.stdout: 322 238294 0 166345 0 34403 0 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 -- 172.21.15.19:0/2609312300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698106b80 msgr2=0x7f1698106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 --2- 172.21.15.19:0/2609312300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698106b80 0x7f1698106f60 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1688009900 tx=0x7f168802a410 comp rx=0 tx=0).stop 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 -- 172.21.15.19:0/2609312300 shutdown_connections 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 --2- 172.21.15.19:0/2609312300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698100aa0 0x7f169810ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 --2- 172.21.15.19:0/2609312300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 0x7f1698100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 --2- 172.21.15.19:0/2609312300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698106b80 0x7f1698106f60 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 -- 172.21.15.19:0/2609312300 >> 172.21.15.19:0/2609312300 conn(0x7f1698077230 msgr2=0x7f16980fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 -- 172.21.15.19:0/2609312300 shutdown_connections 2024-09-12T03:24:05.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.767+0000 7f169da21640 1 -- 172.21.15.19:0/2609312300 wait complete. 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.768+0000 7f169da21640 1 Processor -- start 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 -- start start 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 0x7f16981985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698106b80 0x7f169819cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1698112050 con 0x7f1698100aa0 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111ed0 con 0x7f1698106b80 2024-09-12T03:24:05.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f169da21640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16981121d0 con 0x7f1698100100 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f16967fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 0x7f16981985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f16967fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41144/0 (socket says 172.21.15.19:41144) 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f16967fc640 1 -- 172.21.15.19:0/3897432352 learned_addr learned my addr 172.21.15.19:0/3897432352 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.769+0000 7f16977fe640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698106b80 0x7f169819cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 -- 172.21.15.19:0/3897432352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 msgr2=0x7f16981985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 0x7f16981985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 -- 172.21.15.19:0/3897432352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698106b80 msgr2=0x7f169819cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698106b80 0x7f169819cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 -- 172.21.15.19:0/3897432352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169819d580 con 0x7f1698100aa0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f16967fc640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d950 tx=0x7f168c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f169ca1f640 1 -- 172.21.15.19:0/3897432352 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c017070 con 0x7f1698100aa0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169819d870 con 0x7f1698100aa0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f169ca1f640 1 -- 172.21.15.19:0/3897432352 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f168c009c30 con 0x7f1698100aa0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.770+0000 7f169ca1f640 1 -- 172.21.15.19:0/3897432352 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168c00f6b0 con 0x7f1698100aa0 2024-09-12T03:24:05.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.771+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a5150 con 0x7f1698100aa0 2024-09-12T03:24:05.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.774+0000 7f169ca1f640 1 -- 172.21.15.19:0/3897432352 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f168c0109d0 con 0x7f1698100aa0 2024-09-12T03:24:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.775+0000 7f169fdfa740 1 Processor -- start 2024-09-12T03:24:05.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.775+0000 7f169fdfa740 1 -- start start 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.775+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74badd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169ca1f640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 0x7f16800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 0x55c74bae7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74bae8240 0x55c74baea630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169fdfa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c74ba42620 con 0x55c74bae8240 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169fdfa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c74b9e9e50 con 0x55c74badd060 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169fdfa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c74ba3e5e0 con 0x55c74badd980 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f16977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74badd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f16977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74badd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50288/0 (socket says 172.21.15.19:50288) 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f16977fe640 1 -- 172.21.15.19:0/1177055611 learned_addr learned my addr 172.21.15.19:0/1177055611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f169ca1f640 1 -- 172.21.15.19:0/3897432352 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f168c0cec50 con 0x7f1698100aa0 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f16967fc640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 0x7f16800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 0x55c74bae7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.776+0000 7f16977fe640 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74bae8240 0x55c74baea630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 -- 172.21.15.19:0/1177055611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 msgr2=0x55c74badd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74badd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 -- 172.21.15.19:0/1177055611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74bae8240 msgr2=0x55c74baea630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74bae8240 0x55c74baea630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 -- 172.21.15.19:0/1177055611 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74b9dad40 con 0x55c74badd980 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f16967fc640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 0x7f16800a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f168c09b4a0 tx=0x7f168c00bf60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 0x55c74bae7d00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1688005940 tx=0x7f1688005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=280843f7cd2fd13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f167dffb640 1 -- 172.21.15.19:0/1177055611 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688035040 con 0x55c74badd980 2024-09-12T03:24:05.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f167dffb640 1 -- 172.21.15.19:0/1177055611 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688031970 con 0x55c74badd980 2024-09-12T03:24:05.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.777+0000 7f167dffb640 1 -- 172.21.15.19:0/1177055611 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168803f7e0 con 0x55c74badd980 2024-09-12T03:24:05.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.778+0000 7f169fdfa740 1 -- 172.21.15.19:0/1177055611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 msgr2=0x55c74bae7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 0x55c74bae7d00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1688005940 tx=0x7f1688005be0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 -- 172.21.15.19:0/1177055611 shutdown_connections 2024-09-12T03:24:05.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74bae8240 0x55c74baea630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd980 0x55c74bae7d00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1177055611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74badd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 -- 172.21.15.19:0/1177055611 >> 172.21.15.19:0/1177055611 conn(0x55c74bad95f0 msgr2=0x55c74bad99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 -- 172.21.15.19:0/1177055611 shutdown_connections 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 -- 172.21.15.19:0/1177055611 wait complete. 2024-09-12T03:24:05.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.779+0000 7f169fdfa740 1 -- start start 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f169fdfa740 1 Processor -- start 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74bae5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.780+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41170/0 (socket says 172.21.15.19:41170) 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f16977fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74bae5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f16977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f16977fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50304/0 (socket says 172.21.15.19:50304) 2024-09-12T03:24:05.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 -- 172.21.15.19:0/346720556 learned_addr learned my addr 172.21.15.19:0/346720556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c74baeae10 con 0x55c74badd980 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c74b9caf10 con 0x55c74bae8240 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c74baeaf90 con 0x55c74badd060 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 -- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 msgr2=0x55c74bae5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74bae5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 -- 172.21.15.19:0/346720556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 msgr2=0x55c74bae5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f1696ffd640 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74b751580 con 0x55c74badd980 2024-09-12T03:24:05.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.781+0000 7f16977fe640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 10 client.0 _ll_register_callbacks cb 0x55c74b9eedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1696ffd640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f168802a930 tx=0x7f1688005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0f76f0da56779f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1676ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:05.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688031970 con 0x55c74badd980 2024-09-12T03:24:05.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1676ffd640 1 -- 172.21.15.19:0/346720556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688035040 con 0x55c74badd980 2024-09-12T03:24:05.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1676ffd640 1 -- 172.21.15.19:0/346720556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1688031770 con 0x55c74badd980 2024-09-12T03:24:05.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1676ffd640 1 -- 172.21.15.19:0/346720556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1688004030 con 0x55c74badd980 2024-09-12T03:24:05.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 10 client.16377 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:05.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55c74b81b960 con 0x55c74badd980 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f1676ffd640 1 -- 172.21.15.19:0/346720556 <== mon.0 v2:172.21.15.19:3300/0 4 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f16880041d0 con 0x55c74badd980 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 10 client.16377 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 10 client.16377 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.782+0000 7f169fdfa740 20 client.16377 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74baf78f0 0x55c74baf9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f169fdfa740 4 client.16377 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c74ba4fb90 con 0x55c74baf78f0 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f16977fe640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74baf78f0 0x55c74baf9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f1676ffd640 10 client.16377 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.783+0000 7f16977fe640 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74baf78f0 0x55c74baf9ce0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cfc264048772410 server_cookie=28397ef10c94b139 in_seq=0 out_seq=0 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f1676ffd640 1 -- 172.21.15.19:0/346720556 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c74ba4fb90 con 0x55c74baf78f0 2024-09-12T03:24:05.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f1676ffd640 10 client.16377 handle_command_reply: tid=0 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 client.16377 shutdown 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 msgr2=0x55c74bae56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f168802a930 tx=0x7f1688005e80 comp rx=0 tx=0).stop 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 shutdown_connections 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74baf78f0 0x55c74baf9ce0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74bae5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74bae56a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 --2- 172.21.15.19:0/346720556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74bae5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f1676ffd640 0 client.16377 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.784+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 >> 172.21.15.19:0/346720556 conn(0x55c74bad95f0 msgr2=0x55c74badf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.785+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 shutdown_connections 2024-09-12T03:24:05.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.785+0000 7f169fdfa740 1 -- 172.21.15.19:0/346720556 wait complete. 2024-09-12T03:24:05.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.785+0000 7f169fdfa740 20 client.16377 trim_cache size 0 max 16384 2024-09-12T03:24:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:05 smithi135 ceph-mon[35474]: pgmap v488: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.9 MiB/s wr, 470 op/s 2024-09-12T03:24:05.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 Processor -- start 2024-09-12T03:24:05.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 -- start start 2024-09-12T03:24:05.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74badfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74baffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ab200 con 0x55c74badd980 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ace00 con 0x55c74bae8240 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.803+0000 7f169fdfa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ab8d0 con 0x55c74badd060 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41176/0 (socket says 172.21.15.19:41176) 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 -- 172.21.15.19:0/1968835391 learned_addr learned my addr 172.21.15.19:0/1968835391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f16977fe640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74badfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f16977fe640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74baffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 -- 172.21.15.19:0/1968835391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 msgr2=0x55c74badfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74badfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 -- 172.21.15.19:0/1968835391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 msgr2=0x55c74baffa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74baffa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 -- 172.21.15.19:0/1968835391 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74bae4110 con 0x55c74badd980 2024-09-12T03:24:05.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f16977fe640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74baffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:05.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.804+0000 7f1696ffd640 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1688000c00 tx=0x7f168804d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb21f68278f9dcf5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.805+0000 7f167d7fa640 1 -- 172.21.15.19:0/1968835391 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688035040 con 0x55c74badd980 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.805+0000 7f167d7fa640 1 -- 172.21.15.19:0/1968835391 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688031970 con 0x55c74badd980 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.805+0000 7f167d7fa640 1 -- 172.21.15.19:0/1968835391 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688003a10 con 0x55c74badd980 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 -- 172.21.15.19:0/1968835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 msgr2=0x55c74baff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f1688000c00 tx=0x7f168804d000 comp rx=0 tx=0).stop 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 -- 172.21.15.19:0/1968835391 shutdown_connections 2024-09-12T03:24:05.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74bae8240 0x55c74baffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baff4f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 --2- 172.21.15.19:0/1968835391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74badd060 0x55c74badfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.806+0000 7f169fdfa740 1 -- 172.21.15.19:0/1968835391 >> 172.21.15.19:0/1968835391 conn(0x55c74bad95f0 msgr2=0x55c74bae69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:05.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.807+0000 7f169fdfa740 1 -- 172.21.15.19:0/1968835391 shutdown_connections 2024-09-12T03:24:05.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.807+0000 7f169fdfa740 1 -- 172.21.15.19:0/1968835391 wait complete. 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.807+0000 7f169fdfa740 1 -- start start 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.807+0000 7f169fdfa740 1 Processor -- start 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74baf0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74bae8240 0x55c74baed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f16981ac670 con 0x55c74badd980 2024-09-12T03:24:05.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16981abfa0 con 0x55c74badd060 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f16981a91e0 con 0x55c74bae8240 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f1696ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41184/0 (socket says 172.21.15.19:41184) 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f1696ffd640 1 -- 172.21.15.19:0/3701481625 learned_addr learned my addr 172.21.15.19:0/3701481625 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.808+0000 7f169fdfa740 10 client.0 _ll_register_callbacks cb 0x7f1688005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f16977fe640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74baf0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 -- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74bae8240 msgr2=0x55c74baed210 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74bae8240 0x55c74baed210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 -- 172.21.15.19:0/3701481625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 msgr2=0x55c74baf0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74baf0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 -- 172.21.15.19:0/3701481625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74badd7b0 con 0x55c74badd980 2024-09-12T03:24:05.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f16977fe640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74baf0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:05.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1696ffd640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f168802a930 tx=0x7f1688003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca54afe1d2ab7c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:05.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1676ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:05.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688035040 con 0x55c74badd980 2024-09-12T03:24:05.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1688031970 con 0x55c74badd980 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.809+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1688031770 con 0x55c74badd980 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.810+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f168803c440 con 0x55c74badd980 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.810+0000 7f169fdfa740 10 client.16383 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.810+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c74bb6bbb0 con 0x55c74badd980 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1688031970 con 0x55c74badd980 2024-09-12T03:24:05.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f1688052540 con 0x55c74badd980 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 10 client.16383 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 10 client.16383 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 20 client.16383 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74bb6c130 0x55c74bb6e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 4 client.16383 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6252fa75", "release": true, "await": true}] 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6252fa75", "release": true, "await": true}) -- 0x55c74ba4a0a0 con 0x55c74bb6c130 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.811+0000 7f16977fe640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74bb6c130 0x55c74bb6e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.812+0000 7f1676ffd640 10 client.16383 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:05.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:05.812+0000 7f16977fe640 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74bb6c130 0x55c74bb6e550 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98d3724ae09dd2fc server_cookie=d76a0ad60836114f in_seq=0 out_seq=0 2024-09-12T03:24:05.995 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:06.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:06 smithi019 ceph-mon[30953]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-09-12T03:24:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f1676ffd640 1 -- 172.21.15.19:0/3701481625 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c74ba4a0a0 con 0x55c74bb6c130 2024-09-12T03:24:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f1676ffd640 10 client.16383 handle_command_reply: tid=0 2024-09-12T03:24:06.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 client.16383 shutdown 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 msgr2=0x55c74baeccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f168802a930 tx=0x7f1688003de0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 shutdown_connections 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c74bae8240 0x55c74baed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c74badd980 0x55c74baeccd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.514+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c74bb6c130 0x55c74bb6e550 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169fdfa740 1 --2- 172.21.15.19:0/3701481625 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c74badd060 0x55c74baf0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f1676ffd640 0 client.16383 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 >> 172.21.15.19:0/3701481625 conn(0x55c74bad95f0 msgr2=0x55c74bae5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 shutdown_connections 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169fdfa740 1 -- 172.21.15.19:0/3701481625 wait complete. 2024-09-12T03:24:06.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169fdfa740 20 client.16383 trim_cache size 0 max 16384 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 32, 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 20, 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "6252fa75": { 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 20, 2024-09-12T03:24:06.517 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.2, 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:24:06.518 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:24:06.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 msgr2=0x7f16800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 0x7f16800a5040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f168c09b4a0 tx=0x7f168c00bf60 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 msgr2=0x7f1698198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.515+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f168c00d950 tx=0x7f168c00de10 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 shutdown_connections 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1698106b80 0x7f169819cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f16800a2b80 0x7f16800a5040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1698100aa0 0x7f1698198b10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 --2- 172.21.15.19:0/3897432352 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1698100100 0x7f16981985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 >> 172.21.15.19:0/3897432352 conn(0x7f1698077230 msgr2=0x7f1698104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 shutdown_connections 2024-09-12T03:24:06.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.516+0000 7f169da21640 1 -- 172.21.15.19:0/3897432352 wait complete. 2024-09-12T03:24:06.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:06 smithi121 ceph-mon[34963]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-09-12T03:24:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 -- 172.21.15.19:0/2080293392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035406ea30 msgr2=0x7f035406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 --2- 172.21.15.19:0/2080293392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035406ea30 0x7f035406ee70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f035001a870 tx=0x7f035003b380 comp rx=0 tx=0).stop 2024-09-12T03:24:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 -- 172.21.15.19:0/2080293392 shutdown_connections 2024-09-12T03:24:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 --2- 172.21.15.19:0/2080293392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f035411b140 0x7f035411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 --2- 172.21.15.19:0/2080293392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035406ea30 0x7f035406ee70 secure :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f035001a870 tx=0x7f035003b380 comp rx=0 tx=0).stop 2024-09-12T03:24:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 --2- 172.21.15.19:0/2080293392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035406e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.677+0000 7f035afb4640 1 -- 172.21.15.19:0/2080293392 >> 172.21.15.19:0/2080293392 conn(0x7f0354073180 msgr2=0x7f0354073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.681+0000 7f035afb4640 1 -- 172.21.15.19:0/2080293392 shutdown_connections 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.682+0000 7f035afb4640 1 -- 172.21.15.19:0/2080293392 wait complete. 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 Processor -- start 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 -- start start 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035411b140 0x7f0354116c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f0359fb2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f0359fb2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50342/0 (socket says 172.21.15.19:50342) 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f0359fb2640 1 -- 172.21.15.19:0/3722147827 learned_addr learned my addr 172.21.15.19:0/3722147827 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:06.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f03597b1640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035411b140 0x7f0354116c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03541172e0 0x7f03541ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0354113360 con 0x7f035411b140 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03541131e0 con 0x7f0354110190 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.683+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f03541134e0 con 0x7f03541172e0 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 -- 172.21.15.19:0/3722147827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03541172e0 msgr2=0x7f03541ccf70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03541172e0 0x7f03541ccf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 -- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035411b140 msgr2=0x7f0354116c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035411b140 0x7f0354116c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03541cd4b0 con 0x7f0354110190 2024-09-12T03:24:06.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.684+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f034c00ed80 tx=0x7f034c00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.685+0000 7f034affd640 1 -- 172.21.15.19:0/3722147827 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c01c070 con 0x7f0354110190 2024-09-12T03:24:06.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.685+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03541cd680 con 0x7f0354110190 2024-09-12T03:24:06.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.685+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03541cdad0 con 0x7f0354110190 2024-09-12T03:24:06.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.685+0000 7f034affd640 1 -- 172.21.15.19:0/3722147827 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034c017b20 con 0x7f0354110190 2024-09-12T03:24:06.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.685+0000 7f034affd640 1 -- 172.21.15.19:0/3722147827 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c00fc80 con 0x7f0354110190 2024-09-12T03:24:06.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.690+0000 7f035e5ef740 1 Processor -- start 2024-09-12T03:24:06.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.690+0000 7f035e5ef740 1 -- start start 2024-09-12T03:24:06.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.690+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f034affd640 1 -- 172.21.15.19:0/3722147827 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f034c01f070 con 0x7f0354110190 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1f90 0x5592687bc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687bec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035e5ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592687172e0 con 0x5592687bc850 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035e5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5592686bece0 con 0x5592687b1f90 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:32980/0 (socket says 172.21.15.19:32980) 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035a7b3640 1 -- 172.21.15.19:0/3236504670 learned_addr learned my addr 172.21.15.19:0/3236504670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.692+0000 7f035e5ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592687132a0 con 0x5592687b1670 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1f90 0x5592687bc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 -- 172.21.15.19:0/3236504670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1f90 msgr2=0x5592687bc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1f90 0x5592687bc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 -- 172.21.15.19:0/3236504670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 msgr2=0x5592687bec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687bec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 -- 172.21.15.19:0/3236504670 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592686a3440 con 0x5592687b1670 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.693+0000 7f035a7b3640 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0344009690 tx=0x7f0344009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c360e8d5112fbfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.694+0000 7f034affd640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 0x7f03400a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.697+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 0x7f03400a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.698+0000 7f034affd640 1 -- 172.21.15.19:0/3722147827 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f034c0d01f0 con 0x7f0354110190 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.699+0000 7f033ffff640 1 -- 172.21.15.19:0/3236504670 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344015020 con 0x5592687b1670 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.699+0000 7f033ffff640 1 -- 172.21.15.19:0/3236504670 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034400bda0 con 0x5592687b1670 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.701+0000 7f035e5ef740 1 -- 172.21.15.19:0/3236504670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 msgr2=0x5592687b1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.701+0000 7f035e5ef740 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0344009690 tx=0x7f0344009b50 comp rx=0 tx=0).stop 2024-09-12T03:24:06.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.701+0000 7f0359fb2640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 0x7f03400a5180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f034c019050 tx=0x7f034c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 -- 172.21.15.19:0/3236504670 shutdown_connections 2024-09-12T03:24:06.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687bec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1f90 0x5592687bc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 --2- 172.21.15.19:0/3236504670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1670 0x5592687b1a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 -- 172.21.15.19:0/3236504670 >> 172.21.15.19:0/3236504670 conn(0x5592687adc00 msgr2=0x5592687adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 -- 172.21.15.19:0/3236504670 shutdown_connections 2024-09-12T03:24:06.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.704+0000 7f035e5ef740 1 -- 172.21.15.19:0/3236504670 wait complete. 2024-09-12T03:24:06.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.705+0000 7f035e5ef740 1 -- start start 2024-09-12T03:24:06.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.705+0000 7f035e5ef740 1 Processor -- start 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.705+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687b93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687b9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5592687bf470 con 0x5592687bc850 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5592686d2050 con 0x5592687b1670 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5592687bf5f0 con 0x5592687b1f90 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.706+0000 7f035e5ef740 10 client.0 _ll_register_callbacks cb 0x559268712160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f03597b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687b93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50356/0 (socket says 172.21.15.19:50356) 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 -- 172.21.15.19:0/869787720 learned_addr learned my addr 172.21.15.19:0/869787720 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687b9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 -- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 msgr2=0x5592687b93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687b93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 -- 172.21.15.19:0/869787720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 msgr2=0x5592687b9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687b9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.707+0000 7f035a7b3640 1 -- 172.21.15.19:0/869787720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592684883c0 con 0x5592687b1670 2024-09-12T03:24:06.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.709+0000 7f035a7b3640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f034400b4b0 tx=0x7f0344008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3821a0de6f0b18b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.709+0000 7f033d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:24:06.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.709+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344028060 con 0x5592687b1670 2024-09-12T03:24:06.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.709+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034400bda0 con 0x5592687b1670 2024-09-12T03:24:06.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.709+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f03440105a0 con 0x5592687b1670 2024-09-12T03:24:06.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.710+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0344015070 con 0x5592687b1670 2024-09-12T03:24:06.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.710+0000 7f035e5ef740 10 client.25901 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:06.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.710+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5592687cae40 con 0x5592687b1670 2024-09-12T03:24:06.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.711+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f034400bda0 con 0x5592687b1670 2024-09-12T03:24:06.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.711+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f0344010c00 con 0x5592687b1670 2024-09-12T03:24:06.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 10 client.25901 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:06.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 10 client.25901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:24:06.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 20 client.25901 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:06.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592687cc730 0x5592687ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 4 client.25901 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:06.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.716+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559268724850 con 0x5592687cc730 2024-09-12T03:24:06.721 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.720+0000 7f03597b1640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592687cc730 0x5592687ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.730+0000 7f03597b1640 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592687cc730 0x5592687ceb70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed25247b0e6b41f1 server_cookie=dd7ffc410517a17b in_seq=0 out_seq=0 2024-09-12T03:24:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.732+0000 7f033d7fa640 10 client.25901 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:06.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.733+0000 7f033d7fa640 1 -- 172.21.15.19:0/869787720 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559268724850 con 0x5592687cc730 2024-09-12T03:24:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.733+0000 7f033d7fa640 10 client.25901 handle_command_reply: tid=0 2024-09-12T03:24:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.733+0000 7f035e5ef740 1 client.25901 shutdown 2024-09-12T03:24:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.734+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 msgr2=0x5592687bc120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.734+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f034400b4b0 tx=0x7f0344008040 comp rx=0 tx=0).stop 2024-09-12T03:24:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 shutdown_connections 2024-09-12T03:24:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687b9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687b93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592687cc730 0x5592687ceb70 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 --2- 172.21.15.19:0/869787720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687bc120 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.738+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 >> 172.21.15.19:0/869787720 conn(0x5592687adc00 msgr2=0x5592687b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.739+0000 7f033d7fa640 0 client.25901 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:06.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.739+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 shutdown_connections 2024-09-12T03:24:06.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.742+0000 7f035e5ef740 1 -- 172.21.15.19:0/869787720 wait complete. 2024-09-12T03:24:06.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.742+0000 7f035e5ef740 20 client.25901 trim_cache size 0 max 16384 2024-09-12T03:24:06.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.779+0000 7f035e5ef740 1 Processor -- start 2024-09-12T03:24:06.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.781+0000 7f035e5ef740 1 -- start start 2024-09-12T03:24:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.782+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.782+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.782+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.782+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50372/0 (socket says 172.21.15.19:50372) 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687d49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f035a7b3640 1 -- 172.21.15.19:0/1934179913 learned_addr learned my addr 172.21.15.19:0/1934179913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f03597b1640 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f033800e520 con 0x5592687bc850 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0338010120 con 0x5592687b1670 2024-09-12T03:24:06.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.783+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f033800ebf0 con 0x5592687b1f90 2024-09-12T03:24:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 -- 172.21.15.19:0/1934179913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 msgr2=0x5592687d4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687d4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 -- 172.21.15.19:0/1934179913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 msgr2=0x5592687d49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687d49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 -- 172.21.15.19:0/1934179913 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592687bc490 con 0x5592687b1670 2024-09-12T03:24:06.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.785+0000 7f035a7b3640 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0344009b90 tx=0x7f0344006270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cb5e4f4c0cf8e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.787+0000 7f033f7fe640 1 -- 172.21.15.19:0/1934179913 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344006360 con 0x5592687b1670 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.787+0000 7f033f7fe640 1 -- 172.21.15.19:0/1934179913 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03440091f0 con 0x5592687b1670 2024-09-12T03:24:06.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.787+0000 7f033f7fe640 1 -- 172.21.15.19:0/1934179913 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034400bcc0 con 0x5592687b1670 2024-09-12T03:24:06.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.791+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 msgr2=0x5592687c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.791+0000 7f035e5ef740 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0344009b90 tx=0x7f0344006270 comp rx=0 tx=0).stop 2024-09-12T03:24:06.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:06 smithi135 ceph-mon[35474]: pgmap v489: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.2 MiB/s wr, 437 op/s 2024-09-12T03:24:06.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.794+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 shutdown_connections 2024-09-12T03:24:06.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.794+0000 7f035e5ef740 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687bc850 0x5592687d49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.794+0000 7f035e5ef740 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687b1f90 0x5592687d4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.794+0000 7f035e5ef740 1 --2- 172.21.15.19:0/1934179913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c8750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.794+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 >> 172.21.15.19:0/1934179913 conn(0x5592687adc00 msgr2=0x5592687bb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.795+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 shutdown_connections 2024-09-12T03:24:06.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.796+0000 7f035e5ef740 1 -- 172.21.15.19:0/1934179913 wait complete. 2024-09-12T03:24:06.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.797+0000 7f035e5ef740 1 -- start start 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.797+0000 7f035e5ef740 1 Processor -- start 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.797+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687b1f90 0x5592687c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687bc850 0x559268835450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f033800f990 con 0x5592687b1f90 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f033800f2c0 con 0x5592687b1670 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f033800c500 con 0x5592687bc850 2024-09-12T03:24:06.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035e5ef740 10 client.0 _ll_register_callbacks cb 0x7f0344009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035a7b3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:50388/0 (socket says 172.21.15.19:50388) 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.798+0000 7f035a7b3640 1 -- 172.21.15.19:0/2172064836 learned_addr learned my addr 172.21.15.19:0/2172064836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.799+0000 7f03597b1640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687b1f90 0x5592687c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687bc850 0x559268835450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 -- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687bc850 msgr2=0x559268835450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687bc850 0x559268835450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 -- 172.21.15.19:0/2172064836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687b1f90 msgr2=0x5592687c1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:06.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687b1f90 0x5592687c1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.800+0000 7f035a7b3640 1 -- 172.21.15.19:0/2172064836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592687c2860 con 0x5592687b1670 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.802+0000 7f035a7b3640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03440090c0 tx=0x7f0344005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0850d0dfc2766d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.803+0000 7f033d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.803+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344033040 con 0x5592687b1670 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.803+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f034400bc40 con 0x5592687b1670 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.803+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03440091f0 con 0x5592687b1670 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.804+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0344028030 con 0x5592687b1670 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.805+0000 7f035e5ef740 10 client.25907 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:06.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.805+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559268836ab0 con 0x5592687b1670 2024-09-12T03:24:06.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0344005c70 con 0x5592687b1670 2024-09-12T03:24:06.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f0344003750 con 0x5592687b1670 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 10 client.25907 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 10 client.25907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 20 client.25907 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592688370a0 0x5592688394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 4 client.25907 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:24:06.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.806+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f03440091f0 con 0x5592688370a0 2024-09-12T03:24:06.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.808+0000 7f03597b1640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592688370a0 0x5592688394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:06.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.808+0000 7f03597b1640 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592688370a0 0x5592688394c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740deda12ca1d2d3 server_cookie=bc63b49c7a41192b in_seq=0 out_seq=0 2024-09-12T03:24:06.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.809+0000 7f033d7fa640 10 client.25907 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:06.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.812+0000 7f033d7fa640 1 -- 172.21.15.19:0/2172064836 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f03440091f0 con 0x5592688370a0 2024-09-12T03:24:06.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.812+0000 7f033d7fa640 10 client.25907 handle_command_reply: tid=0 2024-09-12T03:24:06.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.812+0000 7f035e5ef740 1 client.25907 shutdown 2024-09-12T03:24:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.815+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 msgr2=0x5592687c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.815+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03440090c0 tx=0x7f0344005fb0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 shutdown_connections 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5592687bc850 0x559268835450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5592687b1f90 0x5592687c1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5592688370a0 0x5592688394c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 --2- 172.21.15.19:0/2172064836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5592687b1670 0x5592687c1a40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 >> 172.21.15.19:0/2172064836 conn(0x5592687adc00 msgr2=0x5592687ac500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.817+0000 7f033d7fa640 0 client.25907 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.818+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 shutdown_connections 2024-09-12T03:24:06.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.818+0000 7f035e5ef740 1 -- 172.21.15.19:0/2172064836 wait complete. 2024-09-12T03:24:06.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.822+0000 7f035e5ef740 20 client.25907 trim_cache size 0 max 16384 2024-09-12T03:24:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:24:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:24:06.823 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:24:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:24:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 msgr2=0x7f03400a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 0x7f03400a5180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f034c019050 tx=0x7f034c004a40 comp rx=0 tx=0).stop 2024-09-12T03:24:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 msgr2=0x7f035411aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:06.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f034c00ed80 tx=0x7f034c00c6e0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 shutdown_connections 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f03541172e0 0x7f03541ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f03400a2cc0 0x7f03400a5180 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f035411b140 0x7f0354116c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 --2- 172.21.15.19:0/3722147827 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0354110190 0x7f035411aba0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 >> 172.21.15.19:0/3722147827 conn(0x7f0354073180 msgr2=0x7f035411b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.824+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 shutdown_connections 2024-09-12T03:24:06.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:06.825+0000 7f035afb4640 1 -- 172.21.15.19:0/3722147827 wait complete. 2024-09-12T03:24:08.156 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6252fa75', total seconds elapsed: 25.6 2024-09-12T03:24:08.590 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:24:08.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:24:08.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:08 smithi135 ceph-mon[35474]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.7 MiB/s wr, 459 op/s 2024-09-12T03:24:08.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:08 smithi121 ceph-mon[34963]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.7 MiB/s wr, 459 op/s 2024-09-12T03:24:08.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:08 smithi019 ceph-mon[30953]: pgmap v490: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.7 MiB/s wr, 459 op/s 2024-09-12T03:24:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:10 smithi135 ceph-mon[35474]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 780 KiB/s wr, 410 op/s 2024-09-12T03:24:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:10 smithi121 ceph-mon[34963]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 780 KiB/s wr, 410 op/s 2024-09-12T03:24:10.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:10 smithi019 ceph-mon[30953]: pgmap v491: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 780 KiB/s wr, 410 op/s 2024-09-12T03:24:12.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:12 smithi135 ceph-mon[35474]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 733 KiB/s wr, 388 op/s 2024-09-12T03:24:12.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:12 smithi121 ceph-mon[34963]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 733 KiB/s wr, 388 op/s 2024-09-12T03:24:12.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:12 smithi019 ceph-mon[30953]: pgmap v492: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 733 KiB/s wr, 388 op/s 2024-09-12T03:24:13.638 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:14.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:14 smithi019 ceph-mon[30953]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.6 MiB/s wr, 373 op/s 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 -- 172.21.15.19:0/3225256658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 msgr2=0x7f71bc11d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 --2- 172.21.15.19:0/3225256658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc11d530 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009ca0 tx=0x7f71b402a8f0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 -- 172.21.15.19:0/3225256658 shutdown_connections 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 --2- 172.21.15.19:0/3225256658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc11d530 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 --2- 172.21.15.19:0/3225256658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc06ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 --2- 172.21.15.19:0/3225256658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc06e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 -- 172.21.15.19:0/3225256658 >> 172.21.15.19:0/3225256658 conn(0x7f71bc073180 msgr2=0x7f71bc073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 -- 172.21.15.19:0/3225256658 shutdown_connections 2024-09-12T03:24:14.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.650+0000 7f71c1272640 1 -- 172.21.15.19:0/3225256658 wait complete. 2024-09-12T03:24:14.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.651+0000 7f71c1272640 1 Processor -- start 2024-09-12T03:24:14.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 -- start start 2024-09-12T03:24:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc120bf0 con 0x7f71bc11b140 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc120a70 con 0x7f71bc06ea30 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.652+0000 7f71c1272640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f71bc120d70 con 0x7f71bc110190 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.655+0000 7f71bb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.655+0000 7f71bb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57958/0 (socket says 172.21.15.19:57958) 2024-09-12T03:24:14.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.655+0000 7f71bb7fe640 1 -- 172.21.15.19:0/4070955253 learned_addr learned my addr 172.21.15.19:0/4070955253 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.655+0000 7f71c0a71640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.655+0000 7f71bbfff640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bb7fe640 1 -- 172.21.15.19:0/4070955253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 msgr2=0x7f71bc119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bb7fe640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bb7fe640 1 -- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 msgr2=0x7f71bc112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bb7fe640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bb7fe640 1 -- 172.21.15.19:0/4070955253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71bc113310 con 0x7f71bc110190 2024-09-12T03:24:14.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.656+0000 7f71bbfff640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:14.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.659+0000 7f71bb7fe640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00d950 tx=0x7f71ac00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.663+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac017070 con 0x7f71bc110190 2024-09-12T03:24:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.663+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71bc113600 con 0x7f71bc110190 2024-09-12T03:24:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.663+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71bc1175f0 con 0x7f71bc110190 2024-09-12T03:24:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.665+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ac009c30 con 0x7f71bc110190 2024-09-12T03:24:14.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.665+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac00f600 con 0x7f71bc110190 2024-09-12T03:24:14.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.668+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a8005290 con 0x7f71bc110190 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.684+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f71ac00f7a0 con 0x7f71bc110190 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.686+0000 7f71b97fa640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 0x7f71a40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.686+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f71ac0cea60 con 0x7f71bc110190 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.687+0000 7f71bbfff640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 0x7f71a40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:14.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.688+0000 7f71bbfff640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 0x7f71a40a5250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71b000a8b0 tx=0x7f71b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:14.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:14.694+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71ac09b970 con 0x7f71bc110190 2024-09-12T03:24:14.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:14 smithi135 ceph-mon[35474]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.6 MiB/s wr, 373 op/s 2024-09-12T03:24:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:14 smithi121 ceph-mon[34963]: pgmap v493: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 1.6 MiB/s wr, 373 op/s 2024-09-12T03:24:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.208+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71a8005580 con 0x7f71bc110190 2024-09-12T03:24:15.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.213+0000 7f71b97fa640 1 -- 172.21.15.19:0/4070955253 <== mon.1 v2:172.21.15.135:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11988 (secure 0 0 0) 0x7f71ac009de0 con 0x7f71bc110190 2024-09-12T03:24:15.214 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:24:15.214 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":32,"btime":"2024-09-12T03:23:23:197279+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:23:23.152454+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:24:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 32 2024-09-12T03:24:15.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.219+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 msgr2=0x7f71a40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.219+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 0x7f71a40a5250 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71b000a8b0 tx=0x7f71b0008040 comp rx=0 tx=0).stop 2024-09-12T03:24:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.219+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 msgr2=0x7f71bc1127e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:15.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.219+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00d950 tx=0x7f71ac00de10 comp rx=0 tx=0).stop 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 shutdown_connections 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f71a40a2d90 0x7f71a40a5250 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f71bc11b140 0x7f71bc112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f71bc110190 0x7f71bc1127e0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00d950 tx=0x7f71ac00de10 comp rx=0 tx=0).stop 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 --2- 172.21.15.19:0/4070955253 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f71bc06ea30 0x7f71bc119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.220+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 >> 172.21.15.19:0/4070955253 conn(0x7f71bc073180 msgr2=0x7f71bc06b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:15.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.221+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 shutdown_connections 2024-09-12T03:24:15.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:15.222+0000 7f71c1272640 1 -- 172.21.15.19:0/4070955253 wait complete. 2024-09-12T03:24:15.490 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4070955253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:15.700 INFO:tasks.workunit.client.0.smithi019.stdout: 356 1879322 1614 1316108 1640 309232 857 2024-09-12T03:24:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4070955253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:15.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4070955253' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:24:16.418 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:24:16.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:16 smithi019 ceph-mon[30953]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-12T03:24:16.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:16 smithi135 ceph-mon[35474]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-12T03:24:16.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:16 smithi121 ceph-mon[34963]: pgmap v494: 97 pgs: 97 active+clean; 1.2 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 333 op/s 2024-09-12T03:24:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:18 smithi135 ceph-mon[35474]: pgmap v495: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 554 op/s 2024-09-12T03:24:18.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:18 smithi121 ceph-mon[34963]: pgmap v495: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 554 op/s 2024-09-12T03:24:18.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:18 smithi019 ceph-mon[30953]: pgmap v495: 97 pgs: 97 active+clean; 1.3 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.1 MiB/s wr, 554 op/s 2024-09-12T03:24:20.530 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:20 smithi135 ceph-mon[35474]: pgmap v496: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 675 op/s 2024-09-12T03:24:20.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:20 smithi121 ceph-mon[34963]: pgmap v496: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 675 op/s 2024-09-12T03:24:20.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:20 smithi019 ceph-mon[30953]: pgmap v496: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 675 op/s 2024-09-12T03:24:21.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.884+0000 7fd0c359e640 1 -- 172.21.15.19:0/4033841640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c41068d0 msgr2=0x7fd0c4106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.884+0000 7fd0c359e640 1 --2- 172.21.15.19:0/4033841640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c41068d0 0x7fd0c4106cb0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac00d940 tx=0x7fd0ac02e620 comp rx=0 tx=0).stop 2024-09-12T03:24:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.894+0000 7fd0c359e640 1 -- 172.21.15.19:0/4033841640 shutdown_connections 2024-09-12T03:24:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.894+0000 7fd0c359e640 1 --2- 172.21.15.19:0/4033841640 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c4101270 0x7fd0c410d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.894+0000 7fd0c359e640 1 --2- 172.21.15.19:0/4033841640 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.894+0000 7fd0c359e640 1 --2- 172.21.15.19:0/4033841640 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c41068d0 0x7fd0c4106cb0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.894+0000 7fd0c359e640 1 -- 172.21.15.19:0/4033841640 >> 172.21.15.19:0/4033841640 conn(0x7fd0c40fc610 msgr2=0x7fd0c40fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:21.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.897+0000 7fd0c359e640 1 -- 172.21.15.19:0/4033841640 shutdown_connections 2024-09-12T03:24:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.897+0000 7fd0c359e640 1 -- 172.21.15.19:0/4033841640 wait complete. 2024-09-12T03:24:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.898+0000 7fd0c359e640 1 Processor -- start 2024-09-12T03:24:21.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.899+0000 7fd0c359e640 1 -- start start 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4073980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c4101270 0x7fd0c4073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 0x7fd0c406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4118500 con 0x7fd0c4101270 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4118380 con 0x7fd0c41008d0 2024-09-12T03:24:21.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.900+0000 7fd0c359e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4118680 con 0x7fd0c41068d0 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.901+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4073980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.901+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4073980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59484/0 (socket says 172.21.15.19:59484) 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.901+0000 7fd0c259c640 1 -- 172.21.15.19:0/3371345530 learned_addr learned my addr 172.21.15.19:0/3371345530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.901+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 0x7fd0c406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.901+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c4101270 0x7fd0c4073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.902+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/3371345530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 msgr2=0x7fd0c4073980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.902+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4073980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.902+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c4101270 msgr2=0x7fd0c4073ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.902+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c4101270 0x7fd0c4073ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.902+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/3371345530 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c406d4e0 con 0x7fd0c41068d0 2024-09-12T03:24:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.903+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 0x7fd0c406cc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00be20 tx=0x7fd0bc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.905+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/3371345530 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc019070 con 0x7fd0c41068d0 2024-09-12T03:24:21.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.905+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/3371345530 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0bc014920 con 0x7fd0c41068d0 2024-09-12T03:24:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.905+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/3371345530 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0129e0 con 0x7fd0c41068d0 2024-09-12T03:24:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.905+0000 7fd0c359e640 1 -- 172.21.15.19:0/3371345530 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c406d7d0 con 0x7fd0c41068d0 2024-09-12T03:24:21.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.905+0000 7fd0c359e640 1 -- 172.21.15.19:0/3371345530 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c4071740 con 0x7fd0c41068d0 2024-09-12T03:24:21.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.912+0000 7fd0cad97740 1 Processor -- start 2024-09-12T03:24:21.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.914+0000 7fd0cad97740 1 -- start start 2024-09-12T03:24:21.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 0x558a0496e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a04979710 0x558a0497bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a048d42f0 con 0x558a0496e970 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558a0487bcf0 con 0x558a0496fd70 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0cad97740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a048d02b0 con 0x558a04979710 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39414/0 (socket says 172.21.15.19:39414) 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292285435 learned_addr learned my addr 172.21.15.19:0/4292285435 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.916+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 0x558a0496e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.917+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a04979710 0x558a0497bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.920+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/3371345530 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0bc014ad0 con 0x7fd0c41068d0 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292285435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a04979710 msgr2=0x558a0497bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a04979710 0x558a0497bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292285435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 msgr2=0x558a0496e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 0x558a0496e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292285435 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a048603a0 con 0x558a0496e970 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.921+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 0x558a0496e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.922+0000 7fd0bb7fe640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 0x7fd0b00a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.922+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0121c0 tx=0x7fd0ac004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3254cae2f055858b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.923+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 0x7fd0b00a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.923+0000 7fd0c359e640 1 -- 172.21.15.19:0/4292285435 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac043070 con 0x558a0496e970 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.923+0000 7fd0c359e640 1 -- 172.21.15.19:0/4292285435 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac00c920 con 0x558a0496e970 2024-09-12T03:24:21.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.923+0000 7fd0c359e640 1 -- 172.21.15.19:0/4292285435 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac005360 con 0x558a0496e970 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.924+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292285435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 msgr2=0x558a0496edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.924+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac0121c0 tx=0x7fd0ac004db0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0bb7fe640 1 -- 172.21.15.19:0/3371345530 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd0bc099180 con 0x7fd0c41068d0 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 0x7fd0b00a51d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006a50 tx=0x7fd0b40069e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292285435 shutdown_connections 2024-09-12T03:24:21.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a04979710 0x558a0497bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a0496edd0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292285435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496fd70 0x558a0496e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.925+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292285435 >> 172.21.15.19:0/4292285435 conn(0x558a04937310 msgr2=0x558a0496b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:21.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.926+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292285435 shutdown_connections 2024-09-12T03:24:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.926+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292285435 wait complete. 2024-09-12T03:24:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.928+0000 7fd0cad97740 1 -- start start 2024-09-12T03:24:21.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.928+0000 7fd0cad97740 1 Processor -- start 2024-09-12T03:24:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a04976300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 0x558a04976840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558a0497c1c0 con 0x558a04979710 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558a0488f060 con 0x558a0496e970 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558a0497c340 con 0x558a0496fd70 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0cad97740 10 client.0 _ll_register_callbacks cb 0x558a045e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:21.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59490/0 (socket says 172.21.15.19:59490) 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.929+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292732819 learned_addr learned my addr 172.21.15.19:0/4292732819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 0x558a04976840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292732819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 msgr2=0x558a04976300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a04976300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292732819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 msgr2=0x558a04976840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 0x558a04976840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.930+0000 7fd0c259c640 1 -- 172.21.15.19:0/4292732819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a046453c0 con 0x558a0496e970 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.931+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 0x558a04976840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.931+0000 7fd0c259c640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005d40 tx=0x7fd0ac0123a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63625a270e91b276 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.931+0000 7fd0a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.932+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac043070 con 0x558a0496e970 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.932+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd0ac00d310 con 0x558a0496e970 2024-09-12T03:24:21.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.932+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac00c920 con 0x558a0496e970 2024-09-12T03:24:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.933+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0ac00d510 con 0x558a0496e970 2024-09-12T03:24:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.933+0000 7fd0cad97740 10 client.25916 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:21.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.933+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a04987b20 con 0x558a0496e970 2024-09-12T03:24:21.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.933+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac00c920 con 0x558a0496e970 2024-09-12T03:24:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fd0ac003c90 con 0x558a0496e970 2024-09-12T03:24:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 10 client.25916 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 10 client.25916 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:24:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 20 client.25916 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:21.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a04989180 0x558a0498b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 4 client.25916 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:21.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.935+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a048e1860 con 0x558a04989180 2024-09-12T03:24:21.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.940+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a04989180 0x558a0498b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:21.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.944+0000 7fd0bbfff640 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a04989180 0x558a0498b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa29c88bb1b6aa5 server_cookie=69cca1224b9e7a3 in_seq=0 out_seq=0 2024-09-12T03:24:21.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.945+0000 7fd0a9ffb640 10 client.25916 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:24:21.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.953+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/4292732819 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a048e1860 con 0x558a04989180 2024-09-12T03:24:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.953+0000 7fd0a9ffb640 10 client.25916 handle_command_reply: tid=0 2024-09-12T03:24:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.954+0000 7fd0cad97740 1 client.25916 shutdown 2024-09-12T03:24:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.955+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 msgr2=0x558a04979050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:21.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.955+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac005d40 tx=0x7fd0ac0123a0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 shutdown_connections 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a04979710 0x558a04976840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a04989180 0x558a0498b640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a04976300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 --2- 172.21.15.19:0/4292732819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a0496e970 0x558a04979050 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0a9ffb640 0 client.25916 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:24:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.956+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 >> 172.21.15.19:0/4292732819 conn(0x558a04937310 msgr2=0x558a0496a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.957+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 shutdown_connections 2024-09-12T03:24:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.959+0000 7fd0cad97740 1 -- 172.21.15.19:0/4292732819 wait complete. 2024-09-12T03:24:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:21.959+0000 7fd0cad97740 20 client.25916 trim_cache size 0 max 16384 2024-09-12T03:24:22.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.000+0000 7fd0cad97740 1 Processor -- start 2024-09-12T03:24:22.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.004+0000 7fd0cad97740 1 -- start start 2024-09-12T03:24:22.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.004+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a0497ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0c259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39428/0 (socket says 172.21.15.19:39428) 2024-09-12T03:24:22.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0cad97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4116120 con 0x558a0496e970 2024-09-12T03:24:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.005+0000 7fd0c2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.006+0000 7fd0c2d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59506/0 (socket says 172.21.15.19:59506) 2024-09-12T03:24:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.006+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/1520425086 learned_addr learned my addr 172.21.15.19:0/1520425086 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:22.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.006+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41c6440 con 0x558a04979710 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.006+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41167f0 con 0x558a0496fd70 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.007+0000 7fd0c259c640 1 -- 172.21.15.19:0/1520425086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 msgr2=0x558a0497ab00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.007+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a0497ab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.007+0000 7fd0c259c640 1 -- 172.21.15.19:0/1520425086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 msgr2=0x558a0497b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.007+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.007+0000 7fd0c259c640 1 -- 172.21.15.19:0/1520425086 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a0496b540 con 0x558a0496e970 2024-09-12T03:24:22.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.008+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac02e660 tx=0x7fd0ac02ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22599d9f4990937e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.009+0000 7fd0abfff640 1 -- 172.21.15.19:0/1520425086 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac043070 con 0x558a0496e970 2024-09-12T03:24:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.009+0000 7fd0abfff640 1 -- 172.21.15.19:0/1520425086 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac050070 con 0x558a0496e970 2024-09-12T03:24:22.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.009+0000 7fd0abfff640 1 -- 172.21.15.19:0/1520425086 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac005190 con 0x558a0496e970 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 msgr2=0x558a04985460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac02e660 tx=0x7fd0ac02ef00 comp rx=0 tx=0).stop 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 shutdown_connections 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0496fd70 0x558a0497ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1520425086 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496e970 0x558a04985460 secure :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac02e660 tx=0x7fd0ac02ef00 comp rx=0 tx=0).stop 2024-09-12T03:24:22.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.011+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 >> 172.21.15.19:0/1520425086 conn(0x558a04937310 msgr2=0x558a049762c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:22.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.018+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 shutdown_connections 2024-09-12T03:24:22.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.021+0000 7fd0cad97740 1 -- 172.21.15.19:0/1520425086 wait complete. 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.022+0000 7fd0cad97740 1 -- start start 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.022+0000 7fd0cad97740 1 Processor -- start 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 0x558a0497a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41c5cb0 con 0x558a0496fd70 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c4116ec0 con 0x558a04979710 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c41141b0 con 0x558a0497ab10 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0cad97740 10 client.0 _ll_register_callbacks cb 0x7fd0ac0120e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0c2d9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48648/0 (socket says 172.21.15.19:48648) 2024-09-12T03:24:22.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.023+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/1436199624 learned_addr learned my addr 172.21.15.19:0/1436199624 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 msgr2=0x558a0497a5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 0x558a0497a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/1436199624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 msgr2=0x558a0497a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 0x558a0497a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c2d9d640 1 -- 172.21.15.19:0/1436199624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a04976e40 con 0x558a0497ab10 2024-09-12T03:24:22.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.028+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 0x558a0497a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:22.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.029+0000 7fd0c2d9d640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc09ade0 tx=0x7fd0bc096d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eab14d1291bfca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:22.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.032+0000 7fd0a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:24:22.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.032+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc0b3070 con 0x558a0497ab10 2024-09-12T03:24:22.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.032+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd0bc097750 con 0x558a0497ab10 2024-09-12T03:24:22.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.034+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ac050070 con 0x558a0497ab10 2024-09-12T03:24:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.038+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc014e60 con 0x558a0497ab10 2024-09-12T03:24:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.038+0000 7fd0cad97740 10 client.25924 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:22.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.038+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558a04981be0 con 0x558a0497ab10 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0ac050070 con 0x558a0497ab10 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7fd0bc012b80 con 0x558a0497ab10 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 10 client.25924 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 10 client.25924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 20 client.25924 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a049f3f30 0x558a049f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:22.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 4 client.25924 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:22.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.040+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558a048dbd70 con 0x558a049f3f30 2024-09-12T03:24:22.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.041+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a049f3f30 0x558a049f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.043+0000 7fd0a9ffb640 10 client.25924 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:24:22.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.044+0000 7fd0c259c640 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a049f3f30 0x558a049f6350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=db329d534326a5f8 server_cookie=5ae44c50190ca49a in_seq=0 out_seq=0 2024-09-12T03:24:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.051+0000 7fd0a9ffb640 1 -- 172.21.15.19:0/1436199624 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558a048dbd70 con 0x558a049f3f30 2024-09-12T03:24:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.051+0000 7fd0a9ffb640 10 client.25924 handle_command_reply: tid=0 2024-09-12T03:24:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.051+0000 7fd0cad97740 1 client.25924 shutdown 2024-09-12T03:24:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.052+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 msgr2=0x558a049805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.052+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc09ade0 tx=0x7fd0bc096d00 comp rx=0 tx=0).stop 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 shutdown_connections 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x558a049f3f30 0x558a049f6350 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558a0497ab10 0x558a049805d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558a04979710 0x558a0497a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 --2- 172.21.15.19:0/1436199624 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558a0496fd70 0x558a0497a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0a9ffb640 0 client.25924 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:24:22.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 >> 172.21.15.19:0/1436199624 conn(0x558a04937310 msgr2=0x558a0496a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.053+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 shutdown_connections 2024-09-12T03:24:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.054+0000 7fd0cad97740 1 -- 172.21.15.19:0/1436199624 wait complete. 2024-09-12T03:24:22.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.054+0000 7fd0cad97740 20 client.25924 trim_cache size 0 max 16384 2024-09-12T03:24:22.054 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 msgr2=0x7fd0b00a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 0x7fd0b00a51d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4006a50 tx=0x7fd0b40069e0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 msgr2=0x7fd0c406cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 0x7fd0c406cc20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00be20 tx=0x7fd0bc00bf10 comp rx=0 tx=0).stop 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 shutdown_connections 2024-09-12T03:24:22.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0c41068d0 0x7fd0c406cc20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd0b00a2d10 0x7fd0b00a51d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0c4101270 0x7fd0c4073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 --2- 172.21.15.19:0/3371345530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0c41008d0 0x7fd0c4073980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.055+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 >> 172.21.15.19:0/3371345530 conn(0x7fd0c40fc610 msgr2=0x7fd0c40fcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.056+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 shutdown_connections 2024-09-12T03:24:22.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:22.056+0000 7fd0b97fa640 1 -- 172.21.15.19:0/3371345530 wait complete. 2024-09-12T03:24:22.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:22 smithi135 ceph-mon[35474]: pgmap v497: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 658 op/s 2024-09-12T03:24:22.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:22 smithi121 ceph-mon[34963]: pgmap v497: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 658 op/s 2024-09-12T03:24:22.906 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:22 smithi019 ceph-mon[30953]: pgmap v497: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 13 MiB/s wr, 658 op/s 2024-09-12T03:24:23.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:24:23.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:23 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:23.800 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:23 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:23.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:23 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:24:24.790 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:24 smithi019 ceph-mon[30953]: pgmap v498: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-09-12T03:24:24.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:24 smithi135 ceph-mon[35474]: pgmap v498: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-09-12T03:24:24.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:24 smithi121 ceph-mon[34963]: pgmap v498: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 13 MiB/s wr, 676 op/s 2024-09-12T03:24:25.700 INFO:tasks.workunit.client.0.smithi019.stdout: 391 2233370 2580 1531576 1806 386619 1888 2024-09-12T03:24:26.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:26 smithi135 ceph-mon[35474]: pgmap v499: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 669 op/s 2024-09-12T03:24:26.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:26 smithi121 ceph-mon[34963]: pgmap v499: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 669 op/s 2024-09-12T03:24:26.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:26 smithi019 ceph-mon[30953]: pgmap v499: 97 pgs: 97 active+clean; 1.3 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 13 MiB/s wr, 669 op/s 2024-09-12T03:24:28.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:28 smithi121 ceph-mon[34963]: pgmap v500: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T03:24:28.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:28 smithi019 ceph-mon[30953]: pgmap v500: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T03:24:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:28 smithi135 ceph-mon[35474]: pgmap v500: 97 pgs: 97 active+clean; 1.4 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 605 op/s 2024-09-12T03:24:30.006 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:30.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:30 smithi121 ceph-mon[34963]: pgmap v501: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 396 op/s 2024-09-12T03:24:30.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:30 smithi019 ceph-mon[30953]: pgmap v501: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 396 op/s 2024-09-12T03:24:31.043 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:30 smithi135 ceph-mon[35474]: pgmap v501: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.3 MiB/s wr, 396 op/s 2024-09-12T03:24:31.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.434+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/2784763491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109450 msgr2=0x7f6c78110250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.434+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/2784763491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109450 0x7f6c78110250 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009be0 tx=0x7f6c7002a710 comp rx=0 tx=0).stop 2024-09-12T03:24:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.444+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/2784763491 shutdown_connections 2024-09-12T03:24:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.444+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/2784763491 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c7811b140 0x7f6c7811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.444+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/2784763491 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109830 0x7f6c78109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.444+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/2784763491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109450 0x7f6c78110250 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.444+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/2784763491 >> 172.21.15.19:0/2784763491 conn(0x7f6c7806e140 msgr2=0x7f6c7806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.445+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/2784763491 shutdown_connections 2024-09-12T03:24:31.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.445+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/2784763491 wait complete. 2024-09-12T03:24:31.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.446+0000 7f6c7f1d6640 1 Processor -- start 2024-09-12T03:24:31.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.446+0000 7f6c7f1d6640 1 -- start start 2024-09-12T03:24:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.446+0000 7f6c7f1d6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7f1d6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109830 0x7f6c781a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7f1d6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c7811b140 0x7f6c781abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7f1d6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78120df0 con 0x7f6c78109450 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7f1d6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78120c70 con 0x7f6c78109830 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7f1d6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78120f70 con 0x7f6c7811b140 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40678/0 (socket says 172.21.15.19:40678) 2024-09-12T03:24:31.455 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 -- 172.21.15.19:0/3312197109 learned_addr learned my addr 172.21.15.19:0/3312197109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c77fff640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109830 0x7f6c781a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 -- 172.21.15.19:0/3312197109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c7811b140 msgr2=0x7f6c781abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c7811b140 0x7f6c781abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 -- 172.21.15.19:0/3312197109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109830 msgr2=0x7f6c781a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109830 0x7f6c781a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.447+0000 7f6c7cf72640 1 -- 172.21.15.19:0/3312197109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c781ac2e0 con 0x7f6c78109450 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.449+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009d00 tx=0x7f6c70031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.449+0000 7f6c75ffb640 1 -- 172.21.15.19:0/3312197109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70041020 con 0x7f6c78109450 2024-09-12T03:24:31.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.450+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c781ac570 con 0x7f6c78109450 2024-09-12T03:24:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.450+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7807a450 con 0x7f6c78109450 2024-09-12T03:24:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.454+0000 7f6c75ffb640 1 -- 172.21.15.19:0/3312197109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c70030d00 con 0x7f6c78109450 2024-09-12T03:24:31.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.454+0000 7f6c75ffb640 1 -- 172.21.15.19:0/3312197109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70036040 con 0x7f6c78109450 2024-09-12T03:24:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.471+0000 7f6c75ffb640 1 -- 172.21.15.19:0/3312197109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c70044020 con 0x7f6c78109450 2024-09-12T03:24:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.477+0000 7f6c75ffb640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 0x7f6c640a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.477+0000 7f6c77fff640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 0x7f6c640a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.478+0000 7f6c75ffb640 1 -- 172.21.15.19:0/3312197109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c700f0fc0 con 0x7f6c78109450 2024-09-12T03:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.481+0000 7f6c815af740 1 Processor -- start 2024-09-12T03:24:31.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.481+0000 7f6c77fff640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 0x7f6c640a5250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6c7811d940 tx=0x7f6c68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.484+0000 7f6c815af740 1 -- start start 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b442f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b443c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5b43952f0 con 0x55b5b442f5d0 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b5b433ccf0 con 0x55b5b443a370 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c815af740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5b43912b0 con 0x55b5b4430b40 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40692/0 (socket says 172.21.15.19:40692) 2024-09-12T03:24:31.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.485+0000 7f6c77fff640 1 -- 172.21.15.19:0/280386221 learned_addr learned my addr 172.21.15.19:0/280386221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.486+0000 7f6c77fff640 1 -- 172.21.15.19:0/280386221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 msgr2=0x55b5b442f090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.486+0000 7f6c77fff640 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b442f090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.486+0000 7f6c77fff640 1 -- 172.21.15.19:0/280386221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 msgr2=0x55b5b443c760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.486+0000 7f6c77fff640 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b443c760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.486+0000 7f6c77fff640 1 -- 172.21.15.19:0/280386221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b43213a0 con 0x55b5b442f5d0 2024-09-12T03:24:31.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.489+0000 7f6c77fff640 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e630 tx=0x7f6c68014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd337ace08d6644 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.490+0000 7f6c6effd640 1 -- 172.21.15.19:0/280386221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68004020 con 0x55b5b442f5d0 2024-09-12T03:24:31.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.490+0000 7f6c6effd640 1 -- 172.21.15.19:0/280386221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c680041c0 con 0x55b5b442f5d0 2024-09-12T03:24:31.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.491+0000 7f6c815af740 1 -- 172.21.15.19:0/280386221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 msgr2=0x55b5b442fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.491+0000 7f6c815af740 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e630 tx=0x7f6c68014000 comp rx=0 tx=0).stop 2024-09-12T03:24:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.496+0000 7f6c815af740 1 -- 172.21.15.19:0/280386221 shutdown_connections 2024-09-12T03:24:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.496+0000 7f6c815af740 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b443c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.496+0000 7f6c815af740 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b442fa10 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.496+0000 7f6c815af740 1 --2- 172.21.15.19:0/280386221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b442f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.496+0000 7f6c815af740 1 -- 172.21.15.19:0/280386221 >> 172.21.15.19:0/280386221 conn(0x55b5b43f8310 msgr2=0x55b5b442bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.504+0000 7f6c815af740 1 -- 172.21.15.19:0/280386221 shutdown_connections 2024-09-12T03:24:31.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.505+0000 7f6c815af740 1 -- 172.21.15.19:0/280386221 wait complete. 2024-09-12T03:24:31.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.506+0000 7f6c815af740 1 -- start start 2024-09-12T03:24:31.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.506+0000 7f6c815af740 1 Processor -- start 2024-09-12T03:24:31.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b4436fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b44375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5b443d0c0 con 0x55b5b442f5d0 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b5b4350060 con 0x55b5b443a370 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5b443d240 con 0x55b5b4430b40 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.508+0000 7f6c815af740 10 client.0 _ll_register_callbacks cb 0x55b5b40a38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.509+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.509+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40706/0 (socket says 172.21.15.19:40706) 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.509+0000 7f6c7cf72640 1 -- 172.21.15.19:0/219360027 learned_addr learned my addr 172.21.15.19:0/219360027 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.509+0000 7f6c77fff640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b4436fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.509+0000 7f6c7d773640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b44375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.512+0000 7f6c7cf72640 1 -- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 msgr2=0x55b5b4436fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.512+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b4436fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.512+0000 7f6c7cf72640 1 -- 172.21.15.19:0/219360027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 msgr2=0x55b5b44375d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.512+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b44375d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.512+0000 7f6c7cf72640 1 -- 172.21.15.19:0/219360027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b41063c0 con 0x55b5b442f5d0 2024-09-12T03:24:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.516+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c700bd4d0 tx=0x7f6c700bd660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b7466992027a9b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.516+0000 7f6c57fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:31.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.516+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700d6060 con 0x55b5b442f5d0 2024-09-12T03:24:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.516+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c680041c0 con 0x55b5b442f5d0 2024-09-12T03:24:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.516+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c700c61b0 con 0x55b5b442f5d0 2024-09-12T03:24:31.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.517+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c700bded0 con 0x55b5b442f5d0 2024-09-12T03:24:31.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.520+0000 7f6c815af740 10 client.16425 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:31.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.520+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b5b4448c00 con 0x55b5b442f5d0 2024-09-12T03:24:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.521+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c680041c0 con 0x55b5b442f5d0 2024-09-12T03:24:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.521+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f6c70004b40 con 0x55b5b442f5d0 2024-09-12T03:24:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.521+0000 7f6c815af740 10 client.16425 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.521+0000 7f6c815af740 10 client.16425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:24:31.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.521+0000 7f6c815af740 20 client.16425 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.522+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b444a250 0x55b5b444c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.522+0000 7f6c815af740 4 client.16425 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.522+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b43a2860 con 0x55b5b444a250 2024-09-12T03:24:31.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.523+0000 7f6c77fff640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b444a250 0x55b5b444c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.526+0000 7f6c77fff640 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b444a250 0x55b5b444c6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5a8c9cee21310f5 server_cookie=282b3ea1bd903de in_seq=0 out_seq=0 2024-09-12T03:24:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.528+0000 7f6c57fff640 10 client.16425 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:24:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.528+0000 7f6c57fff640 1 -- 172.21.15.19:0/219360027 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5b43a2860 con 0x55b5b444a250 2024-09-12T03:24:31.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.528+0000 7f6c57fff640 10 client.16425 handle_command_reply: tid=0 2024-09-12T03:24:31.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.529+0000 7f6c815af740 1 client.16425 shutdown 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.534+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 msgr2=0x55b5b4439d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.534+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c700bd4d0 tx=0x7f6c700bd660 comp rx=0 tx=0).stop 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 shutdown_connections 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b444a250 0x55b5b444c6b0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b443a370 0x55b5b44375d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b4436fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 --2- 172.21.15.19:0/219360027 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442f5d0 0x55b5b4439d30 secure :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f6c700bd4d0 tx=0x7f6c700bd660 comp rx=0 tx=0).stop 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 >> 172.21.15.19:0/219360027 conn(0x55b5b43f8310 msgr2=0x55b5b4431930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.535+0000 7f6c57fff640 0 client.16425 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:24:31.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.539+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 shutdown_connections 2024-09-12T03:24:31.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.540+0000 7f6c815af740 1 -- 172.21.15.19:0/219360027 wait complete. 2024-09-12T03:24:31.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.542+0000 7f6c815af740 20 client.16425 trim_cache size 0 max 16384 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.585+0000 7f6c815af740 1 Processor -- start 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.587+0000 7f6c815af740 1 -- start start 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 0x55b5b4446440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c7cf72640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56816/0 (socket says 172.21.15.19:56816) 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c7cf72640 1 -- 172.21.15.19:0/1834118949 learned_addr learned my addr 172.21.15.19:0/1834118949 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:31.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 0x55b5b4451f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5800e520 con 0x55b5b442bf20 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58010120 con 0x55b5b4430b40 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5800ebf0 con 0x55b5b443a370 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 0x55b5b4451f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.588+0000 7f6c77fff640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 0x55b5b4446440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 -- 172.21.15.19:0/1834118949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 msgr2=0x55b5b4446440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 0x55b5b4446440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 -- 172.21.15.19:0/1834118949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 msgr2=0x55b5b4436fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7d773640 1 -- 172.21.15.19:0/1834118949 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b4435530 con 0x55b5b442bf20 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.589+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.590+0000 7f6c77fff640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 0x55b5b4446440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:24:31.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.590+0000 7f6c7d773640 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 0x55b5b4451f80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55b5b4430070 tx=0x7f6c6000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634efc7a20091760 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.591+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/1834118949 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000fce0 con 0x55b5b442bf20 2024-09-12T03:24:31.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.591+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/1834118949 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6000fe80 con 0x55b5b442bf20 2024-09-12T03:24:31.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.591+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/1834118949 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60004b20 con 0x55b5b442bf20 2024-09-12T03:24:31.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.593+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 msgr2=0x55b5b4451f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.593+0000 7f6c815af740 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 0x55b5b4451f80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55b5b4430070 tx=0x7f6c6000f360 comp rx=0 tx=0).stop 2024-09-12T03:24:31.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.595+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 shutdown_connections 2024-09-12T03:24:31.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.595+0000 7f6c815af740 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b442bf20 0x55b5b4451f80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.595+0000 7f6c815af740 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b443a370 0x55b5b4446440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.595+0000 7f6c815af740 1 --2- 172.21.15.19:0/1834118949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b4430b40 0x55b5b4436fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.595+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 >> 172.21.15.19:0/1834118949 conn(0x55b5b43f8310 msgr2=0x55b5b44368a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.597+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 shutdown_connections 2024-09-12T03:24:31.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.597+0000 7f6c815af740 1 -- 172.21.15.19:0/1834118949 wait complete. 2024-09-12T03:24:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.597+0000 7f6c815af740 1 -- start start 2024-09-12T03:24:31.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.598+0000 7f6c815af740 1 Processor -- start 2024-09-12T03:24:31.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.599+0000 7f6c815af740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b442bf20 0x55b5b443e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.599+0000 7f6c815af740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b443ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.599+0000 7f6c815af740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c7d773640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c7d773640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40728/0 (socket says 172.21.15.19:40728) 2024-09-12T03:24:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c7d773640 1 -- 172.21.15.19:0/4014782500 learned_addr learned my addr 172.21.15.19:0/4014782500 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5800f990 con 0x55b5b443a370 2024-09-12T03:24:31.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5800f2c0 con 0x55b5b442bf20 2024-09-12T03:24:31.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.600+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c5800c500 con 0x55b5b4430b40 2024-09-12T03:24:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.602+0000 7f6c77fff640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b443ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.602+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b442bf20 0x55b5b443e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.606+0000 7f6c77fff640 1 -- 172.21.15.19:0/4014782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b442bf20 msgr2=0x55b5b443e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.606+0000 7f6c77fff640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b442bf20 0x55b5b443e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.606+0000 7f6c77fff640 1 -- 172.21.15.19:0/4014782500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 msgr2=0x55b5b443ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.606+0000 7f6c815af740 10 client.0 _ll_register_callbacks cb 0x7f6c70009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.607+0000 7f6c77fff640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.607+0000 7f6c77fff640 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b44b40b0 con 0x55b5b4430b40 2024-09-12T03:24:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.607+0000 7f6c7d773640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:31.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.607+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b5b439cd70 con 0x55b5b4430b40 2024-09-12T03:24:31.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.607+0000 7f6c77fff640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b443ea00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e600 tx=0x7f6c6800ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=816c341547f01d5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:31.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.609+0000 7f6c57fff640 10 client.25933 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.613+0000 7f6c57fff640 1 -- 172.21.15.19:0/4014782500 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68037030 con 0x55b5b4430b40 2024-09-12T03:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.613+0000 7f6c57fff640 1 -- 172.21.15.19:0/4014782500 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c68009040 con 0x55b5b4430b40 2024-09-12T03:24:31.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.613+0000 7f6c57fff640 1 -- 172.21.15.19:0/4014782500 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6c680053e0 con 0x55b5b4430b40 2024-09-12T03:24:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.613+0000 7f6c815af740 10 client.25933 fetch_fsmap learned FSMap version 32 2024-09-12T03:24:31.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.614+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b5b44b5180 con 0x55b5b4430b40 2024-09-12T03:24:31.615 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c57fff640 1 -- 172.21.15.19:0/4014782500 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 32) ==== 8394+0+0 (secure 0 0 0) 0x7f6c6801c8b0 con 0x55b5b4430b40 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 10 client.25933 fetch_fsmap finished waiting for FSMap version 32 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 10 client.25933 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 20 client.25933 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b44b57d0 0x55b5b44b7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 4 client.25933 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5b4319170 con 0x55b5b44b57d0 2024-09-12T03:24:31.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.615+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b44b57d0 0x55b5b44b7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:31.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.618+0000 7f6c7cf72640 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b44b57d0 0x55b5b44b7c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44f57247886eec80 server_cookie=3330f740dace41 in_seq=0 out_seq=0 2024-09-12T03:24:31.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.619+0000 7f6c57fff640 10 client.25933 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:24:31.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.619+0000 7f6c57fff640 1 -- 172.21.15.19:0/4014782500 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5b4319170 con 0x55b5b44b57d0 2024-09-12T03:24:31.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.619+0000 7f6c57fff640 10 client.25933 handle_command_reply: tid=0 2024-09-12T03:24:31.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.619+0000 7f6c815af740 1 client.25933 shutdown 2024-09-12T03:24:31.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.624+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 msgr2=0x55b5b443ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.624+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b443ea00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e600 tx=0x7f6c6800ebc0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 shutdown_connections 2024-09-12T03:24:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55b5b44b57d0 0x55b5b44b7c50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b5b443a370 0x55b5b443ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b5b4430b40 0x55b5b443ea00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 --2- 172.21.15.19:0/4014782500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b5b442bf20 0x55b5b443e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 >> 172.21.15.19:0/4014782500 conn(0x55b5b43f8310 msgr2=0x55b5b442a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.628+0000 7f6c57fff640 0 client.25933 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:24:31.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.629+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 shutdown_connections 2024-09-12T03:24:31.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.631+0000 7f6c815af740 1 -- 172.21.15.19:0/4014782500 wait complete. 2024-09-12T03:24:31.633 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:24:31.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.632+0000 7f6c815af740 20 client.25933 trim_cache size 0 max 16384 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.634+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 msgr2=0x7f6c640a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.634+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 0x7f6c640a5250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6c7811d940 tx=0x7f6c68005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.634+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 msgr2=0x7f6c781a7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.634+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009d00 tx=0x7f6c70031a50 comp rx=0 tx=0).stop 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 shutdown_connections 2024-09-12T03:24:31.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c7811b140 0x7f6c781abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c78109830 0x7f6c781a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c640a2d90 0x7f6c640a5250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 --2- 172.21.15.19:0/3312197109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c78109450 0x7f6c781a7330 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.635+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 >> 172.21.15.19:0/3312197109 conn(0x7f6c7806e140 msgr2=0x7f6c7811be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.636+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 shutdown_connections 2024-09-12T03:24:31.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:31.636+0000 7f6c7f1d6640 1 -- 172.21.15.19:0/3312197109 wait complete. 2024-09-12T03:24:32.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:32 smithi121 ceph-mon[34963]: pgmap v502: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-12T03:24:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:32 smithi019 ceph-mon[30953]: pgmap v502: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-12T03:24:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:32 smithi135 ceph-mon[35474]: pgmap v502: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.6 MiB/s wr, 235 op/s 2024-09-12T03:24:33.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:24:34.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:34 smithi121 ceph-mon[34963]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2024-09-12T03:24:34.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:34 smithi019 ceph-mon[30953]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2024-09-12T03:24:35.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:34 smithi135 ceph-mon[35474]: pgmap v503: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.1 MiB/s wr, 242 op/s 2024-09-12T03:24:35.703 INFO:tasks.workunit.client.0.smithi019.stdout: 424 2099548 1818 1442309 1974 405031 1619 2024-09-12T03:24:35.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:35.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:35.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:35 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:24:35.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:35.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:35.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:35 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:24:36.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:35 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:24:37.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:37 smithi019 ceph-mon[30953]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-09-12T03:24:37.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:37 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:37 smithi135 ceph-mon[35474]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-09-12T03:24:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:37 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:37 smithi121 ceph-mon[34963]: pgmap v504: 97 pgs: 97 active+clean; 1.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 291 op/s 2024-09-12T03:24:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:37 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:38.069 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 -- 172.21.15.19:0/2698177970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 msgr2=0x7f4d3c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 --2- 172.21.15.19:0/2698177970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c103110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009890 tx=0x7f4d3802a360 comp rx=0 tx=0).stop 2024-09-12T03:24:38.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 -- 172.21.15.19:0/2698177970 shutdown_connections 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 --2- 172.21.15.19:0/2698177970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 0x7f4d3c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 --2- 172.21.15.19:0/2698177970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 0x7f4d3c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 --2- 172.21.15.19:0/2698177970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c103110 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 -- 172.21.15.19:0/2698177970 >> 172.21.15.19:0/2698177970 conn(0x7f4d3c0fc7d0 msgr2=0x7f4d3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 -- 172.21.15.19:0/2698177970 shutdown_connections 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.801+0000 7f4d43679640 1 -- 172.21.15.19:0/2698177970 wait complete. 2024-09-12T03:24:38.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.802+0000 7f4d43679640 1 Processor -- start 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 -- start start 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 0x7f4d3c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 0x7f4d3c19a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1120a0 con 0x7f4d3c1082a0 2024-09-12T03:24:38.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c111f20 con 0x7f4d3c102d30 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d43679640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c112220 con 0x7f4d3c1036e0 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.803+0000 7f4d41415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56842/0 (socket says 172.21.15.19:56842) 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 -- 172.21.15.19:0/2711917929 learned_addr learned my addr 172.21.15.19:0/2711917929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d40c14640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 0x7f4d3c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41c16640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 0x7f4d3c19a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 -- 172.21.15.19:0/2711917929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 msgr2=0x7f4d3c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 0x7f4d3c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 -- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 msgr2=0x7f4d3c19a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 0x7f4d3c19a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.804+0000 7f4d41415640 1 -- 172.21.15.19:0/2711917929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3c19af10 con 0x7f4d3c102d30 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.805+0000 7f4d41415640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000c00 tx=0x7f4d38033d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.805+0000 7f4d327fc640 1 -- 172.21.15.19:0/2711917929 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3803f020 con 0x7f4d3c102d30 2024-09-12T03:24:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.805+0000 7f4d327fc640 1 -- 172.21.15.19:0/2711917929 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d38032770 con 0x7f4d3c102d30 2024-09-12T03:24:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.805+0000 7f4d327fc640 1 -- 172.21.15.19:0/2711917929 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d38037040 con 0x7f4d3c102d30 2024-09-12T03:24:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.806+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3c19b1a0 con 0x7f4d3c102d30 2024-09-12T03:24:38.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.806+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3c1a7380 con 0x7f4d3c102d30 2024-09-12T03:24:38.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.810+0000 7f4d45a52740 1 Processor -- start 2024-09-12T03:24:38.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.810+0000 7f4d327fc640 1 -- 172.21.15.19:0/2711917929 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d38042020 con 0x7f4d3c102d30 2024-09-12T03:24:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 -- start start 2024-09-12T03:24:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329efa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329fa370 0x5635329fc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635329552f0 con 0x5635329ef5d0 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5635328fccf0 con 0x5635329f0b40 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d45a52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635329512b0 con 0x5635329fa370 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d41c16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d41c16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56848/0 (socket says 172.21.15.19:56848) 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d41c16640 1 -- 172.21.15.19:0/3083018987 learned_addr learned my addr 172.21.15.19:0/3083018987 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d41415640 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.811+0000 7f4d40c14640 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329fa370 0x5635329fc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d41c16640 1 -- 172.21.15.19:0/3083018987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329fa370 msgr2=0x5635329fc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329fa370 0x5635329fc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d41c16640 1 -- 172.21.15.19:0/3083018987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 msgr2=0x5635329efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d327fc640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 0x7f4d280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.812+0000 7f4d41c16640 1 -- 172.21.15.19:0/3083018987 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635328e13a0 con 0x5635329f0b40 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.813+0000 7f4d327fc640 1 -- 172.21.15.19:0/2711917929 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4d380eefe0 con 0x7f4d3c102d30 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.813+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d340099f0 tx=0x7f4d340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3c6b53d2631603a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.813+0000 7f4d40c14640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 0x7f4d280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.813+0000 7f4d277fe640 1 -- 172.21.15.19:0/3083018987 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3401d070 con 0x5635329f0b40 2024-09-12T03:24:38.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.813+0000 7f4d277fe640 1 -- 172.21.15.19:0/3083018987 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d34003e40 con 0x5635329f0b40 2024-09-12T03:24:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.814+0000 7f4d277fe640 1 -- 172.21.15.19:0/3083018987 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3400dba0 con 0x5635329f0b40 2024-09-12T03:24:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.814+0000 7f4d45a52740 1 -- 172.21.15.19:0/3083018987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 msgr2=0x5635329ef090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.814+0000 7f4d40c14640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 0x7f4d280a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5635329f0070 tx=0x7f4d2c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4d340099f0 tx=0x7f4d340095c0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 -- 172.21.15.19:0/3083018987 shutdown_connections 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329fa370 0x5635329fc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3083018987 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329f0b40 0x5635329ef090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 -- 172.21.15.19:0/3083018987 >> 172.21.15.19:0/3083018987 conn(0x5635329b8310 msgr2=0x5635329ebc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 -- 172.21.15.19:0/3083018987 shutdown_connections 2024-09-12T03:24:38.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.815+0000 7f4d45a52740 1 -- 172.21.15.19:0/3083018987 wait complete. 2024-09-12T03:24:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.816+0000 7f4d45a52740 1 -- start start 2024-09-12T03:24:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.816+0000 7f4d45a52740 1 Processor -- start 2024-09-12T03:24:38.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.817+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563532a058d0 con 0x5635329ef5d0 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563532910060 con 0x5635329fa370 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563532a05a50 con 0x5635329f0b40 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40770/0 (socket says 172.21.15.19:40770) 2024-09-12T03:24:38.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 -- 172.21.15.19:0/1061564128 learned_addr learned my addr 172.21.15.19:0/1061564128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:38.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d45a52740 10 client.0 _ll_register_callbacks cb 0x5635326638c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:38.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41c16640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 -- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 msgr2=0x5635329f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 -- 172.21.15.19:0/1061564128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 msgr2=0x5635329f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.818+0000 7f4d41415640 1 -- 172.21.15.19:0/1061564128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635326c63c0 con 0x5635329ef5d0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d41415640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ecaf0 tx=0x7f4d38009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe36dc9f8ec6c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b7af0 con 0x5635329ef5d0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4d380bbab0 con 0x5635329ef5d0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563532962860 con 0x5635329ef5d0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d380bbf50 con 0x5635329ef5d0 2024-09-12T03:24:38.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d45a52740 10 client.16443 fetch_fsmap learned FSMap version 34 2024-09-12T03:24:38.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.819+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563532a11300 con 0x5635329ef5d0 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d380bb850 con 0x5635329ef5d0 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 34) ==== 8402+0+0 (secure 0 0 0) 0x7f4d3803a030 con 0x5635329ef5d0 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 10 client.16443 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 10 client.16443 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 20 client.16443 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:38.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a126f0 0x563532a14b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 4 client.16443 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.820+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563532962860 con 0x563532a126f0 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.821+0000 7f4d40c14640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a126f0 0x563532a14b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.822+0000 7f4d40c14640 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a126f0 0x563532a14b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d39cf4cffca89dad server_cookie=48495f827116b3f8 in_seq=0 out_seq=0 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.822+0000 7f4d24ff9640 10 client.16443 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:24:38.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.823+0000 7f4d24ff9640 1 -- 172.21.15.19:0/1061564128 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563532962860 con 0x563532a126f0 2024-09-12T03:24:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.823+0000 7f4d24ff9640 10 client.16443 handle_command_reply: tid=0 2024-09-12T03:24:38.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 client.16443 shutdown 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 msgr2=0x5635329f9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ecaf0 tx=0x7f4d38009550 comp rx=0 tx=0).stop 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 shutdown_connections 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a126f0 0x563532a14b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 --2- 172.21.15.19:0/1061564128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x5635329f9c70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 >> 172.21.15.19:0/1061564128 conn(0x5635329b8310 msgr2=0x5635329f1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.824+0000 7f4d24ff9640 0 client.16443 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.825+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 shutdown_connections 2024-09-12T03:24:38.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.825+0000 7f4d45a52740 1 -- 172.21.15.19:0/1061564128 wait complete. 2024-09-12T03:24:38.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.825+0000 7f4d45a52740 20 client.16443 trim_cache size 0 max 16384 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 Processor -- start 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 -- start start 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 0x563532a0ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x563532a14700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 0x563532a1a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.844+0000 7f4d45a52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ab4d0 con 0x5635329fa370 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d45a52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ad100 con 0x5635329ef5d0 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d45a52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1abba0 con 0x5635329f0b40 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41c16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 0x563532a1a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 0x563532a0ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41c16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 0x563532a1a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40772/0 (socket says 172.21.15.19:40772) 2024-09-12T03:24:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41c16640 1 -- 172.21.15.19:0/2984261367 learned_addr learned my addr 172.21.15.19:0/2984261367 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:38.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d40c14640 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x563532a14700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 -- 172.21.15.19:0/2984261367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 msgr2=0x563532a14700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x563532a14700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 -- 172.21.15.19:0/2984261367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 msgr2=0x563532a1a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 0x563532a1a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 -- 172.21.15.19:0/2984261367 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635329f5470 con 0x5635329ef5d0 2024-09-12T03:24:38.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d41415640 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 0x563532a0ebf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ece60 tx=0x7f4d380bb270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db32fbe173a4a504 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.845+0000 7f4d26ffd640 1 -- 172.21.15.19:0/2984261367 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b7af0 con 0x5635329ef5d0 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d26ffd640 1 -- 172.21.15.19:0/2984261367 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d380bb850 con 0x5635329ef5d0 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d26ffd640 1 -- 172.21.15.19:0/2984261367 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d380bbab0 con 0x5635329ef5d0 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 -- 172.21.15.19:0/2984261367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 msgr2=0x563532a0ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 0x563532a0ebf0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4d380ece60 tx=0x7f4d380bb270 comp rx=0 tx=0).stop 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 -- 172.21.15.19:0/2984261367 shutdown_connections 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329fa370 0x563532a1a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x563532a14700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 --2- 172.21.15.19:0/2984261367 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329ef5d0 0x563532a0ebf0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.846+0000 7f4d45a52740 1 -- 172.21.15.19:0/2984261367 >> 172.21.15.19:0/2984261367 conn(0x5635329b8310 msgr2=0x5635329f57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.847+0000 7f4d45a52740 1 -- 172.21.15.19:0/2984261367 shutdown_connections 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.847+0000 7f4d45a52740 1 -- 172.21.15.19:0/2984261367 wait complete. 2024-09-12T03:24:38.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.847+0000 7f4d45a52740 1 -- start start 2024-09-12T03:24:38.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.847+0000 7f4d45a52740 1 Processor -- start 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x563532a0b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ac970 con 0x5635329ef5d0 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1ac2a0 con 0x5635329fa370 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4d3c1a9500 con 0x5635329f0b40 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x563532a0b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41415640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x563532a0b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40788/0 (socket says 172.21.15.19:40788) 2024-09-12T03:24:38.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41415640 1 -- 172.21.15.19:0/3514443208 learned_addr learned my addr 172.21.15.19:0/3514443208 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d45a52740 10 client.0 _ll_register_callbacks cb 0x7f4d380f25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d40c14640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 -- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 msgr2=0x5635329f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 -- 172.21.15.19:0/3514443208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 msgr2=0x563532a0b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x563532a0b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.848+0000 7f4d41c16640 1 -- 172.21.15.19:0/3514443208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635329f47e0 con 0x5635329fa370 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d41c16640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f40d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d34010010 tx=0x7f4d34009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=651a6e65c02389ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d24ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3402a070 con 0x5635329fa370 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4d34005800 con 0x5635329fa370 2024-09-12T03:24:38.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d380bb850 con 0x5635329fa370 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4d34036050 con 0x5635329fa370 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d45a52740 10 client.25946 fetch_fsmap learned FSMap version 34 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563532a7d3a0 con 0x5635329fa370 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.849+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3401e040 con 0x5635329fa370 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 34) ==== 8402+0+0 (secure 0 0 0) 0x7f4d34023050 con 0x5635329fa370 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 10 client.25946 fetch_fsmap finished waiting for FSMap version 34 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 10 client.25946 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 20 client.25946 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a7d7e0 0x563532a7fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 4 client.25946 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4d380bb850 con 0x563532a7d7e0 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.850+0000 7f4d41415640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a7d7e0 0x563532a7fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.851+0000 7f4d41415640 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a7d7e0 0x563532a7fc00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=607004c332d2d488 server_cookie=40b53f75f3dec16f in_seq=0 out_seq=0 2024-09-12T03:24:38.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.851+0000 7f4d24ff9640 10 client.25946 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d24ff9640 1 -- 172.21.15.19:0/3514443208 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4d380bb850 con 0x563532a7d7e0 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d24ff9640 10 client.25946 handle_command_reply: tid=0 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 client.25946 shutdown 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 msgr2=0x5635329f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f40d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4d34010010 tx=0x7f4d34009940 comp rx=0 tx=0).stop 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 shutdown_connections 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563532a7d7e0 0x563532a7fc00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d24ff9640 0 client.25946 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635329fa370 0x5635329f40d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635329f0b40 0x5635329f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 --2- 172.21.15.19:0/3514443208 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635329ef5d0 0x563532a0b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 >> 172.21.15.19:0/3514443208 conn(0x5635329b8310 msgr2=0x5635329f1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 shutdown_connections 2024-09-12T03:24:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 1 -- 172.21.15.19:0/3514443208 wait complete. 2024-09-12T03:24:38.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.852+0000 7f4d45a52740 20 client.25946 trim_cache size 0 max 16384 2024-09-12T03:24:38.857 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 msgr2=0x7f4d280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 0x7f4d280a50c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x5635329f0070 tx=0x7f4d2c005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 msgr2=0x7f4d3c1a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4d38000c00 tx=0x7f4d38033d10 comp rx=0 tx=0).stop 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 shutdown_connections 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f4d280a2c00 0x7f4d280a50c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4d3c1082a0 0x7f4d3c19a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4d3c1036e0 0x7f4d3c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 --2- 172.21.15.19:0/2711917929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4d3c102d30 0x7f4d3c1a05d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 >> 172.21.15.19:0/2711917929 conn(0x7f4d3c0fc7d0 msgr2=0x7f4d3c1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.853+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 shutdown_connections 2024-09-12T03:24:38.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:38.854+0000 7f4d43679640 1 -- 172.21.15.19:0/2711917929 wait complete. 2024-09-12T03:24:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:39 smithi135 ceph-mon[35474]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-09-12T03:24:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:39 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:39.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:39 smithi121 ceph-mon[34963]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-09-12T03:24:39.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:39 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:39 smithi019 ceph-mon[30953]: pgmap v505: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 12 MiB/s wr, 624 op/s 2024-09-12T03:24:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:39 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:40.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:24:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:41 smithi135 ceph-mon[35474]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 782 op/s 2024-09-12T03:24:41.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:41 smithi121 ceph-mon[34963]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 782 op/s 2024-09-12T03:24:41.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:41 smithi019 ceph-mon[30953]: pgmap v506: 97 pgs: 97 active+clean; 1.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 782 op/s 2024-09-12T03:24:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:42 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:42.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:42 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:42.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:42 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:43 smithi135 ceph-mon[35474]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 851 op/s 2024-09-12T03:24:43.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:43 smithi121 ceph-mon[34963]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 851 op/s 2024-09-12T03:24:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:43 smithi019 ceph-mon[30953]: pgmap v507: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 851 op/s 2024-09-12T03:24:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:44 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:44.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:44 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:44.660 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:44.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:44 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:24:45.601 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:45 smithi019 ceph-mon[30953]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 887 op/s 2024-09-12T03:24:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:45 smithi121 ceph-mon[34963]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 887 op/s 2024-09-12T03:24:45.713 INFO:tasks.workunit.client.0.smithi019.stdout: 451 1884937 1491 1297039 1533 378422 1395 2024-09-12T03:24:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.781+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2091411806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102ef0 msgr2=0x7f1a4810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.781+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2091411806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102ef0 0x7f1a4810da40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009ca0 tx=0x7f1a4002a980 comp rx=0 tx=0).stop 2024-09-12T03:24:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.786+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2091411806 shutdown_connections 2024-09-12T03:24:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.786+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2091411806 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102ef0 0x7f1a4810da40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.786+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2091411806 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102570 0x7f1a481029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.786+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2091411806 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 0x7f1a48101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.786+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2091411806 >> 172.21.15.19:0/2091411806 conn(0x7f1a480fc7d0 msgr2=0x7f1a480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.791+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2091411806 shutdown_connections 2024-09-12T03:24:45.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.791+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2091411806 wait complete. 2024-09-12T03:24:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:45 smithi135 ceph-mon[35474]: pgmap v508: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 18 MiB/s wr, 887 op/s 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.792+0000 7f1a4fbe2640 1 Processor -- start 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 -- start start 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 0x7f1a481a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102570 0x7f1a481a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48110fc0 con 0x7f1a48102570 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48110e40 con 0x7f1a48102ef0 2024-09-12T03:24:45.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4fbe2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a48111140 con 0x7f1a48100e20 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.793+0000 7f1a4e17f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47338/0 (socket says 172.21.15.19:47338) 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2107743160 learned_addr learned my addr 172.21.15.19:0/2107743160 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 0x7f1a481a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2107743160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 msgr2=0x7f1a481a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 0x7f1a481a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102570 msgr2=0x7f1a481a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102570 0x7f1a481a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.794+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2107743160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a4819f460 con 0x7f1a48102ef0 2024-09-12T03:24:45.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.795+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009dc0 tx=0x7f1a400308b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.796+0000 7f1a3effd640 1 -- 172.21.15.19:0/2107743160 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40034050 con 0x7f1a48102ef0 2024-09-12T03:24:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.796+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a4819f6f0 con 0x7f1a48102ef0 2024-09-12T03:24:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.796+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a4806bc10 con 0x7f1a48102ef0 2024-09-12T03:24:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.800+0000 7f1a51fbb740 1 Processor -- start 2024-09-12T03:24:45.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.801+0000 7f1a51fbb740 1 -- start start 2024-09-12T03:24:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.801+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.808+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ea5d0 0x564c5a0eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.808+0000 7f1a4d17d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.808+0000 7f1a4d17d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:47346/0 (socket says 172.21.15.19:47346) 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2893833167 learned_addr learned my addr 172.21.15.19:0/2893833167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0f5370 0x564c5a0f7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564c5a0502f0 con 0x564c5a0f5370 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c59ff7cf0 con 0x564c5a0ebb40 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c5a04c2b0 con 0x564c5a0ea5d0 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.809+0000 7f1a3effd640 1 -- 172.21.15.19:0/2107743160 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a40030bb0 con 0x7f1a48102ef0 2024-09-12T03:24:45.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2893833167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ea5d0 msgr2=0x564c5a0eaa10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ea5d0 0x564c5a0eaa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2893833167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0f5370 msgr2=0x564c5a0f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0f5370 0x564c5a0f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a3effd640 1 -- 172.21.15.19:0/2107743160 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a40039420 con 0x7f1a48102ef0 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a3effd640 1 -- 172.21.15.19:0/2107743160 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a40037020 con 0x7f1a48102ef0 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.810+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2893833167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c59fdc3a0 con 0x564c5a0ebb40 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.811+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005fd0 tx=0x7f1a3800f300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d9795d4df95ea6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.811+0000 7f1a33fff640 1 -- 172.21.15.19:0/2893833167 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38015030 con 0x564c5a0ebb40 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.811+0000 7f1a33fff640 1 -- 172.21.15.19:0/2893833167 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a38005500 con 0x564c5a0ebb40 2024-09-12T03:24:45.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.811+0000 7f1a33fff640 1 -- 172.21.15.19:0/2893833167 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38004de0 con 0x564c5a0ebb40 2024-09-12T03:24:45.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.812+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 msgr2=0x564c5a0ea090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.812+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1a38005fd0 tx=0x7f1a3800f300 comp rx=0 tx=0).stop 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.816+0000 7f1a3effd640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 0x7f1a340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.817+0000 7f1a3effd640 1 -- 172.21.15.19:0/2107743160 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1a400f0ff0 con 0x7f1a48102ef0 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.818+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 shutdown_connections 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0f5370 0x564c5a0f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ea5d0 0x564c5a0eaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2893833167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ebb40 0x564c5a0ea090 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 >> 172.21.15.19:0/2893833167 conn(0x564c5a0b3310 msgr2=0x564c5a0e6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 shutdown_connections 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.819+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2893833167 wait complete. 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.820+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 0x7f1a340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.821+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 0x7f1a340a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b280 tx=0x7f1a380046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.821+0000 7f1a51fbb740 1 -- start start 2024-09-12T03:24:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.822+0000 7f1a51fbb740 1 Processor -- start 2024-09-12T03:24:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.822+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ebb40 0x564c5a0f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a4d97e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a4d97e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41752/0 (socket says 172.21.15.19:41752) 2024-09-12T03:24:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a4d97e640 1 -- 172.21.15.19:0/2187207276 learned_addr learned my addr 172.21.15.19:0/2187207276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 0x564c5a0f2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ebb40 0x564c5a0f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564c5a0f8070 con 0x564c5a0ea5d0 2024-09-12T03:24:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564c5a00b060 con 0x564c5a0f5370 2024-09-12T03:24:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564c5a0f81f0 con 0x564c5a0ebb40 2024-09-12T03:24:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.823+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 0x564c5a0f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.824+0000 7f1a51fbb740 10 client.0 _ll_register_callbacks cb 0x564c59d5e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2187207276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ebb40 msgr2=0x564c5a0f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ebb40 0x564c5a0f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2187207276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 msgr2=0x564c5a0f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c59dc13c0 con 0x564c5a0f5370 2024-09-12T03:24:45.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.827+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:24:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.831+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 0x564c5a0f2570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a400b9ce0 tx=0x7f1a400bd2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef765d56c9fe47ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.831+0000 7f1a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:24:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.839+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a400d5070 con 0x564c5a0f5370 2024-09-12T03:24:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.839+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a38005500 con 0x564c5a0f5370 2024-09-12T03:24:45.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.839+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1a400bdae0 con 0x564c5a0f5370 2024-09-12T03:24:45.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.840+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a400c1c30 con 0x564c5a0f5370 2024-09-12T03:24:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.841+0000 7f1a51fbb740 10 client.25958 fetch_fsmap learned FSMap version 36 2024-09-12T03:24:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.841+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564c5a0f8070 con 0x564c5a0f5370 2024-09-12T03:24:45.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.841+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a38005500 con 0x564c5a0f5370 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1a40041070 con 0x564c5a0f5370 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 10 client.25958 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 10 client.25958 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 20 client.25958 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1050a0 0x564c5a107500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 4 client.25958 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c5a05d860 con 0x564c5a1050a0 2024-09-12T03:24:45.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.843+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1050a0 0x564c5a107500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.851+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1050a0 0x564c5a107500 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f25f427ecea73c2 server_cookie=c84cad2622c7dcee in_seq=0 out_seq=0 2024-09-12T03:24:45.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.851+0000 7f1a317fa640 10 client.25958 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:24:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.853+0000 7f1a317fa640 1 -- 172.21.15.19:0/2187207276 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c5a05d860 con 0x564c5a1050a0 2024-09-12T03:24:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.853+0000 7f1a317fa640 10 client.25958 handle_command_reply: tid=0 2024-09-12T03:24:45.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.853+0000 7f1a51fbb740 1 client.25958 shutdown 2024-09-12T03:24:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.855+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 msgr2=0x564c5a0f2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.855+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 0x564c5a0f2570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a400b9ce0 tx=0x7f1a400bd2a0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 shutdown_connections 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1050a0 0x564c5a107500 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0f5370 0x564c5a0f2570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0ebb40 0x564c5a0f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2187207276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ea5d0 0x564c5a0f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 >> 172.21.15.19:0/2187207276 conn(0x564c5a0b3310 msgr2=0x564c5a0ec930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.858+0000 7f1a317fa640 0 client.25958 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:24:45.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.859+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 shutdown_connections 2024-09-12T03:24:45.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.859+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2187207276 wait complete. 2024-09-12T03:24:45.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.860+0000 7f1a51fbb740 20 client.25958 trim_cache size 0 max 16384 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.897+0000 7f1a51fbb740 1 Processor -- start 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.897+0000 7f1a51fbb740 1 -- start start 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.897+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.897+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c00e3e0 con 0x564c5a0ebb40 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c0100a0 con 0x564c5a0ea5d0 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c00eab0 con 0x564c5a0f5370 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a4e17f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a4e17f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:51212/0 (socket says 172.21.15.19:51212) 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.898+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2528967849 learned_addr learned my addr 172.21.15.19:0/2528967849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.899+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.899+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2528967849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 msgr2=0x564c5a1013a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.899+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a1013a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.899+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2528967849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 msgr2=0x564c5a0f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.899+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.900+0000 7f1a4e17f640 1 -- 172.21.15.19:0/2528967849 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c5a107b80 con 0x564c5a0f5370 2024-09-12T03:24:45.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.900+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a400dc410 tx=0x7f1a400bd310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2936c81ef275c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.902+0000 7f1a337fe640 1 -- 172.21.15.19:0/2528967849 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a400d5070 con 0x564c5a0f5370 2024-09-12T03:24:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.902+0000 7f1a337fe640 1 -- 172.21.15.19:0/2528967849 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a400c64f0 con 0x564c5a0f5370 2024-09-12T03:24:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.902+0000 7f1a337fe640 1 -- 172.21.15.19:0/2528967849 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a400bdae0 con 0x564c5a0f5370 2024-09-12T03:24:45.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.903+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2528967849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 msgr2=0x564c5a10ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.903+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1a400dc410 tx=0x7f1a400bd310 comp rx=0 tx=0).stop 2024-09-12T03:24:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.905+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2528967849 shutdown_connections 2024-09-12T03:24:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.905+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a10ce10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.906+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.906+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2528967849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.906+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2528967849 >> 172.21.15.19:0/2528967849 conn(0x564c5a0b3310 msgr2=0x564c5a0f3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.907+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2528967849 shutdown_connections 2024-09-12T03:24:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.909+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2528967849 wait complete. 2024-09-12T03:24:45.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.909+0000 7f1a51fbb740 1 -- start start 2024-09-12T03:24:45.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.910+0000 7f1a51fbb740 1 Processor -- start 2024-09-12T03:24:45.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a0fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a0f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c00f910 con 0x564c5a0ebb40 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c00f240 con 0x564c5a0ea5d0 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a51fbb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1a2c00c3b0 con 0x564c5a0f5370 2024-09-12T03:24:45.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a4d17d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a4d17d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41768/0 (socket says 172.21.15.19:41768) 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.911+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2309163148 learned_addr learned my addr 172.21.15.19:0/2309163148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4e17f640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a0f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a51fbb740 10 client.0 _ll_register_callbacks cb 0x7f1a40005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a0fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2309163148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 msgr2=0x564c5a0f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a0f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 msgr2=0x564c5a0fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a0fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.912+0000 7f1a4d17d640 1 -- 172.21.15.19:0/2309163148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5a0e7960 con 0x564c5a0ebb40 2024-09-12T03:24:45.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.913+0000 7f1a4d17d640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800f800 tx=0x7f1a380049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2da9ce4d17df1f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:45.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.913+0000 7f1a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.913+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38036060 con 0x564c5a0ebb40 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.914+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1a3801b880 con 0x564c5a0ebb40 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.914+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a400c64f0 con 0x564c5a0ebb40 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.914+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1a38015070 con 0x564c5a0ebb40 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.914+0000 7f1a51fbb740 10 client.16461 fetch_fsmap learned FSMap version 36 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.914+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564c5a16fb60 con 0x564c5a0ebb40 2024-09-12T03:24:45.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.916+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a400c64f0 con 0x564c5a0ebb40 2024-09-12T03:24:45.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.916+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1a38022040 con 0x564c5a0ebb40 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 10 client.16461 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 10 client.16461 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 20 client.16461 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1700f0 0x564c5a172510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 4 client.16461 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:45.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.917+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564c5a057d70 con 0x564c5a1700f0 2024-09-12T03:24:45.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.920+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1700f0 0x564c5a172510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:45.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.923+0000 7f1a4d97e640 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1700f0 0x564c5a172510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c697d56f1a1edb48 server_cookie=59bd471dcc1c4b52 in_seq=0 out_seq=0 2024-09-12T03:24:45.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.923+0000 7f1a317fa640 10 client.16461 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:24:45.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.925+0000 7f1a317fa640 1 -- 172.21.15.19:0/2309163148 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564c5a057d70 con 0x564c5a1700f0 2024-09-12T03:24:45.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.925+0000 7f1a317fa640 10 client.16461 handle_command_reply: tid=0 2024-09-12T03:24:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.926+0000 7f1a51fbb740 1 client.16461 shutdown 2024-09-12T03:24:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.926+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 msgr2=0x564c5a0f8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.926+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800f800 tx=0x7f1a380049c0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 shutdown_connections 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564c5a1700f0 0x564c5a172510 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564c5a0f5370 0x564c5a0f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564c5a0ebb40 0x564c5a0f8ef0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 --2- 172.21.15.19:0/2309163148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564c5a0ea5d0 0x564c5a0fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 >> 172.21.15.19:0/2309163148 conn(0x564c5a0b3310 msgr2=0x564c5a0f4050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a317fa640 0 client.16461 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.927+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 shutdown_connections 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.928+0000 7f1a51fbb740 1 -- 172.21.15.19:0/2309163148 wait complete. 2024-09-12T03:24:45.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.928+0000 7f1a51fbb740 20 client.16461 trim_cache size 0 max 16384 2024-09-12T03:24:45.929 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:24:45.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 msgr2=0x7f1a340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 0x7f1a340a50c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b280 tx=0x7f1a380046c0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 msgr2=0x7f1a4819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:45.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1a40009dc0 tx=0x7f1a400308b0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 shutdown_connections 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1a48102ef0 0x7f1a4819eca0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1a340a2c00 0x7f1a340a50c0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1a48102570 0x7f1a481a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 --2- 172.21.15.19:0/2107743160 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1a48100e20 0x7f1a481a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:45.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.930+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 >> 172.21.15.19:0/2107743160 conn(0x7f1a480fc7d0 msgr2=0x7f1a4806a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:45.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.933+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 shutdown_connections 2024-09-12T03:24:45.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:45.934+0000 7f1a4fbe2640 1 -- 172.21.15.19:0/2107743160 wait complete. 2024-09-12T03:24:47.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:24:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:47 smithi019 ceph-mon[30953]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 896 op/s 2024-09-12T03:24:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:47 smithi121 ceph-mon[34963]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 896 op/s 2024-09-12T03:24:47.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:47 smithi135 ceph-mon[35474]: pgmap v509: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 18 MiB/s wr, 896 op/s 2024-09-12T03:24:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:49 smithi135 ceph-mon[35474]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 858 op/s 2024-09-12T03:24:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:49 smithi121 ceph-mon[34963]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 858 op/s 2024-09-12T03:24:49.668 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:49 smithi019 ceph-mon[30953]: pgmap v510: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 18 MiB/s wr, 858 op/s 2024-09-12T03:24:51.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:51 smithi121 ceph-mon[34963]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-12T03:24:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:51 smithi019 ceph-mon[30953]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-12T03:24:51.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:51 smithi135 ceph-mon[35474]: pgmap v511: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 10 MiB/s wr, 509 op/s 2024-09-12T03:24:52.924 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:24:53.538 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:53 smithi019 ceph-mon[30953]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 471 op/s 2024-09-12T03:24:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:53 smithi121 ceph-mon[34963]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 471 op/s 2024-09-12T03:24:53.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:53 smithi135 ceph-mon[35474]: pgmap v512: 97 pgs: 97 active+clean; 1.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 8.6 MiB/s wr, 471 op/s 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.089+0000 7f2d98341640 1 -- 172.21.15.19:0/3348024913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102570 msgr2=0x7f2d901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.089+0000 7f2d98341640 1 --2- 172.21.15.19:0/3348024913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102570 0x7f2d901029b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009900 tx=0x7f2d8002a4d0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.090+0000 7f2d98341640 1 -- 172.21.15.19:0/3348024913 shutdown_connections 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.090+0000 7f2d98341640 1 --2- 172.21.15.19:0/3348024913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90102ef0 0x7f2d90069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.090+0000 7f2d98341640 1 --2- 172.21.15.19:0/3348024913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102570 0x7f2d901029b0 secure :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2d80009900 tx=0x7f2d8002a4d0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.090+0000 7f2d98341640 1 --2- 172.21.15.19:0/3348024913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d90100e20 0x7f2d90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.090+0000 7f2d98341640 1 -- 172.21.15.19:0/3348024913 >> 172.21.15.19:0/3348024913 conn(0x7f2d900fc7d0 msgr2=0x7f2d900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.091+0000 7f2d98341640 1 -- 172.21.15.19:0/3348024913 shutdown_connections 2024-09-12T03:24:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.092+0000 7f2d98341640 1 -- 172.21.15.19:0/3348024913 wait complete. 2024-09-12T03:24:54.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.093+0000 7f2d98341640 1 Processor -- start 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.094+0000 7f2d98341640 1 -- start start 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90100e20 0x7f2d90110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102ef0 0x7f2d90111170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90114390 con 0x7f2d90102ef0 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90114210 con 0x7f2d90100e20 2024-09-12T03:24:54.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.098+0000 7f2d98341640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90114510 con 0x7f2d901116b0 2024-09-12T03:24:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.099+0000 7f2d968de640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.099+0000 7f2d968de640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:41738/0 (socket says 172.21.15.19:41738) 2024-09-12T03:24:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.099+0000 7f2d968de640 1 -- 172.21.15.19:0/1273785191 learned_addr learned my addr 172.21.15.19:0/1273785191 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:54.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.099+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102ef0 0x7f2d90111170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.100+0000 7f2d960dd640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90100e20 0x7f2d90110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.101+0000 7f2d968de640 1 -- 172.21.15.19:0/1273785191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90100e20 msgr2=0x7f2d90110c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.101+0000 7f2d968de640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90100e20 0x7f2d90110c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.101+0000 7f2d968de640 1 -- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102ef0 msgr2=0x7f2d90111170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.101+0000 7f2d968de640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102ef0 0x7f2d90111170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.101+0000 7f2d968de640 1 -- 172.21.15.19:0/1273785191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d901a2d90 con 0x7f2d901116b0 2024-09-12T03:24:54.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.103+0000 7f2d968de640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b970 tx=0x7f2d8800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.106+0000 7f2d877fe640 1 -- 172.21.15.19:0/1273785191 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88019070 con 0x7f2d901116b0 2024-09-12T03:24:54.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.106+0000 7f2d877fe640 1 -- 172.21.15.19:0/1273785191 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d8800cb40 con 0x7f2d901116b0 2024-09-12T03:24:54.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.106+0000 7f2d877fe640 1 -- 172.21.15.19:0/1273785191 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d880076a0 con 0x7f2d901116b0 2024-09-12T03:24:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.107+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d901a3080 con 0x7f2d901116b0 2024-09-12T03:24:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.107+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d901a34e0 con 0x7f2d901116b0 2024-09-12T03:24:54.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.114+0000 7f2d9a71a740 1 Processor -- start 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.119+0000 7f2d9a71a740 1 -- start start 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621d9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 0x55c0621d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621e4340 0x55c0621e6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c06213f2f0 con 0x55c0621d95a0 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c0620e6cf0 con 0x55c0621dab10 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c06213b2b0 con 0x55c0621e4340 2024-09-12T03:24:54.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d877fe640 1 -- 172.21.15.19:0/1273785191 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d8801c070 con 0x7f2d901116b0 2024-09-12T03:24:54.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 0x55c0621d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 0x55c0621d99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55306/0 (socket says 172.21.15.19:55306) 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d960dd640 1 -- 172.21.15.19:0/1583829489 learned_addr learned my addr 172.21.15.19:0/1583829489 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621d9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.120+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621e4340 0x55c0621e6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 -- 172.21.15.19:0/1583829489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621e4340 msgr2=0x55c0621e6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621e4340 0x55c0621e6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 -- 172.21.15.19:0/1583829489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 msgr2=0x55c0621d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 0x55c0621d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 -- 172.21.15.19:0/1583829489 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0620cb3a0 con 0x55c0621dab10 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.121+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621d9060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d90101f30 tx=0x7f2d80005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25dd49f82ed1f3d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.122+0000 7f2d877fe640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 0x7f2d7c0a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.124+0000 7f2d960dd640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 0x7f2d7c0a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.124+0000 7f2d877fe640 1 -- 172.21.15.19:0/1273785191 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d880ce610 con 0x7f2d901116b0 2024-09-12T03:24:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.125+0000 7f2d960dd640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 0x7f2d7c0a5250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55c0621d6a90 tx=0x7f2d8c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.124+0000 7f2d84dea640 1 -- 172.21.15.19:0/1583829489 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800035f0 con 0x55c0621dab10 2024-09-12T03:24:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.126+0000 7f2d84dea640 1 -- 172.21.15.19:0/1583829489 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d80005680 con 0x55c0621dab10 2024-09-12T03:24:54.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.126+0000 7f2d84dea640 1 -- 172.21.15.19:0/1583829489 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8003fa70 con 0x55c0621dab10 2024-09-12T03:24:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.128+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1583829489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 msgr2=0x55c0621d9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.128+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621d9060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2d90101f30 tx=0x7f2d80005fd0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.129+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1583829489 shutdown_connections 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.129+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621e4340 0x55c0621e6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.129+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621d95a0 0x55c0621d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.129+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1583829489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621d9060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.129+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1583829489 >> 172.21.15.19:0/1583829489 conn(0x55c0621a2310 msgr2=0x55c0621d5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.133+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1583829489 shutdown_connections 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.135+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1583829489 wait complete. 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.135+0000 7f2d9a71a740 1 -- start start 2024-09-12T03:24:54.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 Processor -- start 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621e3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c0621e6fe0 con 0x55c0621e4340 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c0620fa060 con 0x55c0621dab10 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c0621e7160 con 0x55c0621d95a0 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.136+0000 7f2d9a71a740 10 client.0 _ll_register_callbacks cb 0x55c061e4d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:54.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55320/0 (socket says 172.21.15.19:55320) 2024-09-12T03:24:54.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 -- 172.21.15.19:0/152522609 learned_addr learned my addr 172.21.15.19:0/152522609 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 -- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 msgr2=0x55c0621e3da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621e3da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 -- 172.21.15.19:0/152522609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 msgr2=0x55c0621e1050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e1050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.137+0000 7f2d968de640 1 -- 172.21.15.19:0/152522609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c061eb03c0 con 0x55c0621e4340 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.138+0000 7f2d968de640 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2d88097280 tx=0x7f2d8809a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=918be16526c83708 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.139+0000 7f2d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.139+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8809f1d0 con 0x55c0621e4340 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.139+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d80005680 con 0x55c0621e4340 2024-09-12T03:24:54.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.139+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d880a3600 con 0x55c0621e4340 2024-09-12T03:24:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.139+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2d880a3a90 con 0x55c0621e4340 2024-09-12T03:24:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.141+0000 7f2d9a71a740 10 client.16473 fetch_fsmap learned FSMap version 36 2024-09-12T03:24:54.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.141+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c0621f2b00 con 0x55c0621e4340 2024-09-12T03:24:54.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.142+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d80005680 con 0x55c0621e4340 2024-09-12T03:24:54.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.142+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f2d8809a340 con 0x55c0621e4340 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 10 client.16473 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 20 client.16473 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c0621f4480 0x55c0621f6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 4 client.16473 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:24:54.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c06214c860 con 0x55c0621f4480 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.143+0000 7f2d960dd640 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c0621f4480 0x55c0621f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.145+0000 7f2d960dd640 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c0621f4480 0x55c0621f6940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2682e3be1c9dc21 server_cookie=4af48e754f98604d in_seq=0 out_seq=0 2024-09-12T03:24:54.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.145+0000 7f2d79ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.150+0000 7f2d79ffb640 1 -- 172.21.15.19:0/152522609 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c06214c860 con 0x55c0621f4480 2024-09-12T03:24:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.150+0000 7f2d79ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-12T03:24:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.150+0000 7f2d9a71a740 1 client.16473 shutdown 2024-09-12T03:24:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.151+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 msgr2=0x55c0621e17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.151+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2d88097280 tx=0x7f2d8809a840 comp rx=0 tx=0).stop 2024-09-12T03:24:54.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 shutdown_connections 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e17e0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c0621f4480 0x55c0621f6940 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/152522609 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621e3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 >> 172.21.15.19:0/152522609 conn(0x55c0621a2310 msgr2=0x55c0621db900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.152+0000 7f2d79ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.157+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 shutdown_connections 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.157+0000 7f2d9a71a740 1 -- 172.21.15.19:0/152522609 wait complete. 2024-09-12T03:24:54.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.157+0000 7f2d9a71a740 20 client.16473 trim_cache size 0 max 16384 2024-09-12T03:24:54.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.209+0000 7f2d9a71a740 1 Processor -- start 2024-09-12T03:24:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.213+0000 7f2d9a71a740 1 -- start start 2024-09-12T03:24:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.213+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.213+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:41762/0 (socket says 172.21.15.19:41762) 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d960dd640 1 -- 172.21.15.19:0/1153193490 learned_addr learned my addr 172.21.15.19:0/1153193490 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d958dc640 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621dc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90073f70 con 0x55c0621e4340 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a4f90 con 0x55c0621dab10 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a39a0 con 0x55c0621d95a0 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.214+0000 7f2d968de640 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621dc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.215+0000 7f2d968de640 1 -- 172.21.15.19:0/1153193490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 msgr2=0x55c0621d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.215+0000 7f2d968de640 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.215+0000 7f2d968de640 1 -- 172.21.15.19:0/1153193490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 msgr2=0x55c0621e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.215+0000 7f2d968de640 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.216+0000 7f2d968de640 1 -- 172.21.15.19:0/1153193490 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0621e7150 con 0x55c0621e4340 2024-09-12T03:24:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.217+0000 7f2d968de640 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621dc0f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2d880a3a00 tx=0x7f2d880bb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477dba07008aaaf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.218+0000 7f2d7bfff640 1 -- 172.21.15.19:0/1153193490 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8809f1d0 con 0x55c0621e4340 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.218+0000 7f2d7bfff640 1 -- 172.21.15.19:0/1153193490 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d88097490 con 0x55c0621e4340 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 msgr2=0x55c0621dc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621dc0f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f2d880a3a00 tx=0x7f2d880bb050 comp rx=0 tx=0).stop 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 shutdown_connections 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621dc0f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621dab10 0x55c0621e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/1153193490 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621d95a0 0x55c0621d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.219+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 >> 172.21.15.19:0/1153193490 conn(0x55c0621a2310 msgr2=0x55c0621d56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.222+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 shutdown_connections 2024-09-12T03:24:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.223+0000 7f2d9a71a740 1 -- 172.21.15.19:0/1153193490 wait complete. 2024-09-12T03:24:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.223+0000 7f2d9a71a740 1 -- start start 2024-09-12T03:24:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.223+0000 7f2d9a71a740 1 Processor -- start 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.227+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 0x55c0621ecd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 0x55c0621ecd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d960dd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 0x55c0621ecd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56540/0 (socket says 172.21.15.19:56540) 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d958dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d958dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:41778/0 (socket says 172.21.15.19:41778) 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d9a71a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d960dd640 1 -- 172.21.15.19:0/2047174586 learned_addr learned my addr 172.21.15.19:0/2047174586 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a4800 con 0x55c0621e4340 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d901a4130 con 0x55c0621d95a0 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d968de640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.228+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2d90101200 con 0x55c0621dab10 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d958dc640 1 -- 172.21.15.19:0/2047174586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 msgr2=0x55c0621ecd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d958dc640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 0x55c0621ecd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d958dc640 1 -- 172.21.15.19:0/2047174586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 msgr2=0x55c0621e7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d958dc640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d958dc640 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06225d210 con 0x55c0621dab10 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d9a71a740 10 client.0 _ll_register_callbacks cb 0x7f2d88014c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:24:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c062146d70 con 0x55c0621dab10 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.229+0000 7f2d968de640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.230+0000 7f2d958dc640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55c0621e2600 tx=0x7f2d800052c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9f637669f8c4d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.231+0000 7f2d79ffb640 10 client.25975 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.231+0000 7f2d79ffb640 1 -- 172.21.15.19:0/2047174586 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d80055060 con 0x55c0621dab10 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.231+0000 7f2d79ffb640 1 -- 172.21.15.19:0/2047174586 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2d800046c0 con 0x55c0621dab10 2024-09-12T03:24:54.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.231+0000 7f2d79ffb640 1 -- 172.21.15.19:0/2047174586 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2d80004920 con 0x55c0621dab10 2024-09-12T03:24:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.235+0000 7f2d9a71a740 10 client.25975 fetch_fsmap learned FSMap version 36 2024-09-12T03:24:54.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.235+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=36}) -- 0x55c06225e1b0 con 0x55c0621dab10 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d79ffb640 1 -- 172.21.15.19:0/2047174586 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f2d80004ac0 con 0x55c0621dab10 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 10 client.25975 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 10 client.25975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 20 client.25975 populate_metadata read hostname 'smithi019' 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c06225e7f0 0x55c062260c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 4 client.25975 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0620c3170 con 0x55c06225e7f0 2024-09-12T03:24:54.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.236+0000 7f2d960dd640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c06225e7f0 0x55c062260c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:24:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.243+0000 7f2d960dd640 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c06225e7f0 0x55c062260c70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d64af7e64e9cc0d server_cookie=8b8b09387becfbc in_seq=0 out_seq=0 2024-09-12T03:24:54.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.246+0000 7f2d79ffb640 10 client.25975 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.252+0000 7f2d79ffb640 1 -- 172.21.15.19:0/2047174586 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0620c3170 con 0x55c06225e7f0 2024-09-12T03:24:54.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.252+0000 7f2d79ffb640 10 client.25975 handle_command_reply: tid=0 2024-09-12T03:24:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.252+0000 7f2d9a71a740 1 client.25975 shutdown 2024-09-12T03:24:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.253+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 msgr2=0x55c0621e7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.253+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55c0621e2600 tx=0x7f2d800052c0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 shutdown_connections 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c0621e4340 0x55c0621e7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c0621dab10 0x55c0621e7320 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c06225e7f0 0x55c062260c70 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 --2- 172.21.15.19:0/2047174586 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c0621d95a0 0x55c0621ecd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 >> 172.21.15.19:0/2047174586 conn(0x55c0621a2310 msgr2=0x55c0621e30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.256+0000 7f2d79ffb640 0 client.25975 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:24:54.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.257+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 shutdown_connections 2024-09-12T03:24:54.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.260+0000 7f2d9a71a740 1 -- 172.21.15.19:0/2047174586 wait complete. 2024-09-12T03:24:54.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.261+0000 7f2d9a71a740 20 client.25975 trim_cache size 0 max 16384 2024-09-12T03:24:54.262 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:24:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 msgr2=0x7f2d7c0a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 0x7f2d7c0a5250 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55c0621d6a90 tx=0x7f2d8c005f90 comp rx=0 tx=0).stop 2024-09-12T03:24:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 msgr2=0x7f2d901a2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:24:54.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800b970 tx=0x7f2d8800be30 comp rx=0 tx=0).stop 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 shutdown_connections 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2d901116b0 0x7f2d901a2850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2d7c0a2d90 0x7f2d7c0a5250 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2d90102ef0 0x7f2d90111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 --2- 172.21.15.19:0/1273785191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d90100e20 0x7f2d90110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 >> 172.21.15.19:0/1273785191 conn(0x7f2d900fc7d0 msgr2=0x7f2d900ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 shutdown_connections 2024-09-12T03:24:54.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:24:54.266+0000 7f2d98341640 1 -- 172.21.15.19:0/1273785191 wait complete. 2024-09-12T03:24:55.356 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:55 smithi019 ceph-mon[30953]: pgmap v513: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 557 op/s 2024-09-12T03:24:55.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:55 smithi121 ceph-mon[34963]: pgmap v513: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 557 op/s 2024-09-12T03:24:55.713 INFO:tasks.workunit.client.0.smithi019.stdout: 496 1988270 2312 1375837 2256 407461 882 2024-09-12T03:24:55.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:55 smithi135 ceph-mon[35474]: pgmap v513: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.8 MiB/s wr, 557 op/s 2024-09-12T03:24:56.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:56 smithi121 ceph-mon[34963]: pgmap v514: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-12T03:24:56.667 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:56 smithi019 ceph-mon[30953]: pgmap v514: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-12T03:24:56.671 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:24:56.672 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:24:56.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:56 smithi135 ceph-mon[35474]: pgmap v514: 97 pgs: 97 active+clean; 1.7 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 12 MiB/s wr, 607 op/s 2024-09-12T03:24:58.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:24:58 smithi135 ceph-mon[35474]: pgmap v515: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 807 op/s 2024-09-12T03:24:58.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:24:58 smithi121 ceph-mon[34963]: pgmap v515: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 807 op/s 2024-09-12T03:24:58.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:24:58 smithi019 ceph-mon[30953]: pgmap v515: 97 pgs: 97 active+clean; 1.7 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 16 MiB/s wr, 807 op/s 2024-09-12T03:25:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:00 smithi135 ceph-mon[35474]: pgmap v516: 97 pgs: 97 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 930 op/s 2024-09-12T03:25:00.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:00 smithi121 ceph-mon[34963]: pgmap v516: 97 pgs: 97 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 930 op/s 2024-09-12T03:25:00.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:00 smithi019 ceph-mon[30953]: pgmap v516: 97 pgs: 97 active+clean; 1.8 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 19 MiB/s wr, 930 op/s 2024-09-12T03:25:01.844 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:02.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.615+0000 7f9230307640 1 -- 172.21.15.19:0/256374653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109830 msgr2=0x7f9228109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.615+0000 7f9230307640 1 --2- 172.21.15.19:0/256374653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109830 0x7f9228109c90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9224009950 tx=0x7f922402a460 comp rx=0 tx=0).stop 2024-09-12T03:25:02.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.622+0000 7f9230307640 1 -- 172.21.15.19:0/256374653 shutdown_connections 2024-09-12T03:25:02.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.622+0000 7f9230307640 1 --2- 172.21.15.19:0/256374653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f922811d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.622+0000 7f9230307640 1 --2- 172.21.15.19:0/256374653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109830 0x7f9228109c90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.622+0000 7f9230307640 1 --2- 172.21.15.19:0/256374653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109450 0x7f9228110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.622+0000 7f9230307640 1 -- 172.21.15.19:0/256374653 >> 172.21.15.19:0/256374653 conn(0x7f922806e140 msgr2=0x7f922806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.626+0000 7f9230307640 1 -- 172.21.15.19:0/256374653 shutdown_connections 2024-09-12T03:25:02.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.626+0000 7f9230307640 1 -- 172.21.15.19:0/256374653 wait complete. 2024-09-12T03:25:02.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.627+0000 7f9230307640 1 Processor -- start 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.629+0000 7f9230307640 1 -- start start 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109830 0x7f92281a7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f92281abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9228120df0 con 0x7f9228109450 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9228120c70 con 0x7f922811b140 2024-09-12T03:25:02.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f9230307640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9228120f70 con 0x7f9228109830 2024-09-12T03:25:02.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44620/0 (socket says 172.21.15.19:44620) 2024-09-12T03:25:02.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f922e0a3640 1 -- 172.21.15.19:0/3707956129 learned_addr learned my addr 172.21.15.19:0/3707956129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:02.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.630+0000 7f922e8a4640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f92281abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922d8a2640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109830 0x7f92281a7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922e0a3640 1 -- 172.21.15.19:0/3707956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109830 msgr2=0x7f92281a7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922e0a3640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109830 0x7f92281a7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922e0a3640 1 -- 172.21.15.19:0/3707956129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 msgr2=0x7f92281abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922e0a3640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f92281abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.631+0000 7f922e0a3640 1 -- 172.21.15.19:0/3707956129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92281ac180 con 0x7f9228109450 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.632+0000 7f922e8a4640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f92281abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.633+0000 7f922e0a3640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f921800d950 tx=0x7f921800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.634+0000 7f921f7fe640 1 -- 172.21.15.19:0/3707956129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218017070 con 0x7f9228109450 2024-09-12T03:25:02.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.634+0000 7f921f7fe640 1 -- 172.21.15.19:0/3707956129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9218009c30 con 0x7f9228109450 2024-09-12T03:25:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.638+0000 7f921f7fe640 1 -- 172.21.15.19:0/3707956129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921800f6b0 con 0x7f9228109450 2024-09-12T03:25:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.641+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92281ac470 con 0x7f9228109450 2024-09-12T03:25:02.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.641+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92281b3d50 con 0x7f9228109450 2024-09-12T03:25:02.653 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:02 smithi019 ceph-mon[30953]: pgmap v517: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-12T03:25:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.650+0000 7f921f7fe640 1 -- 172.21.15.19:0/3707956129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f921800f850 con 0x7f9228109450 2024-09-12T03:25:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.651+0000 7f92326e0740 1 Processor -- start 2024-09-12T03:25:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.652+0000 7f921f7fe640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 0x7f92140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.653+0000 7f921f7fe640 1 -- 172.21.15.19:0/3707956129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f92180ce820 con 0x7f9228109450 2024-09-12T03:25:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.656+0000 7f92326e0740 1 -- start start 2024-09-12T03:25:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.656+0000 7f922e8a4640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 0x7f92140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.657+0000 7f922e8a4640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 0x7f92140a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9220005fd0 tx=0x7f92200095f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af00271f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 0x55af0027c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af0027f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af001d7280 con 0x55af00271b60 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55af0017eb10 con 0x55af0027cd40 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f92326e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af001d3240 con 0x55af00272480 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f922e8a4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af0027f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f922e8a4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af0027f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60584/0 (socket says 172.21.15.19:60584) 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.659+0000 7f922e8a4640 1 -- 172.21.15.19:0/1178648241 learned_addr learned my addr 172.21.15.19:0/1178648241 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 0x55af0027c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 -- 172.21.15.19:0/1178648241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 msgr2=0x55af0027f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af0027f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 -- 172.21.15.19:0/1178648241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 msgr2=0x55af00271f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af00271f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 -- 172.21.15.19:0/1178648241 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0018b1f0 con 0x55af00272480 2024-09-12T03:25:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.660+0000 7f922d8a2640 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 0x55af0027c800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9224000c00 tx=0x7f9224005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f70223eb80c99a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.665+0000 7f921cdea640 1 -- 172.21.15.19:0/1178648241 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224035050 con 0x55af00272480 2024-09-12T03:25:02.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.665+0000 7f921cdea640 1 -- 172.21.15.19:0/1178648241 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9224005020 con 0x55af00272480 2024-09-12T03:25:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.666+0000 7f92326e0740 1 -- 172.21.15.19:0/1178648241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 msgr2=0x55af0027c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.666+0000 7f92326e0740 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 0x55af0027c800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9224000c00 tx=0x7f9224005c00 comp rx=0 tx=0).stop 2024-09-12T03:25:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.668+0000 7f92326e0740 1 -- 172.21.15.19:0/1178648241 shutdown_connections 2024-09-12T03:25:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.668+0000 7f92326e0740 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af0027f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.668+0000 7f92326e0740 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00272480 0x55af0027c800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.668+0000 7f92326e0740 1 --2- 172.21.15.19:0/1178648241 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af00271f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.668+0000 7f92326e0740 1 -- 172.21.15.19:0/1178648241 >> 172.21.15.19:0/1178648241 conn(0x55af0026e0f0 msgr2=0x55af0026e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.675+0000 7f92326e0740 1 -- 172.21.15.19:0/1178648241 shutdown_connections 2024-09-12T03:25:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.675+0000 7f92326e0740 1 -- 172.21.15.19:0/1178648241 wait complete. 2024-09-12T03:25:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.676+0000 7f92326e0740 1 -- start start 2024-09-12T03:25:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.676+0000 7f92326e0740 1 Processor -- start 2024-09-12T03:25:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.677+0000 7f92326e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00279c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.677+0000 7f92326e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 0x55af0027a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.677+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00279c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.677+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00279c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48646/0 (socket says 172.21.15.19:48646) 2024-09-12T03:25:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.678+0000 7f922e0a3640 1 -- 172.21.15.19:0/2303272820 learned_addr learned my addr 172.21.15.19:0/2303272820 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.678+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af0027cd40 0x55af0027a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.678+0000 7f922e8a4640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af0027cd40 0x55af0027a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.678+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af0027f960 con 0x55af0027cd40 2024-09-12T03:25:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.678+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55af00191f10 con 0x55af00272480 2024-09-12T03:25:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.679+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af0027fae0 con 0x55af00271b60 2024-09-12T03:25:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.679+0000 7f922d8a2640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 0x55af0027a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.679+0000 7f92326e0740 10 client.0 _ll_register_callbacks cb 0x55af0015aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:02.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.684+0000 7f922d8a2640 1 -- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 msgr2=0x55af00279c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:02.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.684+0000 7f922d8a2640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00279c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.684+0000 7f922d8a2640 1 -- 172.21.15.19:0/2303272820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af0027cd40 msgr2=0x55af0027a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.684+0000 7f922d8a2640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af0027cd40 0x55af0027a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.684+0000 7f922d8a2640 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aefff486a0 con 0x55af00272480 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.685+0000 7f922d8a2640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 0x55af0027a140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92240098a0 tx=0x7f92240047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d19b553f139bbf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.685+0000 7f9211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.685+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9224031750 con 0x55af00272480 2024-09-12T03:25:02.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.686+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f92240051d0 con 0x55af00272480 2024-09-12T03:25:02.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.689+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9224005020 con 0x55af00272480 2024-09-12T03:25:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.690+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f92240318f0 con 0x55af00272480 2024-09-12T03:25:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.691+0000 7f92326e0740 10 client.25985 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:02.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.691+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55af0028b590 con 0x55af00272480 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9224005020 con 0x55af00272480 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f922403a040 con 0x55af00272480 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 10 client.25985 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 10 client.25985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 20 client.25985 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af0028cbc0 0x55af0028f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 4 client.25985 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.695+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af001e47f0 con 0x55af0028cbc0 2024-09-12T03:25:02.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.696+0000 7f922e0a3640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af0028cbc0 0x55af0028f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.700+0000 7f9211ffb640 10 client.25985 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.700+0000 7f922e0a3640 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af0028cbc0 0x55af0028f000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781476cbaeb67e63 server_cookie=fb9425af081f45c7 in_seq=0 out_seq=0 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.701+0000 7f9211ffb640 1 -- 172.21.15.19:0/2303272820 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af001e47f0 con 0x55af0028cbc0 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.701+0000 7f9211ffb640 10 client.25985 handle_command_reply: tid=0 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.701+0000 7f92326e0740 1 client.25985 shutdown 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.707+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 msgr2=0x55af0027a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.707+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 0x55af0027a140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92240098a0 tx=0x7f92240047a0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 shutdown_connections 2024-09-12T03:25:02.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af0027cd40 0x55af0027a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af0028cbc0 0x55af0028f000 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00272480 0x55af0027a140 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f92240098a0 tx=0x7f92240047a0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 --2- 172.21.15.19:0/2303272820 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00279c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 >> 172.21.15.19:0/2303272820 conn(0x55af0026e0f0 msgr2=0x55af00273e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.708+0000 7f9211ffb640 0 client.25985 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:02.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.710+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 shutdown_connections 2024-09-12T03:25:02.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.711+0000 7f92326e0740 1 -- 172.21.15.19:0/2303272820 wait complete. 2024-09-12T03:25:02.716 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.714+0000 7f92326e0740 20 client.25985 trim_cache size 0 max 16384 2024-09-12T03:25:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.763+0000 7f92326e0740 1 Processor -- start 2024-09-12T03:25:02.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.763+0000 7f92326e0740 1 -- start start 2024-09-12T03:25:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af0027cd40 0x55af002948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00294e10 0x55af00276e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92281bc220 con 0x55af00271b60 2024-09-12T03:25:02.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92281bdee0 con 0x55af00294e10 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f92326e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92281bc8f0 con 0x55af0027cd40 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f922e0a3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44646/0 (socket says 172.21.15.19:44646) 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.764+0000 7f922e0a3640 1 -- 172.21.15.19:0/4230382842 learned_addr learned my addr 172.21.15.19:0/4230382842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 -- 172.21.15.19:0/4230382842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af0027cd40 msgr2=0x55af002948d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af0027cd40 0x55af002948d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 -- 172.21.15.19:0/4230382842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00294e10 msgr2=0x55af00276e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00294e10 0x55af00276e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 -- 172.21.15.19:0/4230382842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af0027fae0 con 0x55af00271b60 2024-09-12T03:25:02.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.765+0000 7f922e0a3640 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92180cc380 tx=0x7f9218013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b8412dfc9fc45d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.767+0000 7f9213fff640 1 -- 172.21.15.19:0/4230382842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921809f4c0 con 0x55af00271b60 2024-09-12T03:25:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.767+0000 7f9213fff640 1 -- 172.21.15.19:0/4230382842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92180a3d80 con 0x55af00271b60 2024-09-12T03:25:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.767+0000 7f9213fff640 1 -- 172.21.15.19:0/4230382842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921809a560 con 0x55af00271b60 2024-09-12T03:25:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.768+0000 7f92326e0740 1 -- 172.21.15.19:0/4230382842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 msgr2=0x55af0027a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.768+0000 7f92326e0740 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f92180cc380 tx=0x7f9218013040 comp rx=0 tx=0).stop 2024-09-12T03:25:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.773+0000 7f92326e0740 1 -- 172.21.15.19:0/4230382842 shutdown_connections 2024-09-12T03:25:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.773+0000 7f92326e0740 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af00294e10 0x55af00276e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.773+0000 7f92326e0740 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af0027cd40 0x55af002948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.773+0000 7f92326e0740 1 --2- 172.21.15.19:0/4230382842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00271b60 0x55af0027a4d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.773+0000 7f92326e0740 1 -- 172.21.15.19:0/4230382842 >> 172.21.15.19:0/4230382842 conn(0x55af0026e0f0 msgr2=0x55af0027bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.780+0000 7f92326e0740 1 -- 172.21.15.19:0/4230382842 shutdown_connections 2024-09-12T03:25:02.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.782+0000 7f92326e0740 1 -- 172.21.15.19:0/4230382842 wait complete. 2024-09-12T03:25:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.783+0000 7f92326e0740 1 -- start start 2024-09-12T03:25:02.783 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.783+0000 7f92326e0740 1 Processor -- start 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.784+0000 7f92326e0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00280e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af00281390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92281bd750 con 0x55af00294e10 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92281bd080 con 0x55af0027cd40 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92281ba160 con 0x55af00271b60 2024-09-12T03:25:02.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f92326e0740 10 client.0 _ll_register_callbacks cb 0x7f9224040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.785+0000 7f922e8a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.786+0000 7f922e8a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44654/0 (socket says 172.21.15.19:44654) 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.786+0000 7f922e8a4640 1 -- 172.21.15.19:0/2292957502 learned_addr learned my addr 172.21.15.19:0/2292957502 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 -- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 msgr2=0x55af00280e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00280e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 -- 172.21.15.19:0/2292957502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 msgr2=0x55af00281390 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af00281390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 -- 172.21.15.19:0/2292957502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af00281c70 con 0x55af00294e10 2024-09-12T03:25:02.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.787+0000 7f922e8a4640 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9220005c20 tx=0x7f9220004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc72a2bba26c0cd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f9211ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922002a640 con 0x55af00294e10 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f922002a7e0 con 0x55af00294e10 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92180a3d80 con 0x55af00294e10 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f922002aa20 con 0x55af00294e10 2024-09-12T03:25:02.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f92326e0740 10 client.16494 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.788+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55af002f7fc0 con 0x55af00294e10 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f922002abc0 con 0x55af00294e10 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f9220028020 con 0x55af00294e10 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 10 client.16494 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 10 client.16494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 20 client.16494 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af002f8700 0x55af002fab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 4 client.16494 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:25:02.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.789+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f92180a3d80 con 0x55af002f8700 2024-09-12T03:25:02.793 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:02 smithi135 ceph-mon[35474]: pgmap v517: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-12T03:25:02.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.794+0000 7f922e0a3640 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af002f8700 0x55af002fab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:02.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.798+0000 7f922e0a3640 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af002f8700 0x55af002fab40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e958408f1c8f97d3 server_cookie=6c08d9cdc204d4f6 in_seq=0 out_seq=0 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f9211ffb640 10 client.16494 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f9211ffb640 1 -- 172.21.15.19:0/2292957502 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f92180a3d80 con 0x55af002f8700 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f9211ffb640 10 client.16494 handle_command_reply: tid=0 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 client.16494 shutdown 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 msgr2=0x55af002f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9220005c20 tx=0x7f9220004a40 comp rx=0 tx=0).stop 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 shutdown_connections 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af00294e10 0x55af002f68a0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55af002f8700 0x55af002fab40 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af0027cd40 0x55af00281390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 --2- 172.21.15.19:0/2292957502 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af00271b60 0x55af00280e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 >> 172.21.15.19:0/2292957502 conn(0x55af0026e0f0 msgr2=0x55af0026ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.801+0000 7f9211ffb640 0 client.16494 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:02.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.807+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 shutdown_connections 2024-09-12T03:25:02.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.809+0000 7f92326e0740 1 -- 172.21.15.19:0/2292957502 wait complete. 2024-09-12T03:25:02.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.810+0000 7f92326e0740 20 client.16494 trim_cache size 0 max 16384 2024-09-12T03:25:02.812 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:25:02.812 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:25:02.812 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "11ac9d1f-af0c-474e-bc0a-a5e3ae8bb146", 2024-09-12T03:25:02.812 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:25:02.812 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:25:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.812+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 msgr2=0x7f92140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.812+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 0x7f92140a50c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9220005fd0 tx=0x7f92200095f0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 msgr2=0x7f92281a71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f921800d950 tx=0x7f921800de10 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 shutdown_connections 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f922811b140 0x7f92281abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9228109830 0x7f92281a7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92140a2c00 0x7f92140a50c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 --2- 172.21.15.19:0/3707956129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9228109450 0x7f92281a71d0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 >> 172.21.15.19:0/3707956129 conn(0x7f922806e140 msgr2=0x7f922811bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 shutdown_connections 2024-09-12T03:25:02.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:02.813+0000 7f9230307640 1 -- 172.21.15.19:0/3707956129 wait complete. 2024-09-12T03:25:02.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:02 smithi121 ceph-mon[34963]: pgmap v517: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 20 MiB/s wr, 1.00k op/s 2024-09-12T03:25:03.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:03.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:03.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:03.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:04.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:04.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:25:04.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:04 smithi121 ceph-mon[34963]: pgmap v518: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 921 op/s 2024-09-12T03:25:04.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:25:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:04 smithi019 ceph-mon[30953]: pgmap v518: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 921 op/s 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:25:05.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:04 smithi135 ceph-mon[35474]: pgmap v518: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 19 MiB/s wr, 921 op/s 2024-09-12T03:25:05.713 INFO:tasks.workunit.client.0.smithi019.stdout: 514 1906700 1016 1325224 951 395048 374 2024-09-12T03:25:06.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:06 smithi019 ceph-mon[30953]: pgmap v519: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:06.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:06 smithi121 ceph-mon[34963]: pgmap v519: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:07.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:06 smithi135 ceph-mon[35474]: pgmap v519: 97 pgs: 97 active+clean; 1.8 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:08.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:08 smithi121 ceph-mon[34963]: pgmap v520: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-12T03:25:08.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:08 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:08.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:08 smithi019 ceph-mon[30953]: pgmap v520: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-12T03:25:08.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:08 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:09.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:08 smithi135 ceph-mon[35474]: pgmap v520: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 13 MiB/s wr, 668 op/s 2024-09-12T03:25:09.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:08 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:09.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:09 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:09.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:09 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:10.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:09 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:10 smithi121 ceph-mon[34963]: pgmap v521: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-12T03:25:10.917 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:10 smithi019 ceph-mon[30953]: pgmap v521: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-12T03:25:11.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:10 smithi135 ceph-mon[35474]: pgmap v521: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.6 MiB/s wr, 434 op/s 2024-09-12T03:25:11.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:11 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:11.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:11 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:11.938 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:12.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:11 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:12.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:12 smithi121 ceph-mon[34963]: pgmap v522: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 302 op/s 2024-09-12T03:25:12.909 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:12 smithi019 ceph-mon[30953]: pgmap v522: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 302 op/s 2024-09-12T03:25:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:12 smithi135 ceph-mon[35474]: pgmap v522: 97 pgs: 97 active+clean; 1.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 302 op/s 2024-09-12T03:25:13.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.771+0000 7f1c0cc52640 1 -- 172.21.15.19:0/2166059187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 msgr2=0x7f1c08074970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.771+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/2166059187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08074970 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009900 tx=0x7f1bf802a4a0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.778+0000 7f1c0cc52640 1 -- 172.21.15.19:0/2166059187 shutdown_connections 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.778+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/2166059187 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c08112b90 0x7f1c08114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.778+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/2166059187 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08074eb0 0x7f1c08112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.778+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/2166059187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08074970 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.778+0000 7f1c0cc52640 1 -- 172.21.15.19:0/2166059187 >> 172.21.15.19:0/2166059187 conn(0x7f1c080fd8c0 msgr2=0x7f1c080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.779+0000 7f1c0cc52640 1 -- 172.21.15.19:0/2166059187 shutdown_connections 2024-09-12T03:25:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.779+0000 7f1c0cc52640 1 -- 172.21.15.19:0/2166059187 wait complete. 2024-09-12T03:25:13.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.780+0000 7f1c0cc52640 1 Processor -- start 2024-09-12T03:25:13.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.782+0000 7f1c0cc52640 1 -- start start 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08112360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c08074eb0 0x7f1c0810c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1c08106870 con 0x7f1c08112b90 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081066f0 con 0x7f1c08074590 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c0cc52640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1c081069f0 con 0x7f1c08074eb0 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c06d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c06d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32952/0 (socket says 172.21.15.19:32952) 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.783+0000 7f1c06d9d640 1 -- 172.21.15.19:0/806242863 learned_addr learned my addr 172.21.15.19:0/806242863 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.785+0000 7f1c0659c640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08112360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.785+0000 7f1c06d9d640 1 -- 172.21.15.19:0/806242863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c08074eb0 msgr2=0x7f1c0810c3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:25:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.785+0000 7f1c06d9d640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c08074eb0 0x7f1c0810c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.785+0000 7f1c06d9d640 1 -- 172.21.15.19:0/806242863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 msgr2=0x7f1c08112360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.786+0000 7f1c06d9d640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08112360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.786+0000 7f1c06d9d640 1 -- 172.21.15.19:0/806242863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0810d0f0 con 0x7f1c08112b90 2024-09-12T03:25:13.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.791+0000 7f1c06d9d640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00d720 tx=0x7f1bfc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.792+0000 7f1bf77fe640 1 -- 172.21.15.19:0/806242863 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc017070 con 0x7f1c08112b90 2024-09-12T03:25:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.792+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0811dbf0 con 0x7f1c08112b90 2024-09-12T03:25:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.792+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0811e130 con 0x7f1c08112b90 2024-09-12T03:25:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.792+0000 7f1bf77fe640 1 -- 172.21.15.19:0/806242863 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bfc00de30 con 0x7f1c08112b90 2024-09-12T03:25:13.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.792+0000 7f1bf77fe640 1 -- 172.21.15.19:0/806242863 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc004d90 con 0x7f1c08112b90 2024-09-12T03:25:13.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.803+0000 7f1c0f02b740 1 Processor -- start 2024-09-12T03:25:13.814 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:13 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.815+0000 7f1c0f02b740 1 -- start start 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 0x5615ba526880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba531140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba531680 0x5615ba533a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5615ba48c2e0 con 0x5615ba526dc0 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5615ba433ce0 con 0x5615ba531680 2024-09-12T03:25:13.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.816+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5615ba4882a0 con 0x5615ba5264a0 2024-09-12T03:25:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.817+0000 7f1bf77fe640 1 -- 172.21.15.19:0/806242863 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bfc009bd0 con 0x7f1c08112b90 2024-09-12T03:25:13.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.818+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 0x5615ba526880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.821+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 0x5615ba526880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:37578/0 (socket says 172.21.15.19:37578) 2024-09-12T03:25:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.822+0000 7f1c05d9b640 1 -- 172.21.15.19:0/1829700768 learned_addr learned my addr 172.21.15.19:0/1829700768 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.822+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba531680 0x5615ba533a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.822+0000 7f1bf77fe640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 0x7f1bf00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.822+0000 7f1bf77fe640 1 -- 172.21.15.19:0/806242863 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bfc0ce860 con 0x7f1c08112b90 2024-09-12T03:25:13.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.821+0000 7f1c0659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba531140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c06d9d640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 0x7f1bf00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c0659c640 1 -- 172.21.15.19:0/1829700768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 msgr2=0x5615ba526880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 0x5615ba526880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c0659c640 1 -- 172.21.15.19:0/1829700768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba531680 msgr2=0x5615ba533a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba531680 0x5615ba533a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.824+0000 7f1c0659c640 1 -- 172.21.15.19:0/1829700768 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ba418440 con 0x5615ba526dc0 2024-09-12T03:25:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.825+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba531140 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8033040 tx=0x7f1bf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb117f4a27350a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.825+0000 7f1c06d9d640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 0x7f1bf00a5180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09ad70 tx=0x7f1bfc0bb040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.829+0000 7f1bf4ff9640 1 -- 172.21.15.19:0/1829700768 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8041070 con 0x5615ba526dc0 2024-09-12T03:25:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.829+0000 7f1bf4ff9640 1 -- 172.21.15.19:0/1829700768 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf8005000 con 0x5615ba526dc0 2024-09-12T03:25:13.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.830+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1829700768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 msgr2=0x5615ba531140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.830+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba531140 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8033040 tx=0x7f1bf80046c0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.832+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1829700768 shutdown_connections 2024-09-12T03:25:13.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.832+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba531680 0x5615ba533a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.832+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba531140 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.832+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1829700768 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba5264a0 0x5615ba526880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.832+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1829700768 >> 172.21.15.19:0/1829700768 conn(0x5615ba522a30 msgr2=0x5615ba522e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.834+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1829700768 shutdown_connections 2024-09-12T03:25:13.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.836+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1829700768 wait complete. 2024-09-12T03:25:13.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.836+0000 7f1c0f02b740 1 -- start start 2024-09-12T03:25:13.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 Processor -- start 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba52e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba52e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5615ba5342a0 con 0x5615ba526dc0 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5615ba447050 con 0x5615ba5264a0 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5615ba534420 con 0x5615ba531680 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.837+0000 7f1c0f02b740 10 client.0 _ll_register_callbacks cb 0x5615ba487160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:13.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33846/0 (socket says 172.21.15.19:33846) 2024-09-12T03:25:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 -- 172.21.15.19:0/2272733015 learned_addr learned my addr 172.21.15.19:0/2272733015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba52e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c0659c640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba52e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 -- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 msgr2=0x5615ba52e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba52e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 -- 172.21.15.19:0/2272733015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 msgr2=0x5615ba52e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba52e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.838+0000 7f1c05d9b640 1 -- 172.21.15.19:0/2272733015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ba1fd3c0 con 0x5615ba5264a0 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.839+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c00016300 tx=0x7f1c00004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6043f44e99df48d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.840+0000 7f1c0659c640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba52e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.840+0000 7f1bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.840+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010040 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.840+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1c00016c80 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.840+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bf8005000 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.841+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c00016e80 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.841+0000 7f1c0f02b740 10 client.25994 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.841+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5615ba53fca0 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.842+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf8005000 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.842+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1c00018500 con 0x5615ba5264a0 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 10 client.25994 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:13.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 10 client.25994 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 20 client.25994 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba541660 0x5615ba543b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 4 client.25994 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615ba499850 con 0x5615ba541660 2024-09-12T03:25:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.843+0000 7f1c0659c640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba541660 0x5615ba543b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.857+0000 7f1c0659c640 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba541660 0x5615ba543b00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e4782ac588b72ab server_cookie=4b2a0b34bde46e5f in_seq=0 out_seq=0 2024-09-12T03:25:13.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.857+0000 7f1bedffb640 10 client.25994 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:13.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1bedffb640 1 -- 172.21.15.19:0/2272733015 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5615ba499850 con 0x5615ba541660 2024-09-12T03:25:13.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1bedffb640 10 client.25994 handle_command_reply: tid=0 2024-09-12T03:25:13.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1c0f02b740 1 client.25994 shutdown 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 msgr2=0x5615ba530f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c00016300 tx=0x7f1c00004a40 comp rx=0 tx=0).stop 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 shutdown_connections 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.865+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba52e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.866+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba52e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.866+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba541660 0x5615ba543b00 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.866+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/2272733015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba5264a0 0x5615ba530f40 secure :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c00016300 tx=0x7f1c00004a40 comp rx=0 tx=0).stop 2024-09-12T03:25:13.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.866+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 >> 172.21.15.19:0/2272733015 conn(0x5615ba522a30 msgr2=0x5615ba528760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.868+0000 7f1bedffb640 0 client.25994 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:13.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.869+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 shutdown_connections 2024-09-12T03:25:13.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.869+0000 7f1c0f02b740 1 -- 172.21.15.19:0/2272733015 wait complete. 2024-09-12T03:25:13.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.872+0000 7f1c0f02b740 20 client.25994 trim_cache size 0 max 16384 2024-09-12T03:25:13.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:13 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.934+0000 7f1c0f02b740 1 Processor -- start 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.934+0000 7f1c0f02b740 1 -- start start 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.934+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.934+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba531680 0x5615ba521a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.935+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba549410 0x5615ba549890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.935+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be800e630 con 0x5615ba531680 2024-09-12T03:25:13.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.935+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be80102f0 con 0x5615ba526dc0 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.935+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be800ed00 con 0x5615ba549410 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.936+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.936+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:33848/0 (socket says 172.21.15.19:33848) 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.936+0000 7f1c05d9b640 1 -- 172.21.15.19:0/3834271051 learned_addr learned my addr 172.21.15.19:0/3834271051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.936+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba549410 0x5615ba549890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.937+0000 7f1c05d9b640 1 -- 172.21.15.19:0/3834271051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba549410 msgr2=0x5615ba549890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.937+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba549410 0x5615ba549890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.937+0000 7f1c05d9b640 1 -- 172.21.15.19:0/3834271051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba531680 msgr2=0x5615ba521a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.937+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba531680 0x5615ba521a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.937+0000 7f1c05d9b640 1 -- 172.21.15.19:0/3834271051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615ba52d1a0 con 0x5615ba526dc0 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.938+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c00014c90 tx=0x7f1c00000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907c360d5392e827 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.938+0000 7f1beffff640 1 -- 172.21.15.19:0/3834271051 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00010070 con 0x5615ba526dc0 2024-09-12T03:25:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.938+0000 7f1beffff640 1 -- 172.21.15.19:0/3834271051 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c000013e0 con 0x5615ba526dc0 2024-09-12T03:25:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.938+0000 7f1beffff640 1 -- 172.21.15.19:0/3834271051 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00018d20 con 0x5615ba526dc0 2024-09-12T03:25:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.940+0000 7f1c0f02b740 1 -- 172.21.15.19:0/3834271051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 msgr2=0x5615ba52cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.941+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1c00014c90 tx=0x7f1c00000f40 comp rx=0 tx=0).stop 2024-09-12T03:25:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.943+0000 7f1c0f02b740 1 -- 172.21.15.19:0/3834271051 shutdown_connections 2024-09-12T03:25:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.943+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba549410 0x5615ba549890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.944+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba531680 0x5615ba521a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.944+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/3834271051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba526dc0 0x5615ba52cac0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.944+0000 7f1c0f02b740 1 -- 172.21.15.19:0/3834271051 >> 172.21.15.19:0/3834271051 conn(0x5615ba522a30 msgr2=0x5615ba521370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.946+0000 7f1c0f02b740 1 -- 172.21.15.19:0/3834271051 shutdown_connections 2024-09-12T03:25:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.946+0000 7f1c0f02b740 1 -- 172.21.15.19:0/3834271051 wait complete. 2024-09-12T03:25:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.947+0000 7f1c0f02b740 1 -- start start 2024-09-12T03:25:13.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.947+0000 7f1c0f02b740 1 Processor -- start 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.948+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba539bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.948+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba536f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba539bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c0f02b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba549410 0x5615ba5374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be800fb60 con 0x5615ba526dc0 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be800f490 con 0x5615ba549410 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c0f02b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be800c550 con 0x5615ba531680 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c0f02b740 10 client.0 _ll_register_callbacks cb 0x7f1c00001f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c05d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba539bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:32994/0 (socket says 172.21.15.19:32994) 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.949+0000 7f1c05d9b640 1 -- 172.21.15.19:0/1207498142 learned_addr learned my addr 172.21.15.19:0/1207498142 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba536f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba549410 0x5615ba5374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 -- 172.21.15.19:0/1207498142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba549410 msgr2=0x5615ba5374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba549410 0x5615ba5374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 -- 172.21.15.19:0/1207498142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 msgr2=0x5615ba539bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba539bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.950+0000 7f1c0659c640 1 -- 172.21.15.19:0/1207498142 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615ba537a90 con 0x5615ba531680 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.951+0000 7f1c0659c640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba536f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802aa90 tx=0x7f1bf8004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e262671fcc771b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.951+0000 7f1bedffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.951+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8056060 con 0x5615ba531680 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.951+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bf8005370 con 0x5615ba531680 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.951+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c000013e0 con 0x5615ba531680 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.952+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1bf8041040 con 0x5615ba531680 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.952+0000 7f1c0f02b740 10 client.26002 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:13.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.952+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5615ba5ac170 con 0x5615ba531680 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bf8005570 con 0x5615ba531680 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1bf8003cf0 con 0x5615ba531680 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 10 client.26002 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 10 client.26002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 20 client.26002 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba5ac4b0 0x5615ba5ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:13.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 4 client.26002 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:25:13.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1c000013e0 con 0x5615ba5ac4b0 2024-09-12T03:25:13.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.960+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba5ac4b0 0x5615ba5ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:13.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.961+0000 7f1bedffb640 10 client.26002 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:13.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.962+0000 7f1c05d9b640 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba5ac4b0 0x5615ba5ae8a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f4b707fe68639f server_cookie=a2ed0332e1287868 in_seq=0 out_seq=0 2024-09-12T03:25:13.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.964+0000 7f1bedffb640 1 -- 172.21.15.19:0/1207498142 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f1c000013e0 con 0x5615ba5ac4b0 2024-09-12T03:25:13.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.964+0000 7f1bedffb640 10 client.26002 handle_command_reply: tid=0 2024-09-12T03:25:13.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.964+0000 7f1c0f02b740 1 client.26002 shutdown 2024-09-12T03:25:13.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 msgr2=0x5615ba536f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba536f90 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1bf802aa90 tx=0x7f1bf8004d50 comp rx=0 tx=0).stop 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 shutdown_connections 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5615ba5ac4b0 0x5615ba5ae8a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5615ba549410 0x5615ba5374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5615ba531680 0x5615ba536f90 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 --2- 172.21.15.19:0/1207498142 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5615ba526dc0 0x5615ba539bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 >> 172.21.15.19:0/1207498142 conn(0x5615ba522a30 msgr2=0x5615ba52e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1bedffb640 0 client.26002 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 shutdown_connections 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.965+0000 7f1c0f02b740 1 -- 172.21.15.19:0/1207498142 wait complete. 2024-09-12T03:25:13.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.966+0000 7f1c0f02b740 20 client.26002 trim_cache size 0 max 16384 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (670 inodes in the stack)", 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "11ac9d1f-af0c-474e-bc0a-a5e3ae8bb146": { 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "11ac9d1f-af0c-474e-bc0a-a5e3ae8bb146", 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:25:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:25:13.974 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.974+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 msgr2=0x7f1bf00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.974+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 0x7f1bf00a5180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc09ad70 tx=0x7f1bfc0bb040 comp rx=0 tx=0).stop 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.974+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 msgr2=0x7f1c0810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.974+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00d720 tx=0x7f1bfc00dbe0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 shutdown_connections 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bf00a2cc0 0x7f1bf00a5180 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1c08112b90 0x7f1c0810c920 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1c08074eb0 0x7f1c0810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 --2- 172.21.15.19:0/806242863 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c08074590 0x7f1c08112360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.975+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 >> 172.21.15.19:0/806242863 conn(0x7f1c080fd8c0 msgr2=0x7f1c080ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.976+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 shutdown_connections 2024-09-12T03:25:13.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:13.976+0000 7f1c0cc52640 1 -- 172.21.15.19:0/806242863 wait complete. 2024-09-12T03:25:14.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:13 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:14 smithi121 ceph-mon[34963]: pgmap v523: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 377 op/s 2024-09-12T03:25:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:14 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:14.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:14 smithi019 ceph-mon[30953]: pgmap v523: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 377 op/s 2024-09-12T03:25:14.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:14 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:15.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:14 smithi135 ceph-mon[35474]: pgmap v523: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 7.7 MiB/s wr, 377 op/s 2024-09-12T03:25:15.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:14 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:25:15.715 INFO:tasks.workunit.client.0.smithi019.stdout: 542 1812901 1413 1257211 1518 376002 1000 2024-09-12T03:25:16.635 INFO:tasks.cephfs.filesystem:scrub status for tag:11ac9d1f-af0c-474e-bc0a-a5e3ae8bb146 - {'path': '/', 'tag': '11ac9d1f-af0c-474e-bc0a-a5e3ae8bb146', 'options': 'recursive,force'} 2024-09-12T03:25:16.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:16 smithi121 ceph-mon[34963]: pgmap v524: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.5 MiB/s wr, 417 op/s 2024-09-12T03:25:16.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:16 smithi019 ceph-mon[30953]: pgmap v524: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.5 MiB/s wr, 417 op/s 2024-09-12T03:25:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:16 smithi135 ceph-mon[35474]: pgmap v524: 97 pgs: 97 active+clean; 1.9 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.5 MiB/s wr, 417 op/s 2024-09-12T03:25:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:18 smithi121 ceph-mon[34963]: pgmap v525: 97 pgs: 97 active+clean; 2.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-12T03:25:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:18 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:25:18.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:18 smithi019 ceph-mon[30953]: pgmap v525: 97 pgs: 97 active+clean; 2.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-12T03:25:18.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:18 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:25:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:18 smithi135 ceph-mon[35474]: pgmap v525: 97 pgs: 97 active+clean; 2.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 12 MiB/s wr, 550 op/s 2024-09-12T03:25:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:18 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:25:19.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:25:19.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:25:19.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:25:19.918 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:25:19.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:25:19.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:25:19.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:25:19.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:25:20.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:25:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:25:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:25:20.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:25:20.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:20 smithi019 ceph-mon[30953]: pgmap v526: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 697 op/s 2024-09-12T03:25:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:20 smithi135 ceph-mon[35474]: pgmap v526: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 697 op/s 2024-09-12T03:25:21.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:20 smithi121 ceph-mon[34963]: pgmap v526: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 16 MiB/s wr, 697 op/s 2024-09-12T03:25:22.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:22 smithi019 ceph-mon[30953]: pgmap v527: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 786 op/s 2024-09-12T03:25:23.000 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T03:25:23.000 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:23.043 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:22 smithi135 ceph-mon[35474]: pgmap v527: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 786 op/s 2024-09-12T03:25:23.151 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:22 smithi121 ceph-mon[34963]: pgmap v527: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 17 MiB/s wr, 786 op/s 2024-09-12T03:25:24.951 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:24 smithi121 ceph-mon[34963]: pgmap v528: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 772 op/s 2024-09-12T03:25:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:24 smithi135 ceph-mon[35474]: pgmap v528: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 772 op/s 2024-09-12T03:25:25.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:24 smithi019 ceph-mon[30953]: pgmap v528: 97 pgs: 97 active+clean; 2.0 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 17 MiB/s wr, 772 op/s 2024-09-12T03:25:25.716 INFO:tasks.workunit.client.0.smithi019.stdout: 582 2186035 2205 1513565 2027 459701 1735 2024-09-12T03:25:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:26 smithi135 ceph-mon[35474]: pgmap v529: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 640 op/s 2024-09-12T03:25:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:26 smithi121 ceph-mon[34963]: pgmap v529: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 640 op/s 2024-09-12T03:25:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:26 smithi019 ceph-mon[30953]: pgmap v529: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 640 op/s 2024-09-12T03:25:27.643 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.060+0000 7f77e1a4c640 1 -- 172.21.15.19:0/3199293191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc1008d0 msgr2=0x7f77dc100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.062+0000 7f77e1a4c640 1 --2- 172.21.15.19:0/3199293191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc1008d0 0x7f77dc100d30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f77c4009900 tx=0x7f77c402a400 comp rx=0 tx=0).stop 2024-09-12T03:25:29.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.063+0000 7f77e1a4c640 1 -- 172.21.15.19:0/3199293191 shutdown_connections 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.063+0000 7f77e1a4c640 1 --2- 172.21.15.19:0/3199293191 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc101270 0x7f77dc10d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.063+0000 7f77e1a4c640 1 --2- 172.21.15.19:0/3199293191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc1008d0 0x7f77dc100d30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.063+0000 7f77e1a4c640 1 --2- 172.21.15.19:0/3199293191 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1068d0 0x7f77dc106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.063+0000 7f77e1a4c640 1 -- 172.21.15.19:0/3199293191 >> 172.21.15.19:0/3199293191 conn(0x7f77dc0fc610 msgr2=0x7f77dc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:29.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.064+0000 7f77e1a4c640 1 -- 172.21.15.19:0/3199293191 shutdown_connections 2024-09-12T03:25:29.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.068+0000 7f77e1a4c640 1 -- 172.21.15.19:0/3199293191 wait complete. 2024-09-12T03:25:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.069+0000 7f77e1a4c640 1 Processor -- start 2024-09-12T03:25:29.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.069+0000 7f77e1a4c640 1 -- start start 2024-09-12T03:25:29.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.070+0000 7f77e1a4c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 0x7f77dc1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.070+0000 7f77e0a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 0x7f77dc1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.070+0000 7f77e0a4a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 0x7f77dc1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49060/0 (socket says 172.21.15.19:49060) 2024-09-12T03:25:29.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.070+0000 7f77e1a4c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 0x7f77dc1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e0a4a640 1 -- 172.21.15.19:0/627593126 learned_addr learned my addr 172.21.15.19:0/627593126 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77dbfff640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 0x7f77dc1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e1a4c640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc1068d0 0x7f77dc19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e1a4c640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110cf0 con 0x7f77dc1008d0 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e1a4c640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110b70 con 0x7f77dc101270 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e1a4c640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc110e70 con 0x7f77dc1068d0 2024-09-12T03:25:29.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.071+0000 7f77e124b640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc1068d0 0x7f77dc19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.072+0000 7f77dbfff640 1 -- 172.21.15.19:0/627593126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc1068d0 msgr2=0x7f77dc19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.072+0000 7f77dbfff640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc1068d0 0x7f77dc19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.072+0000 7f77dbfff640 1 -- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 msgr2=0x7f77dc1971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.072+0000 7f77dbfff640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 0x7f77dc1971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.072+0000 7f77dbfff640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc19c160 con 0x7f77dc101270 2024-09-12T03:25:29.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.073+0000 7f77dbfff640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 0x7f77dc1976f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77c402a920 tx=0x7f77c4003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.079+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c403f070 con 0x7f77dc101270 2024-09-12T03:25:29.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.079+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77c4031c30 con 0x7f77dc101270 2024-09-12T03:25:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.080+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4035030 con 0x7f77dc101270 2024-09-12T03:25:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.080+0000 7f77e1a4c640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc19c3f0 con 0x7f77dc101270 2024-09-12T03:25:29.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.081+0000 7f77e1a4c640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc19c850 con 0x7f77dc101270 2024-09-12T03:25:29.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.083+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc102010 con 0x7f77dc101270 2024-09-12T03:25:29.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.087+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f77c4042070 con 0x7f77dc101270 2024-09-12T03:25:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.089+0000 7f77d9ffb640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 0x7f77c80a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:29.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.089+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f77c406de00 con 0x7f77dc101270 2024-09-12T03:25:29.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.090+0000 7f77e0a4a640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 0x7f77c80a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:29.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.091+0000 7f77e0a4a640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 0x7f77c80a72f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fa0 tx=0x7f77cc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:29.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.094+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f77c40bccf0 con 0x7f77dc101270 2024-09-12T03:25:29.170 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:28 smithi019 ceph-mon[30953]: pgmap v530: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-09-12T03:25:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:28 smithi135 ceph-mon[35474]: pgmap v530: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-09-12T03:25:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:28 smithi121 ceph-mon[34963]: pgmap v530: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 555 op/s 2024-09-12T03:25:29.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.583+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77dc198510 con 0x7f77dc101270 2024-09-12T03:25:29.587 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:25:29.587 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"btime":"2024-09-12T03:24:43:270632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:24:43.042444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:25:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.586+0000 7f77d9ffb640 1 -- 172.21.15.19:0/627593126 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12002 (secure 0 0 0) 0x7f77c40bc5f0 con 0x7f77dc101270 2024-09-12T03:25:29.587 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-09-12T03:25:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.591+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 msgr2=0x7f77c80a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.591+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 0x7f77c80a72f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009fa0 tx=0x7f77cc009500 comp rx=0 tx=0).stop 2024-09-12T03:25:29.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.591+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 msgr2=0x7f77dc1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:29.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.591+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 0x7f77dc1976f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77c402a920 tx=0x7f77c4003c10 comp rx=0 tx=0).stop 2024-09-12T03:25:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 shutdown_connections 2024-09-12T03:25:29.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f77dc1068d0 0x7f77dc19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f77dc101270 0x7f77dc1976f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f77c80a4e30 0x7f77c80a72f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 --2- 172.21.15.19:0/627593126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f77dc1008d0 0x7f77dc1971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:29.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.594+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 >> 172.21.15.19:0/627593126 conn(0x7f77dc0fc610 msgr2=0x7f77dc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:29.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.598+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 shutdown_connections 2024-09-12T03:25:29.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:29.602+0000 7f77d37fe640 1 -- 172.21.15.19:0/627593126 wait complete. 2024-09-12T03:25:30.197 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:29 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/627593126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:30.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:29 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/627593126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:30.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:29 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/627593126' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:30 smithi135 ceph-mon[35474]: pgmap v531: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 424 op/s 2024-09-12T03:25:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:30 smithi121 ceph-mon[34963]: pgmap v531: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 424 op/s 2024-09-12T03:25:31.428 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:30 smithi019 ceph-mon[30953]: pgmap v531: 97 pgs: 97 active+clean; 2.0 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 8.1 MiB/s wr, 424 op/s 2024-09-12T03:25:31.520 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0' 2024-09-12T03:25:31.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 140.1 --await-for 45.0 2024-09-12T03:25:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:32 smithi135 ceph-mon[35474]: pgmap v532: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2024-09-12T03:25:33.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:32 smithi121 ceph-mon[34963]: pgmap v532: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2024-09-12T03:25:33.416 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:32 smithi019 ceph-mon[30953]: pgmap v532: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.8 MiB/s wr, 313 op/s 2024-09-12T03:25:35.262 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:35.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:34 smithi135 ceph-mon[35474]: pgmap v533: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 380 op/s 2024-09-12T03:25:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:34 smithi121 ceph-mon[34963]: pgmap v533: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 380 op/s 2024-09-12T03:25:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:34 smithi019 ceph-mon[30953]: pgmap v533: 97 pgs: 97 active+clean; 2.1 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.5 MiB/s wr, 380 op/s 2024-09-12T03:25:35.716 INFO:tasks.workunit.client.0.smithi019.stdout: 605 1805340 1414 1236304 1114 393051 1940 2024-09-12T03:25:36.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.758+0000 7f64e7448640 1 -- 172.21.15.19:0/2384740116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e00742d0 msgr2=0x7f64e0074750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.758+0000 7f64e7448640 1 --2- 172.21.15.19:0/2384740116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e00742d0 0x7f64e0074750 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f64d0009900 tx=0x7f64d002a440 comp rx=0 tx=0).stop 2024-09-12T03:25:36.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.759+0000 7f64e7448640 1 -- 172.21.15.19:0/2384740116 shutdown_connections 2024-09-12T03:25:36.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.759+0000 7f64e7448640 1 --2- 172.21.15.19:0/2384740116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e0075390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.759+0000 7f64e7448640 1 --2- 172.21.15.19:0/2384740116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e00742d0 0x7f64e0074750 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.759+0000 7f64e7448640 1 --2- 172.21.15.19:0/2384740116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e01091f0 0x7f64e0109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.759+0000 7f64e7448640 1 -- 172.21.15.19:0/2384740116 >> 172.21.15.19:0/2384740116 conn(0x7f64e006e140 msgr2=0x7f64e006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.762+0000 7f64e7448640 1 -- 172.21.15.19:0/2384740116 shutdown_connections 2024-09-12T03:25:36.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.762+0000 7f64e7448640 1 -- 172.21.15.19:0/2384740116 wait complete. 2024-09-12T03:25:36.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.763+0000 7f64e7448640 1 Processor -- start 2024-09-12T03:25:36.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 -- start start 2024-09-12T03:25:36.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00742d0 0x7f64e0084bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e01091f0 0x7f64e0080180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64e007e860 con 0x7f64e0074c90 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64e007e6e0 con 0x7f64e01091f0 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e7448640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64e007e9e0 con 0x7f64e00742d0 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e49e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e49e3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57212/0 (socket says 172.21.15.19:57212) 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.767+0000 7f64e49e3640 1 -- 172.21.15.19:0/3081177223 learned_addr learned my addr 172.21.15.19:0/3081177223 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.768+0000 7f64e49e3640 1 -- 172.21.15.19:0/3081177223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00742d0 msgr2=0x7f64e0084bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.768+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00742d0 0x7f64e0084bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.768+0000 7f64e49e3640 1 -- 172.21.15.19:0/3081177223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e01091f0 msgr2=0x7f64e0080180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.768+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e01091f0 0x7f64e0080180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.768+0000 7f64e49e3640 1 -- 172.21.15.19:0/3081177223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00808b0 con 0x7f64e0074c90 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.769+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f64d0005ab0 tx=0x7f64d00055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.769+0000 7f64d67fc640 1 -- 172.21.15.19:0/3081177223 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d0041070 con 0x7f64e0074c90 2024-09-12T03:25:36.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.770+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64e01ccfb0 con 0x7f64e0074c90 2024-09-12T03:25:36.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.770+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64e01cd350 con 0x7f64e0074c90 2024-09-12T03:25:36.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.770+0000 7f64d67fc640 1 -- 172.21.15.19:0/3081177223 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d0004200 con 0x7f64e0074c90 2024-09-12T03:25:36.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.770+0000 7f64d67fc640 1 -- 172.21.15.19:0/3081177223 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d003de50 con 0x7f64e0074c90 2024-09-12T03:25:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.781+0000 7f64e9821740 1 Processor -- start 2024-09-12T03:25:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.782+0000 7f64e9821740 1 -- start start 2024-09-12T03:25:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.782+0000 7f64e9821740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067e2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e9821740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e9821740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ed940 0x5601067efd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:36908/0 (socket says 172.21.15.19:36908) 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e51e4640 1 -- 172.21.15.19:0/1867559829 learned_addr learned my addr 172.21.15.19:0/1867559829 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5601067482a0 con 0x5601067e2760 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5601066efb10 con 0x5601067e3080 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560106744260 con 0x5601067ed940 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.783+0000 7f64d67fc640 1 -- 172.21.15.19:0/3081177223 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64d0044070 con 0x7f64e0074c90 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e49e3640 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ed940 0x5601067efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e51e4640 1 -- 172.21.15.19:0/1867559829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ed940 msgr2=0x5601067efd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e51e4640 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ed940 0x5601067efd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e51e4640 1 -- 172.21.15.19:0/1867559829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 msgr2=0x5601067e2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e51e4640 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067e2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.784+0000 7f64e51e4640 1 -- 172.21.15.19:0/1867559829 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601064b9b90 con 0x5601067e3080 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.785+0000 7f64e51e4640 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00a9f0 tx=0x7f64dc00aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae3b75beca8ca7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.786+0000 7f64d67fc640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 0x7f64cc0a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.787+0000 7f64d67fc640 1 -- 172.21.15.19:0/3081177223 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f64d00efc70 con 0x7f64e0074c90 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.787+0000 7f64cb7fe640 1 -- 172.21.15.19:0/1867559829 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00cde0 con 0x5601067e3080 2024-09-12T03:25:36.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.787+0000 7f64cb7fe640 1 -- 172.21.15.19:0/1867559829 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64dc00f930 con 0x5601067e3080 2024-09-12T03:25:36.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e59e5640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 0x7f64cc0a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 msgr2=0x5601067ed400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00a9f0 tx=0x7f64dc00aeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 shutdown_connections 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ed940 0x5601067efd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067e3080 0x5601067ed400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 --2- 172.21.15.19:0/1867559829 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067e2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.794+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 >> 172.21.15.19:0/1867559829 conn(0x5601067decf0 msgr2=0x5601067df0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.795+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 shutdown_connections 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.795+0000 7f64e9821740 1 -- 172.21.15.19:0/1867559829 wait complete. 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.795+0000 7f64e9821740 1 -- start start 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.795+0000 7f64e9821740 1 Processor -- start 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 0x5601067ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067eab70 0x5601067eafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5601067f0560 con 0x5601067eab70 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5601066d06e0 con 0x5601067ed940 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5601067f06e0 con 0x5601067e2760 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.796+0000 7f64e9821740 10 client.0 _ll_register_callbacks cb 0x5601066f4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.797+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 0x5601067ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.797+0000 7f64e49e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.797+0000 7f64e49e3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:36914/0 (socket says 172.21.15.19:36914) 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.797+0000 7f64e49e3640 1 -- 172.21.15.19:0/3766671155 learned_addr learned my addr 172.21.15.19:0/3766671155 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e59e5640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067eab70 0x5601067eafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e51e4640 1 -- 172.21.15.19:0/3766671155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 msgr2=0x5601067ea5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e51e4640 1 -- 172.21.15.19:0/3766671155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067eab70 msgr2=0x5601067eafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067eab70 0x5601067eafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.798+0000 7f64e51e4640 1 -- 172.21.15.19:0/3766671155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601067f9cb0 con 0x5601067e2760 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.799+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 0x5601067ed330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5601067dfb50 tx=0x7f64dc00a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4e4d3bde2ad8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.799+0000 7f64c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.800+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc00f650 con 0x5601067e2760 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.800+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601067559d0 con 0x5601067e2760 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.800+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f64dc00f7f0 con 0x5601067e2760 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.801+0000 7f64e59e5640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 0x7f64cc0a53f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011fd0 tx=0x7f64d80133c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.801+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f64dc00f9f0 con 0x5601067e2760 2024-09-12T03:25:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.802+0000 7f64e9821740 10 client.26014 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.802+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5601067fc6f0 con 0x5601067e2760 2024-09-12T03:25:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.803+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5601067559d0 con 0x5601067e2760 2024-09-12T03:25:36.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.803+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f64dc020420 con 0x5601067e2760 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 10 client.26014 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 10 client.26014 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 20 client.26014 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5601067fdef0 0x560106800330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 4 client.26014 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:36.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.805+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56010674fd20 con 0x5601067fdef0 2024-09-12T03:25:36.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.808+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5601067fdef0 0x560106800330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.809+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5601067fdef0 0x560106800330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2121bb6456a4fc30 server_cookie=10216478c011482d in_seq=0 out_seq=0 2024-09-12T03:25:36.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.810+0000 7f64c8ff9640 10 client.26014 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.813+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3766671155 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56010674fd20 con 0x5601067fdef0 2024-09-12T03:25:36.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.813+0000 7f64c8ff9640 10 client.26014 handle_command_reply: tid=0 2024-09-12T03:25:36.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.813+0000 7f64e9821740 1 client.26014 shutdown 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.813+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 msgr2=0x5601067ed330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.813+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 0x5601067ed330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x5601067dfb50 tx=0x7f64dc00a730 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 shutdown_connections 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067eab70 0x5601067eafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5601067fdef0 0x560106800330 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 --2- 172.21.15.19:0/3766671155 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067e2760 0x5601067ed330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.814+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 >> 172.21.15.19:0/3766671155 conn(0x5601067decf0 msgr2=0x5601067e4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.815+0000 7f64c8ff9640 0 client.26014 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.818+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 shutdown_connections 2024-09-12T03:25:36.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.818+0000 7f64e9821740 1 -- 172.21.15.19:0/3766671155 wait complete. 2024-09-12T03:25:36.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.821+0000 7f64e9821740 20 client.26014 trim_cache size 0 max 16384 2024-09-12T03:25:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.876+0000 7f64e9821740 1 Processor -- start 2024-09-12T03:25:36.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.878+0000 7f64e9821740 1 -- start start 2024-09-12T03:25:36.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.878+0000 7f64e9821740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e9821740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067eab70 0x5601067e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e9821740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57224/0 (socket says 172.21.15.19:57224) 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e51e4640 1 -- 172.21.15.19:0/2979151859 learned_addr learned my addr 172.21.15.19:0/2979151859 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64c400e630 con 0x5601067e2760 2024-09-12T03:25:36.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64c40102f0 con 0x5601067ed940 2024-09-12T03:25:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.879+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64c400ed00 con 0x5601067eab70 2024-09-12T03:25:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e49e3640 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067eab70 0x5601067e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e59e5640 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e51e4640 1 -- 172.21.15.19:0/2979151859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067eab70 msgr2=0x5601067e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e51e4640 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067eab70 0x5601067e9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e51e4640 1 -- 172.21.15.19:0/2979151859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 msgr2=0x5601067ea3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e51e4640 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.880+0000 7f64e51e4640 1 -- 172.21.15.19:0/2979151859 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601067eb320 con 0x5601067e2760 2024-09-12T03:25:36.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.882+0000 7f64e51e4640 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f64dc020990 tx=0x7f64dc005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c58398b56f46a25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.883+0000 7f64caffd640 1 -- 172.21.15.19:0/2979151859 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc020420 con 0x5601067e2760 2024-09-12T03:25:36.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.883+0000 7f64caffd640 1 -- 172.21.15.19:0/2979151859 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64dc025070 con 0x5601067e2760 2024-09-12T03:25:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.884+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 msgr2=0x5601067de660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:36.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.884+0000 7f64e9821740 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f64dc020990 tx=0x7f64dc005a40 comp rx=0 tx=0).stop 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.885+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 shutdown_connections 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.885+0000 7f64e9821740 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067ed940 0x5601067ea3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.885+0000 7f64e9821740 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067eab70 0x5601067e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.885+0000 7f64e9821740 1 --2- 172.21.15.19:0/2979151859 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067e2760 0x5601067de660 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.885+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 >> 172.21.15.19:0/2979151859 conn(0x5601067decf0 msgr2=0x5601067e58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.886+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 shutdown_connections 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.886+0000 7f64e9821740 1 -- 172.21.15.19:0/2979151859 wait complete. 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.887+0000 7f64e9821740 1 -- start start 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.887+0000 7f64e9821740 1 Processor -- start 2024-09-12T03:25:36.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.888+0000 7f64e9821740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.888+0000 7f64e9821740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067ed940 0x5601067ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.889+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.889+0000 7f64e51e4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:36940/0 (socket says 172.21.15.19:36940) 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.889+0000 7f64e51e4640 1 -- 172.21.15.19:0/3056053728 learned_addr learned my addr 172.21.15.19:0/3056053728 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.889+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ee800 0x5601067f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.889+0000 7f64e59e5640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ee800 0x5601067f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f64c400fb60 con 0x5601067ed940 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f64c400f490 con 0x5601067eab70 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f64c400c550 con 0x5601067ee800 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e9821740 10 client.0 _ll_register_callbacks cb 0x7f64dc005b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:36.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e51e4640 1 -- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ee800 msgr2=0x5601067f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ee800 0x5601067f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e51e4640 1 -- 172.21.15.19:0/3056053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067ed940 msgr2=0x5601067ee2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067ed940 0x5601067ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.890+0000 7f64e51e4640 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601067f50a0 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.894+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5601066cbec0 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.894+0000 7f64e51e4640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64dc020650 tx=0x7f64dc00aeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d78b41ae90e92bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.895+0000 7f64c8ff9640 10 client.26021 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.895+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3056053728 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc043060 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.895+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3056053728 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f64dc00f660 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.895+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3056053728 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f64dc03a030 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.896+0000 7f64e9821740 10 client.26021 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.896+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=36}) -- 0x5601067f6280 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.896+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3056053728 <== mon.2 v2:172.21.15.121:3300/0 4 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f64dc035030 con 0x5601067eab70 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 10 client.26021 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:36.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 10 client.26021 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 20 client.26021 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560106868cd0 0x56010686b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 4 client.26021 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}] 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.1, "await_for": 45.0}) -- 0x5601066cbec0 con 0x560106868cd0 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.897+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560106868cd0 0x56010686b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.898+0000 7f64e49e3640 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560106868cd0 0x56010686b110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e56e05b1a2ad13 server_cookie=4ada7ddbf19148f9 in_seq=0 out_seq=0 2024-09-12T03:25:36.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:36.899+0000 7f64c8ff9640 10 client.26021 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:37 smithi019 ceph-mon[30953]: pgmap v534: 97 pgs: 97 active+clean; 2.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-09-12T03:25:37.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:37 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:37 smithi135 ceph-mon[35474]: pgmap v534: 97 pgs: 97 active+clean; 2.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-09-12T03:25:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:37 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:37 smithi121 ceph-mon[34963]: pgmap v534: 97 pgs: 97 active+clean; 2.2 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 572 op/s 2024-09-12T03:25:37.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:37 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:38.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:25:38.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:25:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:25:38.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:25:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:39 smithi135 ceph-mon[35474]: pgmap v535: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:39 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-12T03:25:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:39 smithi135 ceph-mon[35474]: Unable to set osd_memory_target on smithi019 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-12T03:25:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:39 smithi121 ceph-mon[34963]: pgmap v535: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:39 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-12T03:25:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:39 smithi121 ceph-mon[34963]: Unable to set osd_memory_target on smithi019 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-12T03:25:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:39 smithi019 ceph-mon[30953]: pgmap v535: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 16 MiB/s wr, 761 op/s 2024-09-12T03:25:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:39 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi019 to 177.0M 2024-09-12T03:25:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:39 smithi019 ceph-mon[30953]: Unable to set osd_memory_target on smithi019 to 185687756: error parsing value: Value '185687756' is below minimum 939524096 2024-09-12T03:25:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:41 smithi135 ceph-mon[35474]: pgmap v536: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 899 op/s 2024-09-12T03:25:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:41 smithi121 ceph-mon[34963]: pgmap v536: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 899 op/s 2024-09-12T03:25:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:41 smithi019 ceph-mon[30953]: pgmap v536: 97 pgs: 97 active+clean; 2.2 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 19 MiB/s wr, 899 op/s 2024-09-12T03:25:43.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:43 smithi135 ceph-mon[35474]: pgmap v537: 97 pgs: 97 active+clean; 2.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 896 op/s 2024-09-12T03:25:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:43 smithi121 ceph-mon[34963]: pgmap v537: 97 pgs: 97 active+clean; 2.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 896 op/s 2024-09-12T03:25:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:43 smithi019 ceph-mon[30953]: pgmap v537: 97 pgs: 97 active+clean; 2.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 19 MiB/s wr, 896 op/s 2024-09-12T03:25:45.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:45 smithi135 ceph-mon[35474]: pgmap v538: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-12T03:25:45.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:45 smithi121 ceph-mon[34963]: pgmap v538: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-12T03:25:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:45 smithi019 ceph-mon[30953]: pgmap v538: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 18 MiB/s wr, 901 op/s 2024-09-12T03:25:45.716 INFO:tasks.workunit.client.0.smithi019.stdout: 607 228950 67 158509 131 53631 169 2024-09-12T03:25:46.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:25:46.938 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:47.278 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:47 smithi019 ceph-mon[30953]: pgmap v539: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 826 op/s 2024-09-12T03:25:47.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:47 smithi135 ceph-mon[35474]: pgmap v539: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 826 op/s 2024-09-12T03:25:47.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:47 smithi121 ceph-mon[34963]: pgmap v539: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 16 MiB/s wr, 826 op/s 2024-09-12T03:25:47.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 -- 172.21.15.19:0/4019009873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 msgr2=0x7f6dcc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 --2- 172.21.15.19:0/4019009873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc100ed0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009930 tx=0x7f6dc002a420 comp rx=0 tx=0).stop 2024-09-12T03:25:47.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 -- 172.21.15.19:0/4019009873 shutdown_connections 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 --2- 172.21.15.19:0/4019009873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 --2- 172.21.15.19:0/4019009873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc100ed0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 --2- 172.21.15.19:0/4019009873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 0x7f6dcc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 -- 172.21.15.19:0/4019009873 >> 172.21.15.19:0/4019009873 conn(0x7f6dcc0fc7d0 msgr2=0x7f6dcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 -- 172.21.15.19:0/4019009873 shutdown_connections 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.554+0000 7f6dd2959640 1 -- 172.21.15.19:0/4019009873 wait complete. 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 Processor -- start 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 -- start start 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 0x7f6dcc19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc110ec0 con 0x7f6dcc100a70 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc110d40 con 0x7f6dcc101410 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dd2959640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc111040 con 0x7f6dcc106a70 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:35934/0 (socket says 172.21.15.19:35934) 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.555+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3086216076 learned_addr learned my addr 172.21.15.19:0/3086216076 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 0x7f6dcc19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3086216076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 msgr2=0x7f6dcc19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 0x7f6dcc19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 msgr2=0x7f6dcc198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3086216076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc19d430 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dc002a940 tx=0x7f6dc0003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dc97fa640 1 -- 172.21.15.19:0/3086216076 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc003f070 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dc97fa640 1 -- 172.21.15.19:0/3086216076 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0031c90 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc19d6c0 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dc97fa640 1 -- 172.21.15.19:0/3086216076 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0035030 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.556+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc1a5000 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.559+0000 7f6dc97fa640 1 -- 172.21.15.19:0/3086216076 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6dc0042070 con 0x7f6dcc101410 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.559+0000 7f6dd4d32740 1 Processor -- start 2024-09-12T03:25:47.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.559+0000 7f6dd4d32740 1 -- start start 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 0x55a5b5630a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5b55892e0 con 0x55a5b562e680 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a5b5530ce0 con 0x55a5b56234a0 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5b55852a0 con 0x55a5b5623dc0 2024-09-12T03:25:47.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:35944/0 (socket says 172.21.15.19:35944) 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1553487785 learned_addr learned my addr 172.21.15.19:0/1553487785 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 0x55a5b5630a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1553487785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 msgr2=0x55a5b562e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1553487785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 msgr2=0x55a5b5630a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 0x55a5b5630a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1553487785 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b5515440 con 0x55a5b56234a0 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 0x55a5b5630a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:47.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dc97fa640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 0x7f6db40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8a9929b871316e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.560+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 0x7f6db40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dc97fa640 1 -- 172.21.15.19:0/3086216076 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6dc00ef690 con 0x7f6dcc101410 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6db27fc640 1 -- 172.21.15.19:0/1553487785 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4004d20 con 0x55a5b56234a0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6db27fc640 1 -- 172.21.15.19:0/1553487785 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc4004050 con 0x55a5b56234a0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6db27fc640 1 -- 172.21.15.19:0/1553487785 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4005220 con 0x55a5b56234a0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1553487785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 msgr2=0x55a5b5623880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 0x7f6db40a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009fd0 tx=0x7f6dc40118d0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1553487785 shutdown_connections 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b562e680 0x55a5b5630a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1553487785 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b56234a0 0x55a5b5623880 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1553487785 >> 172.21.15.19:0/1553487785 conn(0x55a5b561fa30 msgr2=0x55a5b561fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1553487785 shutdown_connections 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.561+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1553487785 wait complete. 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 -- start start 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 Processor -- start 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b562df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5b56356b0 con 0x55a5b56234a0 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a5b5544050 con 0x55a5b562e680 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5b5635830 con 0x55a5b5623dc0 2024-09-12T03:25:47.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dd4d32740 10 client.0 _ll_register_callbacks cb 0x55a5b5584160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.562+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b562df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:35958/0 (socket says 172.21.15.19:35958) 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:38460/0 (socket says 172.21.15.19:38460) 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b562df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40636/0 (socket says 172.21.15.19:40636) 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1519474152 learned_addr learned my addr 172.21.15.19:0/1519474152 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 msgr2=0x55a5b562b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1519474152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 msgr2=0x55a5b562df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b562df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 -- 172.21.15.19:0/1519474152 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b52fa3c0 con 0x55a5b562e680 2024-09-12T03:25:47.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401b040 tx=0x7f6dc4004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=890572e665618239 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4028020 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc4004050 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6dc4004410 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6dc4004e60 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd4d32740 10 client.26033 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.563+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a5b5641020 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc4004050 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f6dc400f570 con 0x55a5b562e680 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 10 client.26033 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:47.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 10 client.26033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 20 client.26033 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b5642430 0x55a5b5644820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 4 client.26033 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:47.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.564+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5b5596850 con 0x55a5b5642430 2024-09-12T03:25:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.576+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b5642430 0x55a5b5644820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.576+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b5642430 0x55a5b5644820 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=927503a036b08fab server_cookie=e8c23413a8ccc906 in_seq=0 out_seq=0 2024-09-12T03:25:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.576+0000 7f6dab7fe640 10 client.26033 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dab7fe640 1 -- 172.21.15.19:0/1519474152 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5b5596850 con 0x55a5b5642430 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dab7fe640 10 client.26033 handle_command_reply: tid=0 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 client.26033 shutdown 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 msgr2=0x55a5b562b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6dc401b040 tx=0x7f6dc4004c70 comp rx=0 tx=0).stop 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 shutdown_connections 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b5642430 0x55a5b5644820 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b562e680 0x55a5b562b750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b562b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dab7fe640 0 client.26033 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/1519474152 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b562df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 >> 172.21.15.19:0/1519474152 conn(0x55a5b561fa30 msgr2=0x55a5b5625710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 shutdown_connections 2024-09-12T03:25:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 1 -- 172.21.15.19:0/1519474152 wait complete. 2024-09-12T03:25:47.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.602+0000 7f6dd4d32740 20 client.26033 trim_cache size 0 max 16384 2024-09-12T03:25:47.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 Processor -- start 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 -- start start 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b561fa30 0x55a5b562d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b564a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ab470 con 0x55a5b56234a0 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ad130 con 0x55a5b561fa30 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1abb40 con 0x55a5b5623dc0 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40646/0 (socket says 172.21.15.19:40646) 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3983378288 learned_addr learned my addr 172.21.15.19:0/3983378288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b561fa30 0x55a5b562d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b564a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3983378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 msgr2=0x55a5b564a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b564a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3983378288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b561fa30 msgr2=0x55a5b562d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b561fa30 0x55a5b562d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dcb7fe640 1 -- 172.21.15.19:0/3983378288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5b5620ae0 con 0x55a5b56234a0 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.621+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b564a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:47.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.622+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00ed140 tx=0x7f6dc0003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15f33077ca56064 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.622+0000 7f6db1ffb640 1 -- 172.21.15.19:0/3983378288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0049070 con 0x55a5b56234a0 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.622+0000 7f6db1ffb640 1 -- 172.21.15.19:0/3983378288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dc0031e40 con 0x55a5b56234a0 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.622+0000 7f6db1ffb640 1 -- 172.21.15.19:0/3983378288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00b85c0 con 0x55a5b56234a0 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3983378288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 msgr2=0x55a5b564a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00ed140 tx=0x7f6dc0003f50 comp rx=0 tx=0).stop 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3983378288 shutdown_connections 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b5623dc0 0x55a5b564a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b56234a0 0x55a5b564a020 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3983378288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b561fa30 0x55a5b562d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3983378288 >> 172.21.15.19:0/3983378288 conn(0x55a5b562b8b0 msgr2=0x55a5b562ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3983378288 shutdown_connections 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3983378288 wait complete. 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 -- start start 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 Processor -- start 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b56234a0 0x55a5b56382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.623+0000 7f6dd4d32740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 0x55a5b56387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ac9a0 con 0x55a5b561fa30 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1ac2d0 con 0x55a5b5623dc0 2024-09-12T03:25:47.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd4d32740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc1a93b0 con 0x55a5b56234a0 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40658/0 (socket says 172.21.15.19:40658) 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 -- 172.21.15.19:0/3872424722 learned_addr learned my addr 172.21.15.19:0/3872424722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b56234a0 0x55a5b56382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd4d32740 10 client.0 _ll_register_callbacks cb 0x7f6dbc009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 0x55a5b56387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 -- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b56234a0 msgr2=0x55a5b56382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b56234a0 0x55a5b56382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 -- 172.21.15.19:0/3872424722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 msgr2=0x55a5b56387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 0x55a5b56387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 -- 172.21.15.19:0/3872424722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5b56206c0 con 0x55a5b561fa30 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd0ef6640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 0x55a5b56387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:47.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dcbfff640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55a5b5644f90 tx=0x7f6dbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c073ee91749a7ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00fb50 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5b5590d60 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.624+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6dbc00fcf0 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6dbc0035f0 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 10 client.16539 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a5b56311b0 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a5b5590d60 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f6dbc02b540 con 0x55a5b561fa30 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 10 client.16539 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 20 client.16539 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b562e680 0x55a5b5633950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:47.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 4 client.16539 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:25:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a5b550d210 con 0x55a5b562e680 2024-09-12T03:25:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.625+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b562e680 0x55a5b5633950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.626+0000 7f6dcb7fe640 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b562e680 0x55a5b5633950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b700b453427da38b server_cookie=955d9c3694017002 in_seq=0 out_seq=0 2024-09-12T03:25:47.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.626+0000 7f6dab7fe640 10 client.16539 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.743+0000 7f6dab7fe640 1 -- 172.21.15.19:0/3872424722 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a5b550d210 con 0x55a5b562e680 2024-09-12T03:25:47.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.743+0000 7f6dab7fe640 10 client.16539 handle_command_reply: tid=0 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.743+0000 7f6dd4d32740 1 client.16539 shutdown 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 msgr2=0x55a5b563af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x55a5b5644f90 tx=0x7f6dbc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 shutdown_connections 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5b562e680 0x55a5b5633950 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5b5623dc0 0x55a5b56387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5b56234a0 0x55a5b56382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dab7fe640 0 client.16539 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 --2- 172.21.15.19:0/3872424722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5b561fa30 0x55a5b563af60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 >> 172.21.15.19:0/3872424722 conn(0x55a5b562b8b0 msgr2=0x55a5b56296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 shutdown_connections 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 1 -- 172.21.15.19:0/3872424722 wait complete. 2024-09-12T03:25:47.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd4d32740 20 client.16539 trim_cache size 0 max 16384 2024-09-12T03:25:47.747 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:25:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:25:47.747 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:25:47.747 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 msgr2=0x7f6db40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.744+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 0x7f6db40a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc009fd0 tx=0x7f6dbc005f90 comp rx=0 tx=0).stop 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 msgr2=0x7f6dcc1989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6dc002a940 tx=0x7f6dc0003b20 comp rx=0 tx=0).stop 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 shutdown_connections 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6dcc106a70 0x7f6dcc19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc101410 0x7f6dcc1989c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6db40a2b80 0x7f6db40a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 --2- 172.21.15.19:0/3086216076 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6dcc100a70 0x7f6dcc198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 >> 172.21.15.19:0/3086216076 conn(0x7f6dcc0fc7d0 msgr2=0x7f6dcc104880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 shutdown_connections 2024-09-12T03:25:47.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:47.745+0000 7f6dd2959640 1 -- 172.21.15.19:0/3086216076 wait complete. 2024-09-12T03:25:49.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:49 smithi135 ceph-mon[35474]: pgmap v540: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-09-12T03:25:49.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:49 smithi121 ceph-mon[34963]: pgmap v540: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-09-12T03:25:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:49 smithi019 ceph-mon[30953]: pgmap v540: 97 pgs: 97 active+clean; 2.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 739 op/s 2024-09-12T03:25:49.586 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:25:49.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:25:49.856 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:50.388 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 -- 172.21.15.19:0/278389894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 msgr2=0x7f7900100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 --2- 172.21.15.19:0/278389894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f7900100ee0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009930 tx=0x7f78f402a420 comp rx=0 tx=0).stop 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 -- 172.21.15.19:0/278389894 shutdown_connections 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 --2- 172.21.15.19:0/278389894 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 0x7f790010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 --2- 172.21.15.19:0/278389894 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f7900100ee0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 --2- 172.21.15.19:0/278389894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 0x7f7900106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 -- 172.21.15.19:0/278389894 >> 172.21.15.19:0/278389894 conn(0x7f79000fc7a0 msgr2=0x7f79000febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 -- 172.21.15.19:0/278389894 shutdown_connections 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.368+0000 7f7906408640 1 -- 172.21.15.19:0/278389894 wait complete. 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 Processor -- start 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 -- start start 2024-09-12T03:25:50.389 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 0x7f79000fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 0x7f79000fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7900110e80 con 0x7f7900100a80 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7900110d00 con 0x7f7900106a80 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.369+0000 7f7906408640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7900111000 con 0x7f7900101420 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40678/0 (socket says 172.21.15.19:40678) 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 -- 172.21.15.19:0/2138159119 learned_addr learned my addr 172.21.15.19:0/2138159119 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78ff7fe640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 0x7f79000fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f79049a5640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 0x7f79000fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 -- 172.21.15.19:0/2138159119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 msgr2=0x7f79000fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 0x7f79000fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 -- 172.21.15.19:0/2138159119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 msgr2=0x7f79000fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 0x7f79000fd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 -- 172.21.15.19:0/2138159119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79001aae10 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fffff640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d950 tx=0x7f78f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f0017030 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78f0004020 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79001aafe0 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.370+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f00047a0 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.371+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79001ab470 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.372+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79001021c0 con 0x7f7900100a80 2024-09-12T03:25:50.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.373+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78f00041d0 con 0x7f7900100a80 2024-09-12T03:25:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.374+0000 7f78fd7fa640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 0x7f78e80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.374+0000 7f78ff7fe640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 0x7f78e80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.374+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f78f00ce8e0 con 0x7f7900100a80 2024-09-12T03:25:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.374+0000 7f78ff7fe640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 0x7f78e80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f790019c5a0 tx=0x7f78f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:50.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.378+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78f009b850 con 0x7f7900100a80 2024-09-12T03:25:50.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.596+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f790002d7c0 con 0x7f7900100a80 2024-09-12T03:25:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.597+0000 7f78fd7fa640 1 -- 172.21.15.19:0/2138159119 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12002 (secure 0 0 0) 0x7f78f009b150 con 0x7f7900100a80 2024-09-12T03:25:50.598 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:25:50.598 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"btime":"2024-09-12T03:24:43:270632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:24:43.042444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:25:50.598 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 msgr2=0x7f78e80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 0x7f78e80a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f790019c5a0 tx=0x7f78f40058d0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 msgr2=0x7f79000fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78f000d950 tx=0x7f78f000de10 comp rx=0 tx=0).stop 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 shutdown_connections 2024-09-12T03:25:50.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7900106a80 0x7f79000fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7900101420 0x7f79000fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f78e80a2c00 0x7f78e80a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 --2- 172.21.15.19:0/2138159119 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7900100a80 0x7f79000fcd90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 >> 172.21.15.19:0/2138159119 conn(0x7f79000fc7a0 msgr2=0x7f790006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 shutdown_connections 2024-09-12T03:25:50.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:50.599+0000 7f7906408640 1 -- 172.21.15.19:0/2138159119 wait complete. 2024-09-12T03:25:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:51 smithi135 ceph-mon[35474]: pgmap v541: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 591 op/s 2024-09-12T03:25:51.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:51 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2138159119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:51.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:51 smithi121 ceph-mon[34963]: pgmap v541: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 591 op/s 2024-09-12T03:25:51.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:51 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2138159119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:51 smithi019 ceph-mon[30953]: pgmap v541: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 9.0 MiB/s wr, 591 op/s 2024-09-12T03:25:51.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:51 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2138159119' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:25:51.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:25:51.952 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 -- 172.21.15.19:0/871570410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8106f50 msgr2=0x7f7ce810eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/871570410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8106f50 0x7f7ce810eaf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009930 tx=0x7f7ce002a420 comp rx=0 tx=0).stop 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 -- 172.21.15.19:0/871570410 shutdown_connections 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/871570410 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8106f50 0x7f7ce810eaf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/871570410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce8100ca0 0x7f7ce8106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/871570410 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce81002f0 0x7f7ce81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 -- 172.21.15.19:0/871570410 >> 172.21.15.19:0/871570410 conn(0x7f7ce8077210 msgr2=0x7f7ce80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 -- 172.21.15.19:0/871570410 shutdown_connections 2024-09-12T03:25:52.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.443+0000 7f7cef9f7640 1 -- 172.21.15.19:0/871570410 wait complete. 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 Processor -- start 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 -- start start 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce81002f0 0x7f7ce8101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8106f50 0x7f7ce8101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8111fa0 con 0x7f7ce8100ca0 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8111e20 con 0x7f7ce81002f0 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cef9f7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce8112120 con 0x7f7ce8106f50 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38778/0 (socket says 172.21.15.19:38778) 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cecf92640 1 -- 172.21.15.19:0/1376189185 learned_addr learned my addr 172.21.15.19:0/1376189185 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7ced793640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce81002f0 0x7f7ce8101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.444+0000 7f7cedf94640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8106f50 0x7f7ce8101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 -- 172.21.15.19:0/1376189185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8106f50 msgr2=0x7f7ce8101cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8106f50 0x7f7ce8101cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 -- 172.21.15.19:0/1376189185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce81002f0 msgr2=0x7f7ce8101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce81002f0 0x7f7ce8101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 -- 172.21.15.19:0/1376189185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81a6920 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cecf92640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9c0 tx=0x7f7cd800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cde7fc640 1 -- 172.21.15.19:0/1376189185 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd800cd80 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cde7fc640 1 -- 172.21.15.19:0/1376189185 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7cd8005500 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cde7fc640 1 -- 172.21.15.19:0/1376189185 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8010690 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce81a6c10 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.445+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce81a7100 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.447+0000 7f7cde7fc640 1 -- 172.21.15.19:0/1376189185 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7cd80046c0 con 0x7f7ce8100ca0 2024-09-12T03:25:52.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 Processor -- start 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 -- start start 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3d4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b33a2f0 con 0x55ab4b3df370 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b2e1cf0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b3362b0 con 0x55ab4b3d45d0 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cedf94640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cedf94640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:43978/0 (socket says 172.21.15.19:43978) 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cedf94640 1 -- 172.21.15.19:0/902517575 learned_addr learned my addr 172.21.15.19:0/902517575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7ced793640 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3d4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.448+0000 7f7cde7fc640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 0x7f7cd40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 -- 172.21.15.19:0/902517575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 msgr2=0x55ab4b3d4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3d4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 -- 172.21.15.19:0/902517575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 msgr2=0x55ab4b3e1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3e1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 -- 172.21.15.19:0/902517575 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab4b2c63a0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cde7fc640 1 -- 172.21.15.19:0/1376189185 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7cd80cf320 con 0x7f7ce8100ca0 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 0x7f7cd40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009930 tx=0x7f7ce00315b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ace1b50d7900e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cd37fe640 1 -- 172.21.15.19:0/902517575 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035040 con 0x55ab4b3d5b40 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cd37fe640 1 -- 172.21.15.19:0/902517575 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce00453d0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cedf94640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 0x7f7cd40a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ce004a000 tx=0x7f7ce0047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.449+0000 7f7cd37fe640 1 -- 172.21.15.19:0/902517575 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce003b630 con 0x55ab4b3d5b40 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/902517575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 msgr2=0x55ab4b3d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009930 tx=0x7f7ce00315b0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/902517575 shutdown_connections 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3e1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3d4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/902517575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3d4090 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/902517575 >> 172.21.15.19:0/902517575 conn(0x55ab4b39d310 msgr2=0x55ab4b3d0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/902517575 shutdown_connections 2024-09-12T03:25:52.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/902517575 wait complete. 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 -- start start 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.450+0000 7f7cf1dd0740 1 Processor -- start 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3dc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b3e2070 con 0x55ab4b3d5b40 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b2f5060 con 0x55ab4b3df370 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ab4b3e21f0 con 0x55ab4b3d45d0 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cf1dd0740 10 client.0 _ll_register_callbacks cb 0x55ab4b0488c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7ced793640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cedf94640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3dc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7ced793640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:36370/0 (socket says 172.21.15.19:36370) 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7ced793640 1 -- 172.21.15.19:0/3292832077 learned_addr learned my addr 172.21.15.19:0/3292832077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38792/0 (socket says 172.21.15.19:38792) 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 -- 172.21.15.19:0/3292832077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 msgr2=0x55ab4b3dedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 -- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 msgr2=0x55ab4b3dc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3dc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7cecf92640 1 -- 172.21.15.19:0/3292832077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab4b0ab3c0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.451+0000 7f7ced793640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:52.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7cd809b830 tx=0x7f7cd8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f083b338ee7405 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd809ce60 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7cd80a4510 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce00453d0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7cd809b490 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 10 client.16557 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab4b3edc40 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce00453d0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f7cd80d2cd0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 10 client.16557 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 10 client.16557 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 20 client.16557 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:52.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b3ef100 0x55ab4b3f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 4 client.16557 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.452+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab4b347860 con 0x55ab4b3ef100 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.453+0000 7f7ced793640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b3ef100 0x55ab4b3f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.453+0000 7f7cd0ff9640 10 client.16557 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.453+0000 7f7ced793640 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b3ef100 0x55ab4b3f14f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23982024a26c9fdc server_cookie=3273464d924cb1f3 in_seq=0 out_seq=0 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.455+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3292832077 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab4b347860 con 0x55ab4b3ef100 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.455+0000 7f7cd0ff9640 10 client.16557 handle_command_reply: tid=0 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.455+0000 7f7cf1dd0740 1 client.16557 shutdown 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.455+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 msgr2=0x55ab4b3dc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.455+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7cd809b830 tx=0x7f7cd8005f10 comp rx=0 tx=0).stop 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 shutdown_connections 2024-09-12T03:25:52.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3dc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3dc060 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b3ef100 0x55ab4b3f14f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3292832077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3dedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cd0ff9640 0 client.16557 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 >> 172.21.15.19:0/3292832077 conn(0x55ab4b39d310 msgr2=0x55ab4b3d6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 shutdown_connections 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3292832077 wait complete. 2024-09-12T03:25:52.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.456+0000 7f7cf1dd0740 20 client.16557 trim_cache size 0 max 16384 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 Processor -- start 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 -- start start 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3db920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3da930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81af910 con 0x55ab4b3df370 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81b15d0 con 0x55ab4b3d5b40 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81affe0 con 0x55ab4b3d45d0 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cedf94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cedf94640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38804/0 (socket says 172.21.15.19:38804) 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cecf92640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3da930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7cedf94640 1 -- 172.21.15.19:0/925313951 learned_addr learned my addr 172.21.15.19:0/925313951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.469+0000 7f7ced793640 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3db920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 -- 172.21.15.19:0/925313951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 msgr2=0x55ab4b3db920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3db920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 -- 172.21.15.19:0/925313951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 msgr2=0x55ab4b3da930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3da930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 -- 172.21.15.19:0/925313951 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab4b3dcb60 con 0x55ab4b3df370 2024-09-12T03:25:52.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7ced793640 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cedf94640 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009930 tx=0x7f7ce0045ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8174af56a17642d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cd2ffd640 1 -- 172.21.15.19:0/925313951 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0058070 con 0x55ab4b3df370 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cd2ffd640 1 -- 172.21.15.19:0/925313951 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce0045d10 con 0x55ab4b3df370 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.470+0000 7f7cd2ffd640 1 -- 172.21.15.19:0/925313951 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0035070 con 0x55ab4b3df370 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/925313951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 msgr2=0x55ab4b3dae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009930 tx=0x7f7ce0045ca0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/925313951 shutdown_connections 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3df370 0x55ab4b3dae70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3da930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/925313951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3db920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/925313951 >> 172.21.15.19:0/925313951 conn(0x55ab4b39d310 msgr2=0x55ab4b3d72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/925313951 shutdown_connections 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/925313951 wait complete. 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 -- start start 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.471+0000 7f7cf1dd0740 1 Processor -- start 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3e2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81b0e40 con 0x55ab4b3d45d0 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81b0770 con 0x55ab4b3df370 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce81ad830 con 0x55ab4b3d5b40 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cf1dd0740 10 client.0 _ll_register_callbacks cb 0x7f7cd8009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38820/0 (socket says 172.21.15.19:38820) 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 -- 172.21.15.19:0/3314173419 learned_addr learned my addr 172.21.15.19:0/3314173419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cedf94640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3e2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 -- 172.21.15.19:0/3314173419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 msgr2=0x55ab4b3e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 -- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 msgr2=0x55ab4b3e2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3e2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 -- 172.21.15.19:0/3314173419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab4b457f50 con 0x55ab4b3d45d0 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7ced793640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40139e0 tx=0x7f7ce4013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc69d3873e0cec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.472+0000 7f7cd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:52.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4003970 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7ce4003b10 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce0045d10 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ce4003d10 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 10 client.16563 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ab4b459290 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ce0045d10 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f7ce400f040 con 0x55ab4b3d45d0 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 10 client.16563 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 10 client.16563 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 20 client.16563 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b459a80 0x55ab4b45bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 4 client.16563 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:52.487 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.473+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab4b341d70 con 0x55ab4b459a80 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.474+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b459a80 0x55ab4b45bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.474+0000 7f7cecf92640 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b459a80 0x55ab4b45bea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=526890cb5454a0c server_cookie=4f2fb031caf3dc47 in_seq=0 out_seq=0 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.474+0000 7f7cd0ff9640 10 client.16563 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cd0ff9640 1 -- 172.21.15.19:0/3314173419 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab4b341d70 con 0x55ab4b459a80 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cd0ff9640 10 client.16563 handle_command_reply: tid=0 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 client.16563 shutdown 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 msgr2=0x55ab4b3e2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40139e0 tx=0x7f7ce4013f90 comp rx=0 tx=0).stop 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 shutdown_connections 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ab4b3df370 0x55ab4b3e2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55ab4b459a80 0x55ab4b45bea0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ab4b3d5b40 0x55ab4b3e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 --2- 172.21.15.19:0/3314173419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ab4b3d45d0 0x55ab4b3e2470 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 >> 172.21.15.19:0/3314173419 conn(0x55ab4b39d310 msgr2=0x55ab4b3dad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cd0ff9640 0 client.16563 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:25:52.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 shutdown_connections 2024-09-12T03:25:52.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 1 -- 172.21.15.19:0/3314173419 wait complete. 2024-09-12T03:25:52.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.477+0000 7f7cf1dd0740 20 client.16563 trim_cache size 0 max 16384 2024-09-12T03:25:52.493 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 msgr2=0x7f7cd40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 0x7f7cd40a50c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7ce004a000 tx=0x7f7ce0047000 comp rx=0 tx=0).stop 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 msgr2=0x7f7ce81017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800e9c0 tx=0x7f7cd800ee80 comp rx=0 tx=0).stop 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 shutdown_connections 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7ce8106f50 0x7f7ce8101cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7cd40a2c00 0x7f7cd40a50c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7ce8100ca0 0x7f7ce81017b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 --2- 172.21.15.19:0/1376189185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ce81002f0 0x7f7ce8101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:52.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 >> 172.21.15.19:0/1376189185 conn(0x7f7ce8077210 msgr2=0x7f7ce8105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:52.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 shutdown_connections 2024-09-12T03:25:52.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:52.478+0000 7f7cef9f7640 1 -- 172.21.15.19:0/1376189185 wait complete. 2024-09-12T03:25:53.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:53 smithi135 ceph-mon[35474]: pgmap v542: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-09-12T03:25:53.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:53 smithi121 ceph-mon[34963]: pgmap v542: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-09-12T03:25:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:53 smithi019 ceph-mon[30953]: pgmap v542: 97 pgs: 97 active+clean; 2.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.3 MiB/s wr, 437 op/s 2024-09-12T03:25:53.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:25:53.738 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 -- 172.21.15.19:0/1543067281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc100e20 msgr2=0x7efddc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 --2- 172.21.15.19:0/1543067281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc100e20 0x7efddc101200 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009900 tx=0x7efdd802a480 comp rx=0 tx=0).stop 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 -- 172.21.15.19:0/1543067281 shutdown_connections 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 --2- 172.21.15.19:0/1543067281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102ef0 0x7efddc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 --2- 172.21.15.19:0/1543067281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc102570 0x7efddc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 --2- 172.21.15.19:0/1543067281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc100e20 0x7efddc101200 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 -- 172.21.15.19:0/1543067281 >> 172.21.15.19:0/1543067281 conn(0x7efddc0fc7d0 msgr2=0x7efddc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 -- 172.21.15.19:0/1543067281 shutdown_connections 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.405+0000 7efde3bd0640 1 -- 172.21.15.19:0/1543067281 wait complete. 2024-09-12T03:25:54.456 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 Processor -- start 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 -- start start 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc100e20 0x7efddc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102570 0x7efddc1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efddc110f80 con 0x7efddc102ef0 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efddc110e00 con 0x7efddc100e20 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde3bd0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efddc111100 con 0x7efddc102570 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38842/0 (socket says 172.21.15.19:38842) 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde196c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc100e20 0x7efddc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 -- 172.21.15.19:0/962005846 learned_addr learned my addr 172.21.15.19:0/962005846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:54.457 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde116b640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102570 0x7efddc1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 -- 172.21.15.19:0/962005846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102570 msgr2=0x7efddc1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102570 0x7efddc1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 -- 172.21.15.19:0/962005846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc100e20 msgr2=0x7efddc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc100e20 0x7efddc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.406+0000 7efde216d640 1 -- 172.21.15.19:0/962005846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc19d440 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efde216d640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efdd400cce0 tx=0x7efdd400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efdd2ffd640 1 -- 172.21.15.19:0/962005846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd400f450 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc19d730 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efdd2ffd640 1 -- 172.21.15.19:0/962005846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd400fad0 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efdd2ffd640 1 -- 172.21.15.19:0/962005846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd4018c10 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.407+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1a5010 con 0x7efddc102ef0 2024-09-12T03:25:54.458 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.409+0000 7efdd2ffd640 1 -- 172.21.15.19:0/962005846 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efdd400f5f0 con 0x7efddc102ef0 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.409+0000 7efde5fa9740 1 Processor -- start 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.409+0000 7efde5fa9740 1 -- start start 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562776c8a2f0 con 0x562776d2f370 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562776c31cf0 con 0x562776d25b40 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde5fa9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562776c862b0 con 0x562776d245d0 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efdd2ffd640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 0x7efdc80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde116b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde116b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38848/0 (socket says 172.21.15.19:38848) 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:36406/0 (socket says 172.21.15.19:36406) 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde116b640 1 -- 172.21.15.19:0/1123301048 learned_addr learned my addr 172.21.15.19:0/1123301048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde116b640 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde216d640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 0x7efdc80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 -- 172.21.15.19:0/1123301048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 msgr2=0x562776d24090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d24090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 -- 172.21.15.19:0/1123301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 msgr2=0x562776d31760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde196c640 1 -- 172.21.15.19:0/1123301048 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562776c163a0 con 0x562776d245d0 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efde116b640 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.410+0000 7efdd2ffd640 1 -- 172.21.15.19:0/962005846 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efdd40cf670 con 0x7efddc102ef0 2024-09-12T03:25:54.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efde196c640 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005940 tx=0x7efdd80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43afc9f08527bd0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efde216d640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 0x7efdc80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdd40a4620 tx=0x7efdd400af30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efdc7fff640 1 -- 172.21.15.19:0/1123301048 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8035030 con 0x562776d245d0 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efdc7fff640 1 -- 172.21.15.19:0/1123301048 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd8040070 con 0x562776d245d0 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efdc7fff640 1 -- 172.21.15.19:0/1123301048 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8003970 con 0x562776d245d0 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efde5fa9740 1 -- 172.21.15.19:0/1123301048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 msgr2=0x562776d24a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.411+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005940 tx=0x7efdd80040c0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 -- 172.21.15.19:0/1123301048 shutdown_connections 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d31760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d24a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1123301048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d24090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 -- 172.21.15.19:0/1123301048 >> 172.21.15.19:0/1123301048 conn(0x562776ced310 msgr2=0x562776d20c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 -- 172.21.15.19:0/1123301048 shutdown_connections 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 -- 172.21.15.19:0/1123301048 wait complete. 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 -- start start 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.412+0000 7efde5fa9740 1 Processor -- start 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d2edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d2c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562776d32010 con 0x562776d2f370 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562776c45060 con 0x562776d25b40 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562776d32190 con 0x562776d245d0 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 10 client.0 _ll_register_callbacks cb 0x5627769988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38852/0 (socket says 172.21.15.19:38852) 2024-09-12T03:25:54.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 -- 172.21.15.19:0/1994048246 learned_addr learned my addr 172.21.15.19:0/1994048246 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde196c640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d2c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d2edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 -- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 msgr2=0x562776d2edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d2edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 -- 172.21.15.19:0/1994048246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 msgr2=0x562776d2c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d2c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 -- 172.21.15.19:0/1994048246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627769fb3c0 con 0x562776d2f370 2024-09-12T03:25:54.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde196c640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d2c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde116b640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efdcc012d70 tx=0x7efdcc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d12b3df9978ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efdc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc01a040 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efdcc004230 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.413+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdd8040070 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efdcc00fe60 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 10 client.16575 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562776d3db00 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdd8040070 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7efdcc004a40 con 0x562776d2f370 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 10 client.16575 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:54.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 10 client.16575 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 20 client.16575 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776d3f060 0x562776d41450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 4 client.16575 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.414+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562776c97860 con 0x562776d3f060 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.417+0000 7efde196c640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776d3f060 0x562776d41450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.417+0000 7efdc57fa640 10 client.16575 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.417+0000 7efde196c640 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776d3f060 0x562776d41450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=21639e9120f0ae76 server_cookie=e15df9f33c15528c in_seq=0 out_seq=0 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efdc57fa640 1 -- 172.21.15.19:0/1994048246 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562776c97860 con 0x562776d3f060 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efdc57fa640 10 client.16575 handle_command_reply: tid=0 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 client.16575 shutdown 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 msgr2=0x562776d2c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efdcc012d70 tx=0x7efdcc004730 comp rx=0 tx=0).stop 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 shutdown_connections 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776d3f060 0x562776d41450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d2f370 0x562776d2c7f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d25b40 0x562776d2c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 --2- 172.21.15.19:0/1994048246 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d245d0 0x562776d2edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 >> 172.21.15.19:0/1994048246 conn(0x562776ced310 msgr2=0x562776d26960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efdc57fa640 0 client.16575 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 shutdown_connections 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.421+0000 7efde5fa9740 1 -- 172.21.15.19:0/1994048246 wait complete. 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.422+0000 7efde5fa9740 20 client.16575 trim_cache size 0 max 16384 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.434+0000 7efde5fa9740 1 Processor -- start 2024-09-12T03:25:54.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.434+0000 7efde5fa9740 1 -- start start 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d2b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d2dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1ad500 con 0x562776d25b40 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1af1c0 con 0x562776d245d0 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde5fa9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1adbd0 con 0x562776d2f370 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde196c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d2b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44040/0 (socket says 172.21.15.19:44040) 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 -- 172.21.15.19:0/3941099906 learned_addr learned my addr 172.21.15.19:0/3941099906 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:54.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d2dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 -- 172.21.15.19:0/3941099906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 msgr2=0x562776d2dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d2dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 -- 172.21.15.19:0/3941099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 msgr2=0x562776d2b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d2b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 -- 172.21.15.19:0/3941099906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562776d417c0 con 0x562776d245d0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efde116b640 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efdcc012c60 tx=0x7efdcc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c571abf44096f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.435+0000 7efdc77fe640 1 -- 172.21.15.19:0/3941099906 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc02f040 con 0x562776d245d0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efdc77fe640 1 -- 172.21.15.19:0/3941099906 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc0047a0 con 0x562776d245d0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efdc77fe640 1 -- 172.21.15.19:0/3941099906 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc005380 con 0x562776d245d0 2024-09-12T03:25:54.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- 172.21.15.19:0/3941099906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 msgr2=0x562776d25300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efdcc012c60 tx=0x7efdcc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- 172.21.15.19:0/3941099906 shutdown_connections 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d2dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d2b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 --2- 172.21.15.19:0/3941099906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d25300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- 172.21.15.19:0/3941099906 >> 172.21.15.19:0/3941099906 conn(0x562776ced310 msgr2=0x562776d2d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- 172.21.15.19:0/3941099906 shutdown_connections 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- 172.21.15.19:0/3941099906 wait complete. 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.436+0000 7efde5fa9740 1 -- start start 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 Processor -- start 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d378c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d33130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1aea30 con 0x562776d25b40 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1ae360 con 0x562776d245d0 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efddc1ab420 con 0x562776d2f370 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde5fa9740 10 client.0 _ll_register_callbacks cb 0x7efdcc009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38870/0 (socket says 172.21.15.19:38870) 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde116b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 -- 172.21.15.19:0/2691922028 learned_addr learned my addr 172.21.15.19:0/2691922028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde116b640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d33130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 -- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 msgr2=0x562776d33130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d33130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 -- 172.21.15.19:0/2691922028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 msgr2=0x562776d378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d378c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.437+0000 7efde196c640 1 -- 172.21.15.19:0/2691922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562776d33670 con 0x562776d25b40 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efde196c640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005e90 tx=0x7efdd8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73ac2e1e8e0f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdd8035040 con 0x562776d25b40 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efdd8005440 con 0x562776d25b40 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdcc0047a0 con 0x562776d25b40 2024-09-12T03:25:54.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efdd80046c0 con 0x562776d25b40 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efde5fa9740 10 client.16581 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562776da9b10 con 0x562776d25b40 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdcc0047a0 con 0x562776d25b40 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.438+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7efdd8031880 con 0x562776d25b40 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 10 client.16581 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 10 client.16581 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 20 client.16581 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776da9fc0 0x562776dac3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:54.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 4 client.16581 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.439+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562776c91d70 con 0x562776da9fc0 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.443+0000 7efde116b640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776da9fc0 0x562776dac3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.443+0000 7efde116b640 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776da9fc0 0x562776dac3e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dd37ab776ed92a72 server_cookie=778939cc46401d79 in_seq=0 out_seq=0 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.443+0000 7efdc57fa640 10 client.16581 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.445+0000 7efdc57fa640 1 -- 172.21.15.19:0/2691922028 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562776c91d70 con 0x562776da9fc0 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.445+0000 7efdc57fa640 10 client.16581 handle_command_reply: tid=0 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.445+0000 7efde5fa9740 1 client.16581 shutdown 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 msgr2=0x562776d32bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7efdd8005e90 tx=0x7efdd8031400 comp rx=0 tx=0).stop 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 shutdown_connections 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562776da9fc0 0x562776dac3e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562776d2f370 0x562776d33130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562776d25b40 0x562776d32bf0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efdc57fa640 0 client.16581 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 --2- 172.21.15.19:0/2691922028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562776d245d0 0x562776d378c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 >> 172.21.15.19:0/2691922028 conn(0x562776ced310 msgr2=0x562776d25160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 shutdown_connections 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 1 -- 172.21.15.19:0/2691922028 wait complete. 2024-09-12T03:25:54.474 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde5fa9740 20 client.16581 trim_cache size 0 max 16384 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 msgr2=0x7efdc80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 0x7efdc80a5040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efdd40a4620 tx=0x7efdd400af30 comp rx=0 tx=0).stop 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 msgr2=0x7efddc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efdd400cce0 tx=0x7efdd400a620 comp rx=0 tx=0).stop 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 shutdown_connections 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdc80a2b80 0x7efdc80a5040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efddc102ef0 0x7efddc19cd60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efddc102570 0x7efddc1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.446+0000 7efde3bd0640 1 --2- 172.21.15.19:0/962005846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efddc100e20 0x7efddc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.447+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 >> 172.21.15.19:0/962005846 conn(0x7efddc0fc7d0 msgr2=0x7efddc06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.447+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 shutdown_connections 2024-09-12T03:25:54.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:54.447+0000 7efde3bd0640 1 -- 172.21.15.19:0/962005846 wait complete. 2024-09-12T03:25:55.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:55 smithi135 ceph-mon[35474]: pgmap v543: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 412 op/s 2024-09-12T03:25:55.328 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:55 smithi019 ceph-mon[30953]: pgmap v543: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 412 op/s 2024-09-12T03:25:55.355 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:25:55.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:55 smithi121 ceph-mon[34963]: pgmap v543: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.8 MiB/s wr, 412 op/s 2024-09-12T03:25:55.618 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- 172.21.15.19:0/1602687520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102ef0 msgr2=0x7f921410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 --2- 172.21.15.19:0/1602687520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102ef0 0x7f921410da40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f920c009ca0 tx=0x7f920c02a950 comp rx=0 tx=0).stop 2024-09-12T03:25:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- 172.21.15.19:0/1602687520 shutdown_connections 2024-09-12T03:25:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 --2- 172.21.15.19:0/1602687520 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102ef0 0x7f921410da40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 --2- 172.21.15.19:0/1602687520 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102570 0x7f92141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 --2- 172.21.15.19:0/1602687520 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f9214101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- 172.21.15.19:0/1602687520 >> 172.21.15.19:0/1602687520 conn(0x7f92140fc7d0 msgr2=0x7f92140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- 172.21.15.19:0/1602687520 shutdown_connections 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- 172.21.15.19:0/1602687520 wait complete. 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 Processor -- start 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.103+0000 7f921ab7b640 1 -- start start 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102ef0 0x7f92140fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9214110fd0 con 0x7f9214102570 2024-09-12T03:25:56.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9214110e50 con 0x7f9214100e20 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f921ab7b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9214111150 con 0x7f9214102ef0 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44076/0 (socket says 172.21.15.19:44076) 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 -- 172.21.15.19:0/3470119294 learned_addr learned my addr 172.21.15.19:0/3470119294 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9213fff640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38884/0 (socket says 172.21.15.19:38884) 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9219118640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102ef0 0x7f92140fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 -- 172.21.15.19:0/3470119294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102ef0 msgr2=0x7f92140fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102ef0 0x7f92140fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 -- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 msgr2=0x7f92140fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 -- 172.21.15.19:0/3470119294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92141a6920 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9213fff640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9218917640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f920800e9f0 tx=0x7f920800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.104+0000 7f9211ffb640 1 -- 172.21.15.19:0/3470119294 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920800cd80 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.105+0000 7f9211ffb640 1 -- 172.21.15.19:0/3470119294 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9208010910 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.105+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92141a6c10 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.105+0000 7f9211ffb640 1 -- 172.21.15.19:0/3470119294 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208018990 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.105+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92141a7150 con 0x7f9214100e20 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.107+0000 7f921cf54740 1 Processor -- start 2024-09-12T03:25:56.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.107+0000 7f9211ffb640 1 -- 172.21.15.19:0/3470119294 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9208010ac0 con 0x7f9214100e20 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.107+0000 7f921cf54740 1 -- start start 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.107+0000 7f921cf54740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x555681fec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f921cf54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555681feca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f921cf54740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f921cf54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555681f522f0 con 0x555681fec5d0 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f921cf54740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555681ef9cf0 con 0x555681ff7370 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f921cf54740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555681f4e2b0 con 0x555681fedb40 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9218917640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555681feca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9219118640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x555681fec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44092/0 (socket says 172.21.15.19:44092) 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 -- 172.21.15.19:0/531230195 learned_addr learned my addr 172.21.15.19:0/531230195 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 -- 172.21.15.19:0/531230195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 msgr2=0x555681fec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x555681fec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 -- 172.21.15.19:0/531230195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 msgr2=0x555681feca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555681feca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 -- 172.21.15.19:0/531230195 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555681ede3a0 con 0x555681ff7370 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9211ffb640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 0x7f92040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.108+0000 7f9213fff640 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9200011470 tx=0x7f9200011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e1d3e3b6b4e394e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f9219118640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 0x7f92040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f91feffd640 1 -- 172.21.15.19:0/531230195 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920001e070 con 0x555681ff7370 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f9211ffb640 1 -- 172.21.15.19:0/3470119294 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f92080cf320 con 0x7f9214100e20 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f91feffd640 1 -- 172.21.15.19:0/531230195 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92000035f0 con 0x555681ff7370 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f91feffd640 1 -- 172.21.15.19:0/531230195 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920000f860 con 0x555681ff7370 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 -- 172.21.15.19:0/531230195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 msgr2=0x555681ff9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f9219118640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 0x7f92040a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9200011470 tx=0x7f9200011930 comp rx=0 tx=0).stop 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 -- 172.21.15.19:0/531230195 shutdown_connections 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681ff7370 0x555681ff9760 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555681feca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 --2- 172.21.15.19:0/531230195 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x555681fec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.109+0000 7f921cf54740 1 -- 172.21.15.19:0/531230195 >> 172.21.15.19:0/531230195 conn(0x555681fb5310 msgr2=0x555681fe8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.110+0000 7f921cf54740 1 -- 172.21.15.19:0/531230195 shutdown_connections 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.110+0000 7f921cf54740 1 -- 172.21.15.19:0/531230195 wait complete. 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.110+0000 7f921cf54740 1 -- start start 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.110+0000 7f921cf54740 1 Processor -- start 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555681ff6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x555681ff45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555681ffa070 con 0x555681fedb40 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555681f0d060 con 0x555681fec5d0 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555681ffa1f0 con 0x555681ff7370 2024-09-12T03:25:56.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 10 client.0 _ll_register_callbacks cb 0x555681c608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9218917640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555681ff6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38896/0 (socket says 172.21.15.19:38896) 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 -- 172.21.15.19:0/1061079979 learned_addr learned my addr 172.21.15.19:0/1061079979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9219118640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x555681ff45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 -- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 msgr2=0x555681ff45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x555681ff45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 -- 172.21.15.19:0/1061079979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 msgr2=0x555681ff6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555681ff6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 -- 172.21.15.19:0/1061079979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555681cc33c0 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f9213fff640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9200011e30 tx=0x7f9200004210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ea92f909053651 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f91f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920001e040 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92000035f0 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.111+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9200005040 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9200005240 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 10 client.16593 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555682005b10 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92000035f0 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f9200021070 con 0x555681fedb40 2024-09-12T03:25:56.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 10 client.16593 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 10 client.16593 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 20 client.16593 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682006f10 0x555682009300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 4 client.16593 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555681f5f860 con 0x555682006f10 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.112+0000 7f9218917640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682006f10 0x555682009300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.113+0000 7f9218917640 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682006f10 0x555682009300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5ed6a6aa4156d4aa server_cookie=3f986065db60d404 in_seq=0 out_seq=0 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.113+0000 7f91f3fff640 10 client.16593 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f91f3fff640 1 -- 172.21.15.19:0/1061079979 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555681f5f860 con 0x555682006f10 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f91f3fff640 10 client.16593 handle_command_reply: tid=0 2024-09-12T03:25:56.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 client.16593 shutdown 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 msgr2=0x555681ff4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9200011e30 tx=0x7f9200004210 comp rx=0 tx=0).stop 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 shutdown_connections 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682006f10 0x555682009300 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x555681ff45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fedb40 0x555681ff4000 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 --2- 172.21.15.19:0/1061079979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555681ff6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 >> 172.21.15.19:0/1061079979 conn(0x555681fb5310 msgr2=0x555681fee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f91f3fff640 0 client.16593 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:25:56.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 shutdown_connections 2024-09-12T03:25:56.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 1 -- 172.21.15.19:0/1061079979 wait complete. 2024-09-12T03:25:56.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.119+0000 7f921cf54740 20 client.16593 trim_cache size 0 max 16384 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.133+0000 7f921cf54740 1 Processor -- start 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 -- start start 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 0x555681ff5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x55568200ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92141af830 con 0x555681fec5d0 2024-09-12T03:25:56.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b14f0 con 0x555681fedb40 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f921cf54740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92141aff00 con 0x555681ff7370 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38904/0 (socket says 172.21.15.19:38904) 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 -- 172.21.15.19:0/2352215176 learned_addr learned my addr 172.21.15.19:0/2352215176 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9213fff640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 0x555681ff5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9219118640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x55568200ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 -- 172.21.15.19:0/2352215176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 msgr2=0x55568200ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x55568200ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 -- 172.21.15.19:0/2352215176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 msgr2=0x555681ff5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 0x555681ff5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9218917640 1 -- 172.21.15.19:0/2352215176 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555681ff6900 con 0x555681fec5d0 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.134+0000 7f9213fff640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 0x555681ff5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f9218917640 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9208098610 tx=0x7f92080d2e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c702461db096b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f91fe7fc640 1 -- 172.21.15.19:0/2352215176 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920809b5a0 con 0x555681fec5d0 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f91fe7fc640 1 -- 172.21.15.19:0/2352215176 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920809bc40 con 0x555681fec5d0 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f91fe7fc640 1 -- 172.21.15.19:0/2352215176 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92080982b0 con 0x555681fec5d0 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 -- 172.21.15.19:0/2352215176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 msgr2=0x555682003340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9208098610 tx=0x7f92080d2e00 comp rx=0 tx=0).stop 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 -- 172.21.15.19:0/2352215176 shutdown_connections 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681ff7370 0x55568200ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fedb40 0x555681ff5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 --2- 172.21.15.19:0/2352215176 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681fec5d0 0x555682003340 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.135+0000 7f921cf54740 1 -- 172.21.15.19:0/2352215176 >> 172.21.15.19:0/2352215176 conn(0x555681fb5310 msgr2=0x555681ff5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- 172.21.15.19:0/2352215176 shutdown_connections 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- 172.21.15.19:0/2352215176 wait complete. 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- start start 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 Processor -- start 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x55568206a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 0x555681ffaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b0d60 con 0x555681ff7370 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92141b0690 con 0x555681fec5d0 2024-09-12T03:25:56.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f92141ad750 con 0x555681fedb40 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.136+0000 7f921cf54740 10 client.0 _ll_register_callbacks cb 0x7f920001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44136/0 (socket says 172.21.15.19:44136) 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 -- 172.21.15.19:0/906917028 learned_addr learned my addr 172.21.15.19:0/906917028 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9213fff640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x55568206a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9219118640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 0x555681ffaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 -- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 msgr2=0x55568206a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x55568206a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 -- 172.21.15.19:0/906917028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 msgr2=0x555681ffaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:56.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 0x555681ffaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 -- 172.21.15.19:0/906917028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555681ffb250 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9219118640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 0x555681ffaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f9218917640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92080980a0 tx=0x7f9208005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb36314c58e732d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f91f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920809ff40 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9208018b30 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.137+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f920809bc40 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f92080a4800 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 10 client.26099 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5556820716a0 con 0x555681fec5d0 2024-09-12T03:25:56.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920809bc40 con 0x555681fec5d0 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f92080ce040 con 0x555681fec5d0 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 10 client.26099 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 10 client.26099 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 20 client.26099 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682071e20 0x555682074240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 4 client.26099 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.138+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555681f59d70 con 0x555682071e20 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.140+0000 7f9213fff640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682071e20 0x555682074240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.140+0000 7f9213fff640 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682071e20 0x555682074240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54bad9af1b68d718 server_cookie=c29d7550d48f72e2 in_seq=0 out_seq=0 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.140+0000 7f91f3fff640 10 client.26099 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:25:56.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f91f3fff640 1 -- 172.21.15.19:0/906917028 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555681f59d70 con 0x555682071e20 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f91f3fff640 10 client.26099 handle_command_reply: tid=0 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 client.26099 shutdown 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 msgr2=0x555682069f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92080980a0 tx=0x7f9208005bf0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 shutdown_connections 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x555682071e20 0x555682074240 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555681ff7370 0x555681ffaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555681fedb40 0x55568206a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f91f3fff640 0 client.26099 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 --2- 172.21.15.19:0/906917028 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555681fec5d0 0x555682069f00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.141+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 >> 172.21.15.19:0/906917028 conn(0x555681fb5310 msgr2=0x555681ff5860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 shutdown_connections 2024-09-12T03:25:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921cf54740 1 -- 172.21.15.19:0/906917028 wait complete. 2024-09-12T03:25:56.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921cf54740 20 client.26099 trim_cache size 0 max 16384 2024-09-12T03:25:56.150 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 msgr2=0x7f92040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 0x7f92040a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f920c006010 tx=0x7f920c005d80 comp rx=0 tx=0).stop 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 msgr2=0x7f92140fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.142+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f920800e9f0 tx=0x7f920800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 shutdown_connections 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9214102ef0 0x7f92140fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f92040a2b80 0x7f92040a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9214102570 0x7f92140fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 --2- 172.21.15.19:0/3470119294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9214100e20 0x7f92140fcc80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 >> 172.21.15.19:0/3470119294 conn(0x7f92140fc7d0 msgr2=0x7f921406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 shutdown_connections 2024-09-12T03:25:56.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:56.143+0000 7f921ab7b640 1 -- 172.21.15.19:0/3470119294 wait complete. 2024-09-12T03:25:56.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:25:57.159 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:57.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:57 smithi019 ceph-mon[30953]: pgmap v544: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.6 MiB/s wr, 368 op/s 2024-09-12T03:25:57.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:57 smithi135 ceph-mon[35474]: pgmap v544: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.6 MiB/s wr, 368 op/s 2024-09-12T03:25:57.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:57 smithi121 ceph-mon[34963]: pgmap v544: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.6 MiB/s wr, 368 op/s 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.626+0000 7f2a685e6640 1 -- 172.21.15.19:0/4078713341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 msgr2=0x7f2a60106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.626+0000 7f2a685e6640 1 --2- 172.21.15.19:0/4078713341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a60106e60 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009930 tx=0x7f2a5c02a420 comp rx=0 tx=0).stop 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 -- 172.21.15.19:0/4078713341 shutdown_connections 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 --2- 172.21.15.19:0/4078713341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60101420 0x7f2a6010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 --2- 172.21.15.19:0/4078713341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60100a80 0x7f2a60100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 --2- 172.21.15.19:0/4078713341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a60106e60 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 -- 172.21.15.19:0/4078713341 >> 172.21.15.19:0/4078713341 conn(0x7f2a600fc7a0 msgr2=0x7f2a600febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 -- 172.21.15.19:0/4078713341 shutdown_connections 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 -- 172.21.15.19:0/4078713341 wait complete. 2024-09-12T03:25:57.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 Processor -- start 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 -- start start 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60100a80 0x7f2a60199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.627+0000 7f2a685e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60101420 0x7f2a60199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a685e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a685e6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60110e80 con 0x7f2a60106a80 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a685e6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60110d00 con 0x7f2a60100a80 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a685e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a60111000 con 0x7f2a60101420 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38930/0 (socket says 172.21.15.19:38930) 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60101420 0x7f2a60199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 -- 172.21.15.19:0/3937208549 learned_addr learned my addr 172.21.15.19:0/3937208549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:57.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66382640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60100a80 0x7f2a60199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 -- 172.21.15.19:0/3937208549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60101420 msgr2=0x7f2a60199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60101420 0x7f2a60199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 -- 172.21.15.19:0/3937208549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60100a80 msgr2=0x7f2a60199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60100a80 0x7f2a60199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 -- 172.21.15.19:0/3937208549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a601a48d0 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a66b83640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800eaa0 tx=0x7f2a5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a577fe640 1 -- 172.21.15.19:0/3937208549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5800cdb0 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a577fe640 1 -- 172.21.15.19:0/3937208549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a58005020 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a601a4aa0 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.628+0000 7f2a577fe640 1 -- 172.21.15.19:0/3937208549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a580116b0 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.629+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a601a4fb0 con 0x7f2a60106a80 2024-09-12T03:25:57.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.631+0000 7f2a577fe640 1 -- 172.21.15.19:0/3937208549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a58005560 con 0x7f2a60106a80 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.631+0000 7f2a6a9bf740 1 Processor -- start 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.631+0000 7f2a6a9bf740 1 -- start start 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 0x555ac359ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ac35042f0 con 0x555ac359fb40 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555ac34abcf0 con 0x555ac35a9370 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555ac35002b0 con 0x555ac359e5d0 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38944/0 (socket says 172.21.15.19:38944) 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a65b81640 1 -- 172.21.15.19:0/3580246361 learned_addr learned my addr 172.21.15.19:0/3580246361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 0x555ac359ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a577fe640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 0x7f2a4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66b83640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 0x7f2a4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a577fe640 1 -- 172.21.15.19:0/3937208549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a580cf110 con 0x7f2a60106a80 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 -- 172.21.15.19:0/3580246361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 msgr2=0x555ac35ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 -- 172.21.15.19:0/3580246361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 msgr2=0x555ac359e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a66382640 1 -- 172.21.15.19:0/3580246361 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac34903a0 con 0x555ac359e5d0 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.632+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.633+0000 7f2a66382640 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 0x555ac359ea10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005940 tx=0x7f2a5c005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5249585f51ddd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.633+0000 7f2a66b83640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 0x7f2a4c0a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a5809b9c0 tx=0x7f2a580666b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.633+0000 7f2a54ff9640 1 -- 172.21.15.19:0/3580246361 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c040070 con 0x555ac359e5d0 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.633+0000 7f2a54ff9640 1 -- 172.21.15.19:0/3580246361 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c031930 con 0x555ac359e5d0 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.633+0000 7f2a54ff9640 1 -- 172.21.15.19:0/3580246361 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c035040 con 0x555ac359e5d0 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3580246361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 msgr2=0x555ac359ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 0x555ac359ea10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c005940 tx=0x7f2a5c005c00 comp rx=0 tx=0).stop 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3580246361 shutdown_connections 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359e5d0 0x555ac359ea10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3580246361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359fb40 0x555ac359e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3580246361 >> 172.21.15.19:0/3580246361 conn(0x555ac3567310 msgr2=0x555ac359ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3580246361 shutdown_connections 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3580246361 wait complete. 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- start start 2024-09-12T03:25:57.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 Processor -- start 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555ac35ac010 con 0x555ac359e5d0 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555ac34bf060 con 0x555ac359fb40 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555ac35ac190 con 0x555ac35a9370 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a6a9bf740 10 client.0 _ll_register_callbacks cb 0x555ac32128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.634+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38946/0 (socket says 172.21.15.19:38946) 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 -- 172.21.15.19:0/3928481397 learned_addr learned my addr 172.21.15.19:0/3928481397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a66382640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 -- 172.21.15.19:0/3928481397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 msgr2=0x555ac35a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 -- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 msgr2=0x555ac35a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 -- 172.21.15.19:0/3928481397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac32753c0 con 0x555ac359e5d0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a66382640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a65b81640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000bfb0 tx=0x7f2a5000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45730d787327dc2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50003e80 con 0x555ac359e5d0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a50013b70 con 0x555ac359e5d0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c031930 con 0x555ac359e5d0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2a50005020 con 0x555ac359e5d0 2024-09-12T03:25:57.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a6a9bf740 10 client.16611 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.635+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555ac35b7960 con 0x555ac359e5d0 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c031930 con 0x555ac359e5d0 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f2a50010460 con 0x555ac359e5d0 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 10 client.16611 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 10 client.16611 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 20 client.16611 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac35bb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 4 client.16611 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ac3511860 con 0x555ac35b8e20 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.636+0000 7f2a66382640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac35bb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a66382640 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac35bb230 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=62903207c749fa03 server_cookie=98e9853db2b1a651 in_seq=0 out_seq=0 2024-09-12T03:25:57.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a49ffb640 10 client.16611 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a49ffb640 1 -- 172.21.15.19:0/3928481397 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555ac3511860 con 0x555ac35b8e20 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a49ffb640 10 client.16611 handle_command_reply: tid=0 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 client.16611 shutdown 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 msgr2=0x555ac35a8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000bfb0 tx=0x7f2a5000b390 comp rx=0 tx=0).stop 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 shutdown_connections 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac35bb230 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a49ffb640 0 client.16611 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/3928481397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35a8c00 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.637+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 >> 172.21.15.19:0/3928481397 conn(0x555ac3567310 msgr2=0x555ac35a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.638+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 shutdown_connections 2024-09-12T03:25:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.638+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/3928481397 wait complete. 2024-09-12T03:25:57.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.638+0000 7f2a6a9bf740 20 client.16611 trim_cache size 0 max 16384 2024-09-12T03:25:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.651+0000 7f2a6a9bf740 1 Processor -- start 2024-09-12T03:25:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.651+0000 7f2a6a9bf740 1 -- start start 2024-09-12T03:25:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35aa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601ad620 con 0x555ac359e5d0 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601af2e0 con 0x555ac359fb40 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601adcf0 con 0x555ac35a9370 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38962/0 (socket says 172.21.15.19:38962) 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a65b81640 1 -- 172.21.15.19:0/1396112138 learned_addr learned my addr 172.21.15.19:0/1396112138 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a65b81640 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 -- 172.21.15.19:0/1396112138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 msgr2=0x555ac35aa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35aa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 -- 172.21.15.19:0/1396112138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 msgr2=0x555ac35b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 -- 172.21.15.19:0/1396112138 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ac359f780 con 0x555ac359fb40 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.652+0000 7f2a66382640 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a12c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006010 tx=0x7f2a5c005460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beaf3f30cc0127f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a4bfff640 1 -- 172.21.15.19:0/1396112138 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c040040 con 0x555ac359fb40 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a4bfff640 1 -- 172.21.15.19:0/1396112138 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c004230 con 0x555ac359fb40 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a4bfff640 1 -- 172.21.15.19:0/1396112138 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c035040 con 0x555ac359fb40 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/1396112138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 msgr2=0x555ac35a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a12c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c006010 tx=0x7f2a5c005460 comp rx=0 tx=0).stop 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/1396112138 shutdown_connections 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac35a9370 0x555ac35aa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac359fb40 0x555ac35a12c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.653+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/1396112138 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/1396112138 >> 172.21.15.19:0/1396112138 conn(0x555ac3567310 msgr2=0x555ac35a4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/1396112138 shutdown_connections 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/1396112138 wait complete. 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- start start 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 Processor -- start 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 0x555ac35c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601aeb50 con 0x555ac359e5d0 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601ae480 con 0x555ac35a9370 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2a601ab540 con 0x555ac359fb40 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a6a9bf740 10 client.0 _ll_register_callbacks cb 0x7f2a5000bce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:57.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.654+0000 7f2a65b81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38972/0 (socket says 172.21.15.19:38972) 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 -- 172.21.15.19:0/2996348249 learned_addr learned my addr 172.21.15.19:0/2996348249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a66382640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 0x555ac35c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 -- 172.21.15.19:0/2996348249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 msgr2=0x555ac35c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 0x555ac35c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 -- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 msgr2=0x555ac35c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 -- 172.21.15.19:0/2996348249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ac35af7d0 con 0x555ac359e5d0 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a66382640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 0x555ac35c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a65b81640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000bf50 tx=0x7f2a5000b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4ead0b914da5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:57.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5002a070 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2a500051c0 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c004230 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.655+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2a500053c0 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 10 client.16617 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555ac3623c20 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a5c004230 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f2a500045a0 con 0x555ac359e5d0 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 10 client.16617 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 10 client.16617 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 20 client.16617 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac3626260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:57.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 4 client.16617 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555ac350bd70 con 0x555ac35b8e20 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.656+0000 7f2a66382640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac3626260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a49ffb640 10 client.16617 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a66382640 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac3626260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2de925f413d186ec server_cookie=c8d706ca96a4f489 in_seq=0 out_seq=0 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a49ffb640 1 -- 172.21.15.19:0/2996348249 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555ac350bd70 con 0x555ac35b8e20 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a49ffb640 10 client.16617 handle_command_reply: tid=0 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 client.16617 shutdown 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 msgr2=0x555ac35b1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a5000bf50 tx=0x7f2a5000b9d0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 shutdown_connections 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x555ac35b8e20 0x555ac3626260 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555ac35a9370 0x555ac35c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555ac359fb40 0x555ac35c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 --2- 172.21.15.19:0/2996348249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555ac359e5d0 0x555ac35b1cb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 >> 172.21.15.19:0/2996348249 conn(0x555ac3567310 msgr2=0x555ac35a4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a49ffb640 0 client.16617 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 shutdown_connections 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 1 -- 172.21.15.19:0/2996348249 wait complete. 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.657+0000 7f2a6a9bf740 20 client.16617 trim_cache size 0 max 16384 2024-09-12T03:25:57.665 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 msgr2=0x7f2a4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 0x7f2a4c0a50c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f2a5809b9c0 tx=0x7f2a580666b0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 msgr2=0x7f2a601a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800eaa0 tx=0x7f2a5800ef60 comp rx=0 tx=0).stop 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 shutdown_connections 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2a4c0a2c00 0x7f2a4c0a50c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2a60106a80 0x7f2a601a4390 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2a60101420 0x7f2a60199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 --2- 172.21.15.19:0/3937208549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2a60100a80 0x7f2a60199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 >> 172.21.15.19:0/3937208549 conn(0x7f2a600fc7a0 msgr2=0x7f2a600fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 shutdown_connections 2024-09-12T03:25:57.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:57.658+0000 7f2a685e6640 1 -- 172.21.15.19:0/3937208549 wait complete. 2024-09-12T03:25:58.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:25:58.896 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:25:59.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:25:59 smithi135 ceph-mon[35474]: pgmap v545: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 360 op/s 2024-09-12T03:25:59.348 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:25:59 smithi019 ceph-mon[30953]: pgmap v545: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 360 op/s 2024-09-12T03:25:59.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:25:59 smithi121 ceph-mon[34963]: pgmap v545: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 8.7 MiB/s wr, 360 op/s 2024-09-12T03:25:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 -- 172.21.15.19:0/2233876880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 msgr2=0x7f32b8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 --2- 172.21.15.19:0/2233876880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8106840 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009900 tx=0x7f32a802a410 comp rx=0 tx=0).stop 2024-09-12T03:25:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 -- 172.21.15.19:0/2233876880 shutdown_connections 2024-09-12T03:25:59.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 --2- 172.21.15.19:0/2233876880 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 0x7f32b810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 --2- 172.21.15.19:0/2233876880 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8106840 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 --2- 172.21.15.19:0/2233876880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 0x7f32b81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 -- 172.21.15.19:0/2233876880 >> 172.21.15.19:0/2233876880 conn(0x7f32b80fc5c0 msgr2=0x7f32b80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 -- 172.21.15.19:0/2233876880 shutdown_connections 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.565+0000 7f32bf98a640 1 -- 172.21.15.19:0/2233876880 wait complete. 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 Processor -- start 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 -- start start 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 0x7f32b8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.589 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 0x7f32b819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8111f90 con 0x7f32b8100eb0 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8111e10 con 0x7f32b8100500 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bf98a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8112110 con 0x7f32b8106f70 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38992/0 (socket says 172.21.15.19:38992) 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bcf25640 1 -- 172.21.15.19:0/788552397 learned_addr learned my addr 172.21.15.19:0/788552397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.566+0000 7f32bd726640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 0x7f32b8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bdf27640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 0x7f32b819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 -- 172.21.15.19:0/788552397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 msgr2=0x7f32b819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 0x7f32b819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 -- 172.21.15.19:0/788552397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 msgr2=0x7f32b8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.590 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 0x7f32b8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 -- 172.21.15.19:0/788552397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b819d400 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bcf25640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005940 tx=0x7f32a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32ae7fc640 1 -- 172.21.15.19:0/788552397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a803f030 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32ae7fc640 1 -- 172.21.15.19:0/788552397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a8003ce0 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b819d690 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32ae7fc640 1 -- 172.21.15.19:0/788552397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8035070 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.567+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a4fd0 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.569+0000 7f32ae7fc640 1 -- 172.21.15.19:0/788552397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32a8042030 con 0x7f32b8100eb0 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 Processor -- start 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 -- start start 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374354b40 0x562374353090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32ae7fc640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 0x7f32a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.591 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374364b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5623742b92f0 con 0x5623743535d0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562374260cf0 con 0x562374362780 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32c1d63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623742b52b0 con 0x562374354b40 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.570+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bd726640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 0x7f32a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39000/0 (socket says 172.21.15.19:39000) 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 -- 172.21.15.19:0/2463135967 learned_addr learned my addr 172.21.15.19:0/2463135967 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bdf27640 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374364b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32ae7fc640 1 -- 172.21.15.19:0/788552397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32a80ef560 con 0x7f32b8100eb0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bdf27640 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374354b40 0x562374353090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 -- 172.21.15.19:0/2463135967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374354b40 msgr2=0x562374353090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374354b40 0x562374353090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 -- 172.21.15.19:0/2463135967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 msgr2=0x562374364b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374364b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 -- 172.21.15.19:0/2463135967 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623742453a0 con 0x5623743535d0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bcf25640 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ed070 tx=0x7f32a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2623418510e5cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32bd726640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 0x7f32a40a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005fd0 tx=0x7f32b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32a37fe640 1 -- 172.21.15.19:0/2463135967 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8049040 con 0x5623743535d0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32a37fe640 1 -- 172.21.15.19:0/2463135967 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a80c5900 con 0x5623743535d0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.571+0000 7f32a37fe640 1 -- 172.21.15.19:0/2463135967 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8049e40 con 0x5623743535d0 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- 172.21.15.19:0/2463135967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 msgr2=0x562374353a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ed070 tx=0x7f32a8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- 172.21.15.19:0/2463135967 shutdown_connections 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374364b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x562374353a10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2463135967 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374354b40 0x562374353090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- 172.21.15.19:0/2463135967 >> 172.21.15.19:0/2463135967 conn(0x56237431c310 msgr2=0x56237434fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- 172.21.15.19:0/2463135967 shutdown_connections 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- 172.21.15.19:0/2463135967 wait complete. 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 -- start start 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.572+0000 7f32c1d63740 1 Processor -- start 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 0x562374362150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562374365240 con 0x562374354b40 2024-09-12T03:25:59.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562374274060 con 0x5623743535d0 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623743653c0 con 0x562374362780 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32c1d63740 10 client.0 _ll_register_callbacks cb 0x562373fc78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39004/0 (socket says 172.21.15.19:39004) 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 -- 172.21.15.19:0/389703083 learned_addr learned my addr 172.21.15.19:0/389703083 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bd726640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 0x562374362150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bdf27640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 -- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 msgr2=0x56237435f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 -- 172.21.15.19:0/389703083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 msgr2=0x562374362150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 0x562374362150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 -- 172.21.15.19:0/389703083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56237402a3c0 con 0x562374354b40 2024-09-12T03:25:59.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bd726640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 0x562374362150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32bcf25640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b87a0 tx=0x7f32a80044e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247c18540c90bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.573+0000 7f32a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8049040 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32a80b8440 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a80c5900 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32a80bbdd0 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 10 client.16629 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562374365240 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32a80c5900 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f32a80d1030 con 0x562374354b40 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 10 client.16629 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:59.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 10 client.16629 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 20 client.16629 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562374371db0 0x5623743741a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 4 client.16629 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623742c6860 con 0x562374371db0 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.574+0000 7f32bd726640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562374371db0 0x5623743741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.583+0000 7f32a0ff9640 10 client.16629 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.583+0000 7f32bd726640 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562374371db0 0x5623743741a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d05f4d7902fd9d server_cookie=9fa2efd985d8524f in_seq=0 out_seq=0 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32a0ff9640 1 -- 172.21.15.19:0/389703083 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623742c6860 con 0x562374371db0 2024-09-12T03:25:59.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32a0ff9640 10 client.16629 handle_command_reply: tid=0 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 client.16629 shutdown 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 msgr2=0x56237435f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b87a0 tx=0x7f32a80044e0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 shutdown_connections 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237435f400 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562374371db0 0x5623743741a0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 --2- 172.21.15.19:0/389703083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623743535d0 0x562374362150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32a0ff9640 0 client.16629 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:59.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 >> 172.21.15.19:0/389703083 conn(0x56237431c310 msgr2=0x562374355990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.586+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 shutdown_connections 2024-09-12T03:25:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.587+0000 7f32c1d63740 1 -- 172.21.15.19:0/389703083 wait complete. 2024-09-12T03:25:59.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.587+0000 7f32c1d63740 20 client.16629 trim_cache size 0 max 16384 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 Processor -- start 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 -- start start 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623743535d0 0x56237435f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237436e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ab400 con 0x562374354b40 2024-09-12T03:25:59.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ad0c0 con 0x562374362780 2024-09-12T03:25:59.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32c1d63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81abad0 con 0x5623743535d0 2024-09-12T03:25:59.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44230/0 (socket says 172.21.15.19:44230) 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 -- 172.21.15.19:0/2269849078 learned_addr learned my addr 172.21.15.19:0/2269849078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bd726640 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623743535d0 0x56237435f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bcf25640 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237436e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 -- 172.21.15.19:0/2269849078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623743535d0 msgr2=0x56237435f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623743535d0 0x56237435f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 -- 172.21.15.19:0/2269849078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 msgr2=0x56237436e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237436e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.600+0000 7f32bdf27640 1 -- 172.21.15.19:0/2269849078 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562374374510 con 0x562374362780 2024-09-12T03:25:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.601+0000 7f32bdf27640 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f32b0014310 tx=0x7f32b0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13743977990118a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.601+0000 7f32a2ffd640 1 -- 172.21.15.19:0/2269849078 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0014db0 con 0x562374362780 2024-09-12T03:25:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.601+0000 7f32a2ffd640 1 -- 172.21.15.19:0/2269849078 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b001f960 con 0x562374362780 2024-09-12T03:25:59.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.601+0000 7f32a2ffd640 1 -- 172.21.15.19:0/2269849078 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012980 con 0x562374362780 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 -- 172.21.15.19:0/2269849078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 msgr2=0x562374379a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f32b0014310 tx=0x7f32b0004a40 comp rx=0 tx=0).stop 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 -- 172.21.15.19:0/2269849078 shutdown_connections 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374362780 0x562374379a20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562374354b40 0x56237436e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 --2- 172.21.15.19:0/2269849078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623743535d0 0x56237435f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 -- 172.21.15.19:0/2269849078 >> 172.21.15.19:0/2269849078 conn(0x56237431c310 msgr2=0x56237435ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 -- 172.21.15.19:0/2269849078 shutdown_connections 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.602+0000 7f32c1d63740 1 -- 172.21.15.19:0/2269849078 wait complete. 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 -- start start 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 Processor -- start 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x56237435c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ac930 con 0x5623743535d0 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ac260 con 0x562374354b40 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81a9340 con 0x562374362780 2024-09-12T03:25:59.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32c1d63740 10 client.0 _ll_register_callbacks cb 0x7f32b0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32bd726640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x56237435c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.603+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:44232/0 (socket says 172.21.15.19:44232) 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 -- 172.21.15.19:0/1394732951 learned_addr learned my addr 172.21.15.19:0/1394732951 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 -- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 msgr2=0x56237435cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bdf27640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 -- 172.21.15.19:0/1394732951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 msgr2=0x56237435c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x56237435c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 -- 172.21.15.19:0/1394732951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623743db9f0 con 0x562374354b40 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bd726640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x56237435c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bdf27640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:25:59.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32bcf25640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32a80bc430 tx=0x7f32a802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2d5b0156ae25949 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.604+0000 7f32a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80d8070 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32a80c45b0 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b001f960 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32a809f050 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 10 client.26135 fetch_fsmap learned FSMap version 36 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5623743db9f0 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b001f960 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f32a80a2020 con 0x562374354b40 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 10 client.26135 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 10 client.26135 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 20 client.26135 populate_metadata read hostname 'smithi019' 2024-09-12T03:25:59.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623743dd0d0 0x5623743df4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:25:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 4 client.26135 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:25:59.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.605+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623742c0d70 con 0x5623743dd0d0 2024-09-12T03:25:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.612+0000 7f32bd726640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623743dd0d0 0x5623743df4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:25:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.612+0000 7f32bd726640 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623743dd0d0 0x5623743df4f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7194bbf1f12ce6b4 server_cookie=8a3beb289eaa0c8b in_seq=0 out_seq=0 2024-09-12T03:25:59.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.612+0000 7f32a0ff9640 10 client.26135 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.616+0000 7f32a0ff9640 1 -- 172.21.15.19:0/1394732951 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623742c0d70 con 0x5623743dd0d0 2024-09-12T03:25:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.616+0000 7f32a0ff9640 10 client.26135 handle_command_reply: tid=0 2024-09-12T03:25:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.616+0000 7f32c1d63740 1 client.26135 shutdown 2024-09-12T03:25:59.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.616+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 msgr2=0x56237435c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.616+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f32a80bc430 tx=0x7f32a802a410 comp rx=0 tx=0).stop 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 shutdown_connections 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562374362780 0x56237435cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623743dd0d0 0x5623743df4f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562374354b40 0x56237435c960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32a0ff9640 0 client.26135 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 --2- 172.21.15.19:0/1394732951 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623743535d0 0x56237435c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 >> 172.21.15.19:0/1394732951 conn(0x56237431c310 msgr2=0x562374361480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 shutdown_connections 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 1 -- 172.21.15.19:0/1394732951 wait complete. 2024-09-12T03:25:59.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32c1d63740 20 client.26135 trim_cache size 0 max 16384 2024-09-12T03:25:59.619 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:25:59.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 msgr2=0x7f32a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 0x7f32a40a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f32b4005fd0 tx=0x7f32b400b410 comp rx=0 tx=0).stop 2024-09-12T03:25:59.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 msgr2=0x7f32b8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:25:59.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f32a8005940 tx=0x7f32a80042f0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 shutdown_connections 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.617+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32b8106f70 0x7f32b819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32a40a2b80 0x7f32a40a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32b8100eb0 0x7f32b8198990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 --2- 172.21.15.19:0/788552397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32b8100500 0x7f32b8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 >> 172.21.15.19:0/788552397 conn(0x7f32b80fc5c0 msgr2=0x7f32b80fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 shutdown_connections 2024-09-12T03:25:59.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:25:59.618+0000 7f32bf98a640 1 -- 172.21.15.19:0/788552397 wait complete. 2024-09-12T03:26:01.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:01 smithi135 ceph-mon[35474]: pgmap v546: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 235 op/s 2024-09-12T03:26:01.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:01 smithi121 ceph-mon[34963]: pgmap v546: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 235 op/s 2024-09-12T03:26:01.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:01 smithi019 ceph-mon[30953]: pgmap v546: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 7.5 MiB/s wr, 235 op/s 2024-09-12T03:26:01.460 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:26:01.461 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:26:01.731 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- 172.21.15.19:0/3770992209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14102d40 msgr2=0x7efe14103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 --2- 172.21.15.19:0/3770992209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14102d40 0x7efe14103120 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7efe04009860 tx=0x7efe0402a3d0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- 172.21.15.19:0/3770992209 shutdown_connections 2024-09-12T03:26:02.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 --2- 172.21.15.19:0/3770992209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe1410c170 0x7efe1410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 --2- 172.21.15.19:0/3770992209 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14103660 0x7efe1410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 --2- 172.21.15.19:0/3770992209 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe14102d40 0x7efe14103120 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- 172.21.15.19:0/3770992209 >> 172.21.15.19:0/3770992209 conn(0x7efe140fc7a0 msgr2=0x7efe140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- 172.21.15.19:0/3770992209 shutdown_connections 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- 172.21.15.19:0/3770992209 wait complete. 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 Processor -- start 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.239+0000 7efe1aa28640 1 -- start start 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14102d40 0x7efe141009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14103660 0x7efe14100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe14111f70 con 0x7efe1410c170 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe14111df0 con 0x7efe14103660 2024-09-12T03:26:02.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe1aa28640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe141120f0 con 0x7efe14102d40 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50670/0 (socket says 172.21.15.19:50670) 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 -- 172.21.15.19:0/774542599 learned_addr learned my addr 172.21.15.19:0/774542599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14102d40 0x7efe141009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe137fe640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14103660 0x7efe14100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 -- 172.21.15.19:0/774542599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14102d40 msgr2=0x7efe141009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14102d40 0x7efe141009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 -- 172.21.15.19:0/774542599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14103660 msgr2=0x7efe14100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14103660 0x7efe14100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 -- 172.21.15.19:0/774542599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe141a68c0 con 0x7efe1410c170 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.240+0000 7efe18fc5640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00eb10 tx=0x7efe0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.241+0000 7efe117fa640 1 -- 172.21.15.19:0/774542599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c004b20 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.241+0000 7efe117fa640 1 -- 172.21.15.19:0/774542599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe0c004cc0 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.241+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe141a6bb0 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.241+0000 7efe117fa640 1 -- 172.21.15.19:0/774542599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c010760 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.241+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141a70a0 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.243+0000 7efe117fa640 1 -- 172.21.15.19:0/774542599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe0c005020 con 0x7efe1410c170 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 Processor -- start 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 -- start start 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe117fa640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 0x7efdfc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0bc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d09b014280 con 0x55d09b0aeb60 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d09afbbb10 con 0x55d09b0af480 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe1ce01740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d09b010240 con 0x55d09b0b9d40 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe117fa640 1 -- 172.21.15.19:0/774542599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe0c0cf630 con 0x7efe1410c170 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe137fe640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 0x7efdfc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe137fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe13fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe137fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:51440/0 (socket says 172.21.15.19:51440) 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe137fe640 1 -- 172.21.15.19:0/443405222 learned_addr learned my addr 172.21.15.19:0/443405222 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.244+0000 7efe18fc5640 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0bc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 -- 172.21.15.19:0/443405222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 msgr2=0x55d09b0bc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0bc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 -- 172.21.15.19:0/443405222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 msgr2=0x55d09b0aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 -- 172.21.15.19:0/443405222 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09afc81f0 con 0x55d09b0af480 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe13fff640 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 0x7efdfc0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe08009600 tx=0x7efe08009b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efe137fe640 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe08005b30 tx=0x7efe0801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6581a488aead1952 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efdfa7fc640 1 -- 172.21.15.19:0/443405222 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08010da0 con 0x55d09b0af480 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efdfa7fc640 1 -- 172.21.15.19:0/443405222 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe08005500 con 0x55d09b0af480 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.245+0000 7efdfa7fc640 1 -- 172.21.15.19:0/443405222 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08004b40 con 0x55d09b0af480 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- 172.21.15.19:0/443405222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 msgr2=0x55d09b0b9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efe08005b30 tx=0x7efe0801c000 comp rx=0 tx=0).stop 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- 172.21.15.19:0/443405222 shutdown_connections 2024-09-12T03:26:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0bc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b9800 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 --2- 172.21.15.19:0/443405222 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0aef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- 172.21.15.19:0/443405222 >> 172.21.15.19:0/443405222 conn(0x55d09b0ab0f0 msgr2=0x55d09b0ab4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- 172.21.15.19:0/443405222 shutdown_connections 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- 172.21.15.19:0/443405222 wait complete. 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.246+0000 7efe1ce01740 1 -- start start 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 Processor -- start 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0b7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d09b0bc8b0 con 0x55d09b0aeb60 2024-09-12T03:26:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d09afcef10 con 0x55d09b0af480 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d09b0bca30 con 0x55d09b0b9d40 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe1ce01740 10 client.0 _ll_register_callbacks cb 0x55d09af97a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe13fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe13fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50696/0 (socket says 172.21.15.19:50696) 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe137fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe13fff640 1 -- 172.21.15.19:0/1546893201 learned_addr learned my addr 172.21.15.19:0/1546893201 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.247+0000 7efe18fc5640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0b7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 -- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 msgr2=0x55d09b0b7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0b7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 -- 172.21.15.19:0/1546893201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 msgr2=0x55d09b0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 -- 172.21.15.19:0/1546893201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09ad856a0 con 0x55d09b0aeb60 2024-09-12T03:26:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe137fe640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe13fff640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efe04034000 tx=0x7efe04032e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1814412556e426 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efdf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04030670 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe04030810 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe08005500 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efe04005040 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe1ce01740 10 client.16647 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d09b0c84e0 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.248+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe08005500 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7efe0402f570 con 0x55d09b0aeb60 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 10 client.16647 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 10 client.16647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 20 client.16647 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b0c9a40 0x55d09b0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 4 client.16647 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.249+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d09b0217f0 con 0x55d09b0c9a40 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.252+0000 7efe137fe640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b0c9a40 0x55d09b0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.252+0000 7efe137fe640 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b0c9a40 0x55d09b0cbe30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=337d424b7b228470 server_cookie=b3b334fda5edcf4b in_seq=0 out_seq=0 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.252+0000 7efdf37fe640 10 client.16647 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.254+0000 7efdf37fe640 1 -- 172.21.15.19:0/1546893201 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d09b0217f0 con 0x55d09b0c9a40 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.254+0000 7efdf37fe640 10 client.16647 handle_command_reply: tid=0 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.254+0000 7efe1ce01740 1 client.16647 shutdown 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 msgr2=0x55d09b0b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7efe04034000 tx=0x7efe04032e80 comp rx=0 tx=0).stop 2024-09-12T03:26:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 shutdown_connections 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0b9d40 0x55d09b0b7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b0c9a40 0x55d09b0cbe30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efdf37fe640 0 client.16647 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 --2- 172.21.15.19:0/1546893201 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0aeb60 0x55d09b0b6c00 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 >> 172.21.15.19:0/1546893201 conn(0x55d09b0ab0f0 msgr2=0x55d09b0aa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 shutdown_connections 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 1 -- 172.21.15.19:0/1546893201 wait complete. 2024-09-12T03:26:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.255+0000 7efe1ce01740 20 client.16647 trim_cache size 0 max 16384 2024-09-12T03:26:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 Processor -- start 2024-09-12T03:26:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 -- start start 2024-09-12T03:26:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0b9d40 0x55d09b0d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe141af700 con 0x55d09b0af480 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b13c0 con 0x55d09b0b9d40 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe1ce01740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe141afdd0 con 0x55d09b0aeb60 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50710/0 (socket says 172.21.15.19:50710) 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 -- 172.21.15.19:0/4266745236 learned_addr learned my addr 172.21.15.19:0/4266745236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe18fc5640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0b9d40 0x55d09b0d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe13fff640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 -- 172.21.15.19:0/4266745236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 msgr2=0x55d09b0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 -- 172.21.15.19:0/4266745236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0b9d40 msgr2=0x55d09b0d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0b9d40 0x55d09b0d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe137fe640 1 -- 172.21.15.19:0/4266745236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d09b0bc4d0 con 0x55d09b0af480 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.274+0000 7efe13fff640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.275+0000 7efe137fe640 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efe080107a0 tx=0x7efe08009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b6219229d399174 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.275+0000 7efdf9ffb640 1 -- 172.21.15.19:0/4266745236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0801f740 con 0x55d09b0af480 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.275+0000 7efdf9ffb640 1 -- 172.21.15.19:0/4266745236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe08038070 con 0x55d09b0af480 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.275+0000 7efdf9ffb640 1 -- 172.21.15.19:0/4266745236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0802b6d0 con 0x55d09b0af480 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- 172.21.15.19:0/4266745236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 msgr2=0x55d09b0b89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7efe080107a0 tx=0x7efe08009630 comp rx=0 tx=0).stop 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- 172.21.15.19:0/4266745236 shutdown_connections 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0b9d40 0x55d09b0d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0af480 0x55d09b0b89f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 --2- 172.21.15.19:0/4266745236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- 172.21.15.19:0/4266745236 >> 172.21.15.19:0/4266745236 conn(0x55d09b0ab0f0 msgr2=0x55d09b0b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- 172.21.15.19:0/4266745236 shutdown_connections 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- 172.21.15.19:0/4266745236 wait complete. 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.276+0000 7efe1ce01740 1 -- start start 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 Processor -- start 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0b9d40 0x55d09b0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b0c30 con 0x55d09b0b9d40 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b0560 con 0x55d09b0af480 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe141ad620 con 0x55d09b0aeb60 2024-09-12T03:26:02.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe1ce01740 10 client.0 _ll_register_callbacks cb 0x7efe040096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe13fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:51474/0 (socket says 172.21.15.19:51474) 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 -- 172.21.15.19:0/940218280 learned_addr learned my addr 172.21.15.19:0/940218280 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe18fc5640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0b9d40 0x55d09b0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 -- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 msgr2=0x55d09b0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 -- 172.21.15.19:0/940218280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0b9d40 msgr2=0x55d09b0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0b9d40 0x55d09b0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.277+0000 7efe137fe640 1 -- 172.21.15.19:0/940218280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d09b0b4440 con 0x55d09b0af480 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe137fe640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe08000c00 tx=0x7efe08009810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ed812e35c497a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:02.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08012250 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe0801f740 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe08038070 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7efe0801f940 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 10 client.26153 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d09b1345f0 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe08038070 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7efe08045070 con 0x55d09b0af480 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 10 client.26153 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 10 client.26153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 20 client.26153 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b134bc0 0x55d09b136fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.278+0000 7efe1ce01740 4 client.26153 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:26:02.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.279+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d09b01bd00 con 0x55d09b134bc0 2024-09-12T03:26:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.282+0000 7efe13fff640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b134bc0 0x55d09b136fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.282+0000 7efe13fff640 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b134bc0 0x55d09b136fe0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484d4fbb76274f76 server_cookie=365fb4e5313c75ea in_seq=0 out_seq=0 2024-09-12T03:26:02.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.282+0000 7efdf37fe640 10 client.26153 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.285+0000 7efdf37fe640 1 -- 172.21.15.19:0/940218280 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d09b01bd00 con 0x55d09b134bc0 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.285+0000 7efdf37fe640 10 client.26153 handle_command_reply: tid=0 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.285+0000 7efe1ce01740 1 client.26153 shutdown 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 msgr2=0x55d09b0b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe08000c00 tx=0x7efe08009810 comp rx=0 tx=0).stop 2024-09-12T03:26:02.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 shutdown_connections 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d09b0b9d40 0x55d09b0b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d09b134bc0 0x55d09b136fe0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d09b0af480 0x55d09b0b3810 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 --2- 172.21.15.19:0/940218280 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d09b0aeb60 0x55d09b0c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efdf37fe640 0 client.26153 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 >> 172.21.15.19:0/940218280 conn(0x55d09b0ab0f0 msgr2=0x55d09b0b8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 shutdown_connections 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 1 -- 172.21.15.19:0/940218280 wait complete. 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1ce01740 20 client.26153 trim_cache size 0 max 16384 2024-09-12T03:26:02.287 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:26:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "7f3f9c64-ef0a-438b-a300-71c4f834ceae", 2024-09-12T03:26:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:26:02.288 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:02.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 msgr2=0x7efdfc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 0x7efdfc0a5040 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efe08009600 tx=0x7efe08009b90 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 msgr2=0x7efe14101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.286+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00eb10 tx=0x7efe0c00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 shutdown_connections 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efdfc0a2b80 0x7efdfc0a5040 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe1410c170 0x7efe14101420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe14103660 0x7efe14100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 --2- 172.21.15.19:0/774542599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe14102d40 0x7efe141009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 >> 172.21.15.19:0/774542599 conn(0x7efe140fc7a0 msgr2=0x7efe140fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 shutdown_connections 2024-09-12T03:26:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:02.287+0000 7efe1aa28640 1 -- 172.21.15.19:0/774542599 wait complete. 2024-09-12T03:26:03.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:26:03.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: pgmap v547: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 242 op/s 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:03.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:03 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: pgmap v547: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 242 op/s 2024-09-12T03:26:03.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:03.295 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:03 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:03.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: pgmap v547: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.5 MiB/s wr, 242 op/s 2024-09-12T03:26:03.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:03.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:03 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:03.428 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- 172.21.15.19:0/526524042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 msgr2=0x7f70b810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/526524042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b810da40 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b002a480 comp rx=0 tx=0).stop 2024-09-12T03:26:03.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- 172.21.15.19:0/526524042 shutdown_connections 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/526524042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b810da40 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/526524042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 0x7f70b81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/526524042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- 172.21.15.19:0/526524042 >> 172.21.15.19:0/526524042 conn(0x7f70b80fc7d0 msgr2=0x7f70b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- 172.21.15.19:0/526524042 shutdown_connections 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- 172.21.15.19:0/526524042 wait complete. 2024-09-12T03:26:03.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 Processor -- start 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.936+0000 7f70bfb8c640 1 -- start start 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 0x7f70b81978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8110f80 con 0x7f70b8102ef0 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8110e00 con 0x7f70b8100e20 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bfb8c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8111100 con 0x7f70b8102570 2024-09-12T03:26:03.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:51484/0 (socket says 172.21.15.19:51484) 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 -- 172.21.15.19:0/3480672848 learned_addr learned my addr 172.21.15.19:0/3480672848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70be129640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd127640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 0x7f70b81978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 -- 172.21.15.19:0/3480672848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 msgr2=0x7f70b81978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 0x7f70b81978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 -- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 msgr2=0x7f70b819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 -- 172.21.15.19:0/3480672848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b819c330 con 0x7f70b8100e20 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70bd928640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f70b400ebc0 tx=0x7f70b400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.937+0000 7f70aeffd640 1 -- 172.21.15.19:0/3480672848 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4010630 con 0x7f70b8100e20 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.938+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b819c620 con 0x7f70b8100e20 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.938+0000 7f70aeffd640 1 -- 172.21.15.19:0/3480672848 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b4010cb0 con 0x7f70b8100e20 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.938+0000 7f70aeffd640 1 -- 172.21.15.19:0/3480672848 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4018bd0 con 0x7f70b8100e20 2024-09-12T03:26:03.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.938+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81a3f00 con 0x7f70b8100e20 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70aeffd640 1 -- 172.21.15.19:0/3480672848 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70b40107d0 con 0x7f70b8100e20 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 Processor -- start 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 -- start start 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b6879bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687aab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b686ff2e0 con 0x562b6879cd00 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562b686a6ce0 con 0x562b687a8740 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.940+0000 7f70c1f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b686fb2a0 con 0x562b6879bad0 2024-09-12T03:26:03.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd928640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b6879bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70be129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70be129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50732/0 (socket says 172.21.15.19:50732) 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70be129640 1 -- 172.21.15.19:0/2865027718 learned_addr learned my addr 172.21.15.19:0/2865027718 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd928640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b6879bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:34528/0 (socket says 172.21.15.19:34528) 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687aab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 -- 172.21.15.19:0/2865027718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 msgr2=0x562b6879bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b6879bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 -- 172.21.15.19:0/2865027718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 msgr2=0x562b6879b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 -- 172.21.15.19:0/2865027718 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6868b440 con 0x562b687a8740 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70be129640 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70aeffd640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 0x7f70a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70be129640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 0x7f70a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70bd127640 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687aab30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a800bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18487fed9e6bddb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70aeffd640 1 -- 172.21.15.19:0/3480672848 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70b40cf6b0 con 0x7f70b8100e20 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.941+0000 7f70a3fff640 1 -- 172.21.15.19:0/2865027718 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8003bd0 con 0x562b687a8740 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70a3fff640 1 -- 172.21.15.19:0/2865027718 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8015070 con 0x562b687a8740 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70a3fff640 1 -- 172.21.15.19:0/2865027718 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8004d50 con 0x562b687a8740 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 -- 172.21.15.19:0/2865027718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 msgr2=0x562b687aab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687aab30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f70a8005fd0 tx=0x7f70a800bb20 comp rx=0 tx=0).stop 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70be129640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 0x7f70a40a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005ab0 tx=0x7f70b0005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 -- 172.21.15.19:0/2865027718 shutdown_connections 2024-09-12T03:26:03.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687aab30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b6879bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 --2- 172.21.15.19:0/2865027718 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b6879b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 -- 172.21.15.19:0/2865027718 >> 172.21.15.19:0/2865027718 conn(0x562b68797c40 msgr2=0x562b68798020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 -- 172.21.15.19:0/2865027718 shutdown_connections 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.942+0000 7f70c1f65740 1 -- 172.21.15.19:0/2865027718 wait complete. 2024-09-12T03:26:03.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 -- start start 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 Processor -- start 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879cd00 0x562b687a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562b687ab650 con 0x562b6879bad0 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x562b686ba050 con 0x562b687a8740 2024-09-12T03:26:03.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562b687ab7d0 con 0x562b6879cd00 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70c1f65740 10 client.0 _ll_register_callbacks cb 0x562b686fa160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70bd928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70bd928640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50748/0 (socket says 172.21.15.19:50748) 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.943+0000 7f70bd928640 1 -- 172.21.15.19:0/3105003996 learned_addr learned my addr 172.21.15.19:0/3105003996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd127640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879cd00 0x562b687a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70be129640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 -- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879cd00 msgr2=0x562b687a53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879cd00 0x562b687a53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 -- 172.21.15.19:0/3105003996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 msgr2=0x562b687a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 -- 172.21.15.19:0/3105003996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b684703c0 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70bd928640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f70b409bc70 tx=0x7f70b4097bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f868f6038459e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b4010e60 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70b409b3c0 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a8015070 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f70b409b8d0 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70c1f65740 10 client.16665 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b687b70f0 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.944+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8015070 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f70b40d1b70 con 0x562b6879bad0 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 10 client.16665 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 10 client.16665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:03.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 20 client.16665 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b687baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 4 client.16665 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.945+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b6870c850 con 0x562b687b85b0 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.947+0000 7f70bd127640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b687baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.947+0000 7f70bd127640 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b687baa20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90e3911d550c6b31 server_cookie=e86835088147752 in_seq=0 out_seq=0 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.947+0000 7f70a17fa640 10 client.16665 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70a17fa640 1 -- 172.21.15.19:0/3105003996 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562b6870c850 con 0x562b687b85b0 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70a17fa640 10 client.16665 handle_command_reply: tid=0 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 client.16665 shutdown 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 msgr2=0x562b687a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f70b409bc70 tx=0x7f70b4097bb0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 shutdown_connections 2024-09-12T03:26:03.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b687baa20 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b687a8740 0x562b687a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879cd00 0x562b687a53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70a17fa640 0 client.16665 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3105003996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879bad0 0x562b687a8110 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 >> 172.21.15.19:0/3105003996 conn(0x562b68797c40 msgr2=0x562b6879d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 shutdown_connections 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 1 -- 172.21.15.19:0/3105003996 wait complete. 2024-09-12T03:26:03.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.949+0000 7f70c1f65740 20 client.16665 trim_cache size 0 max 16384 2024-09-12T03:26:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 Processor -- start 2024-09-12T03:26:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 -- start start 2024-09-12T03:26:03.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 0x562b687b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b687a8740 0x562b687c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ab350 con 0x562b6879cd00 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ad010 con 0x562b6879bad0 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70c1f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81aba20 con 0x562b687a8740 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70bd127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70bd127640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50756/0 (socket says 172.21.15.19:50756) 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.967+0000 7f70bd127640 1 -- 172.21.15.19:0/3227210776 learned_addr learned my addr 172.21.15.19:0/3227210776 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70be129640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b687a8740 0x562b687c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd928640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 0x562b687b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 -- 172.21.15.19:0/3227210776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b687a8740 msgr2=0x562b687c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b687a8740 0x562b687c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 -- 172.21.15.19:0/3227210776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 msgr2=0x562b687b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 0x562b687b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 -- 172.21.15.19:0/3227210776 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b687a4390 con 0x562b6879cd00 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd928640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 0x562b687b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70bd127640 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004580 tx=0x7f70a8005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddf05d54275571a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70a37fe640 1 -- 172.21.15.19:0/3227210776 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a800be70 con 0x562b6879cd00 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70a37fe640 1 -- 172.21.15.19:0/3227210776 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8015040 con 0x562b6879cd00 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.968+0000 7f70a37fe640 1 -- 172.21.15.19:0/3227210776 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8011440 con 0x562b6879cd00 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- 172.21.15.19:0/3227210776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 msgr2=0x562b687969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004580 tx=0x7f70a8005c50 comp rx=0 tx=0).stop 2024-09-12T03:26:03.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- 172.21.15.19:0/3227210776 shutdown_connections 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b687a8740 0x562b687c02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b6879cd00 0x562b687969a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 --2- 172.21.15.19:0/3227210776 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879bad0 0x562b687b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- 172.21.15.19:0/3227210776 >> 172.21.15.19:0/3227210776 conn(0x562b68797c40 msgr2=0x562b68796290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- 172.21.15.19:0/3227210776 shutdown_connections 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- 172.21.15.19:0/3227210776 wait complete. 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 -- start start 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.969+0000 7f70c1f65740 1 Processor -- start 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b687ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879cd00 0x562b687adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ac880 con 0x562b687a8740 2024-09-12T03:26:03.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ac1b0 con 0x562b6879cd00 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81a9270 con 0x562b6879bad0 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70c1f65740 10 client.0 _ll_register_callbacks cb 0x7f70b40099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50764/0 (socket says 172.21.15.19:50764) 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 -- 172.21.15.19:0/217063459 learned_addr learned my addr 172.21.15.19:0/217063459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70bd928640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b687ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70bd127640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879cd00 0x562b687adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 -- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 msgr2=0x562b687ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b687ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 -- 172.21.15.19:0/217063459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879cd00 msgr2=0x562b687adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879cd00 0x562b687adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.970+0000 7f70be129640 1 -- 172.21.15.19:0/217063459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b687a8480 con 0x562b687a8740 2024-09-12T03:26:03.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70be129640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80648b1b1041560c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0035040 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f70b0005020 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70a8015040 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f70b0005220 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 10 client.16671 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562b68824250 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70a8015040 con 0x562b687a8740 2024-09-12T03:26:03.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f70b0044040 con 0x562b687a8740 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 10 client.16671 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 20 client.16671 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b68826b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 4 client.16671 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.971+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b68706d60 con 0x562b687b85b0 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.972+0000 7f70bd928640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b68826b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.972+0000 7f70a17fa640 10 client.16671 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.972+0000 7f70bd928640 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b68826b50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eae583d158f6897 server_cookie=f66bb02ed64cc97d in_seq=0 out_seq=0 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70a17fa640 1 -- 172.21.15.19:0/217063459 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+269 (crc 0 0 0) 0x562b68706d60 con 0x562b687b85b0 2024-09-12T03:26:03.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70a17fa640 10 client.16671 handle_command_reply: tid=0 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 client.16671 shutdown 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 msgr2=0x562b687ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009900 tx=0x7f70b0009260 comp rx=0 tx=0).stop 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 shutdown_connections 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562b687a8740 0x562b687ae4e0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562b687b85b0 0x562b68826b50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562b6879cd00 0x562b687adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 --2- 172.21.15.19:0/217063459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562b6879bad0 0x562b687ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70a17fa640 0 client.16671 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.973+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 >> 172.21.15.19:0/217063459 conn(0x562b68797c40 msgr2=0x562b68796ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 shutdown_connections 2024-09-12T03:26:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70c1f65740 1 -- 172.21.15.19:0/217063459 wait complete. 2024-09-12T03:26:03.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70c1f65740 20 client.16671 trim_cache size 0 max 16384 2024-09-12T03:26:03.986 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (12267 inodes in the stack)", 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "7f3f9c64-ef0a-438b-a300-71c4f834ceae": { 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "7f3f9c64-ef0a-438b-a300-71c4f834ceae", 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:03.987 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 msgr2=0x7f70a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 0x7f70a40a5040 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70b0005ab0 tx=0x7f70b0005a40 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 msgr2=0x7f70b8197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f70b400ebc0 tx=0x7f70b400c6a0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 shutdown_connections 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f70a40a2b80 0x7f70a40a5040 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f70b8102ef0 0x7f70b819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f70b8102570 0x7f70b81978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 --2- 172.21.15.19:0/3480672848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70b8100e20 0x7f70b8197380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 >> 172.21.15.19:0/3480672848 conn(0x7f70b80fc7d0 msgr2=0x7f70b8106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:03.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 shutdown_connections 2024-09-12T03:26:03.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:03.974+0000 7f70bfb8c640 1 -- 172.21.15.19:0/3480672848 wait complete. 2024-09-12T03:26:04.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:04 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:26:04.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:04.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:04.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:04.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:04.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:04 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:26:04.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:04.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:04.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:04.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:04 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:26:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:04.713 INFO:tasks.cephfs.filesystem:scrub status for tag:7f3f9c64-ef0a-438b-a300-71c4f834ceae - {'path': '/', 'tag': '7f3f9c64-ef0a-438b-a300-71c4f834ceae', 'options': 'recursive,force'} 2024-09-12T03:26:05.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:05 smithi121 ceph-mon[34963]: pgmap v548: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 230 op/s 2024-09-12T03:26:05.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:05 smithi019 ceph-mon[30953]: pgmap v548: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 230 op/s 2024-09-12T03:26:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:05 smithi135 ceph-mon[35474]: pgmap v548: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 6.2 MiB/s wr, 230 op/s 2024-09-12T03:26:07.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:07 smithi121 ceph-mon[34963]: pgmap v549: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-12T03:26:07.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:07 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:07 smithi019 ceph-mon[30953]: pgmap v549: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-12T03:26:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:07 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:07 smithi135 ceph-mon[35474]: pgmap v549: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.6 MiB/s wr, 292 op/s 2024-09-12T03:26:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:07 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:09.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:09 smithi121 ceph-mon[34963]: pgmap v550: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-12T03:26:09.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:09 smithi019 ceph-mon[30953]: pgmap v550: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-12T03:26:09.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:09 smithi135 ceph-mon[35474]: pgmap v550: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 302 op/s 2024-09-12T03:26:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:11 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:11.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:11 smithi121 ceph-mon[34963]: pgmap v551: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-12T03:26:11.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:11 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:11 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:11 smithi019 ceph-mon[30953]: pgmap v551: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-12T03:26:11.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:11 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:11 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:11 smithi135 ceph-mon[35474]: pgmap v551: 97 pgs: 97 active+clean; 2.4 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-12T03:26:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:11 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:13.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:13 smithi121 ceph-mon[34963]: pgmap v552: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-09-12T03:26:13.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:13 smithi019 ceph-mon[30953]: pgmap v552: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-09-12T03:26:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:13 smithi135 ceph-mon[35474]: pgmap v552: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 313 op/s 2024-09-12T03:26:15.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:15 smithi121 ceph-mon[34963]: pgmap v553: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-09-12T03:26:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:15 smithi019 ceph-mon[30953]: pgmap v553: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-09-12T03:26:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:15 smithi135 ceph-mon[35474]: pgmap v553: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 237 op/s 2024-09-12T03:26:15.566 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 36, 2024-09-12T03:26:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:26:15.566 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 22, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "e82c68e9": { 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 22, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.1, 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:26:15.567 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:15.568 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64c8ff9640 1 -- 172.21.15.19:0/3056053728 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5601066cbec0 con 0x560106868cd0 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64c8ff9640 10 client.26021 handle_command_reply: tid=0 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 client.26021 shutdown 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 msgr2=0x5601067edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64dc020650 tx=0x7f64dc00aeb0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 shutdown_connections 2024-09-12T03:26:15.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5601067ee800 0x5601067f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5601067ed940 0x5601067ee2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560106868cd0 0x56010686b110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 --2- 172.21.15.19:0/3056053728 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5601067eab70 0x5601067edd80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64c8ff9640 0 client.26021 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.565+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 >> 172.21.15.19:0/3056053728 conn(0x5601067decf0 msgr2=0x5601067e9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 shutdown_connections 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e9821740 1 -- 172.21.15.19:0/3056053728 wait complete. 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e9821740 20 client.26021 trim_cache size 0 max 16384 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 msgr2=0x7f64cc0a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 0x7f64cc0a53f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64d8011fd0 tx=0x7f64d80133c0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 msgr2=0x7f64e007fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:15.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f64d0005ab0 tx=0x7f64d00055c0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 shutdown_connections 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f64e01091f0 0x7f64e0080180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f64cc0a2f30 0x7f64cc0a53f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f64e0074c90 0x7f64e007fc40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 --2- 172.21.15.19:0/3081177223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f64e00742d0 0x7f64e0084bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.566+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 >> 172.21.15.19:0/3081177223 conn(0x7f64e006e140 msgr2=0x7f64e010a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.567+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 shutdown_connections 2024-09-12T03:26:15.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:15.567+0000 7f64e7448640 1 -- 172.21.15.19:0/3081177223 wait complete. 2024-09-12T03:26:17.031 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e82c68e9', quiesce took 54.0 seconds. Will release after: -33.9 2024-09-12T03:26:17.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:17.299 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:17.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:17 smithi121 ceph-mon[34963]: pgmap v554: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-09-12T03:26:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:17 smithi019 ceph-mon[30953]: pgmap v554: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-09-12T03:26:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:17 smithi135 ceph-mon[35474]: pgmap v554: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 446 op/s 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 -- 172.21.15.19:0/2774764850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 msgr2=0x7f919010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 --2- 172.21.15.19:0/2774764850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919010ea90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9188009ca0 tx=0x7f918802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 -- 172.21.15.19:0/2774764850 shutdown_connections 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 --2- 172.21.15.19:0/2774764850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919010ea90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 --2- 172.21.15.19:0/2774764850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 0x7f91901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 --2- 172.21.15.19:0/2774764850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f91901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 -- 172.21.15.19:0/2774764850 >> 172.21.15.19:0/2774764850 conn(0x7f91900fd8c0 msgr2=0x7f91900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 -- 172.21.15.19:0/2774764850 shutdown_connections 2024-09-12T03:26:17.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.792+0000 7f919829e640 1 -- 172.21.15.19:0/2774764850 wait complete. 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 Processor -- start 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 -- start start 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f919019a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 0x7f919019ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9190111f70 con 0x7f9190106f10 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9190111df0 con 0x7f9190101de0 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919829e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f91901120f0 con 0x7f9190102790 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919683b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919683b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35342/0 (socket says 172.21.15.19:35342) 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919683b640 1 -- 172.21.15.19:0/1978323219 learned_addr learned my addr 172.21.15.19:0/1978323219 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:17.828 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f919603a640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f919019a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.793+0000 7f9195839640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 0x7f919019ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 -- 172.21.15.19:0/1978323219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 msgr2=0x7f919019ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 0x7f919019ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 -- 172.21.15.19:0/1978323219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 msgr2=0x7f919019a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f919019a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 -- 172.21.15.19:0/1978323219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919019f680 con 0x7f9190106f10 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919603a640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f919019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919683b640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9188031a20 tx=0x7f9188031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188003c30 con 0x7f9190106f10 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9188003dd0 con 0x7f9190106f10 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919019f910 con 0x7f9190106f10 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918802fc40 con 0x7f9190106f10 2024-09-12T03:26:17.829 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.794+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91901a7250 con 0x7f9190106f10 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.795+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91901021c0 con 0x7f9190106f10 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.797+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9188004050 con 0x7f9190106f10 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.797+0000 7f91877fe640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 0x7f917c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.797+0000 7f919603a640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 0x7f917c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.797+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f91880f02d0 con 0x7f9190106f10 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.798+0000 7f919603a640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 0x7f917c0a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f918c009fc0 tx=0x7f918c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:17.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:17.801+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f91880bd360 con 0x7f9190106f10 2024-09-12T03:26:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.010+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91901a7420 con 0x7f9190106f10 2024-09-12T03:26:18.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.011+0000 7f91877fe640 1 -- 172.21.15.19:0/1978323219 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12002 (secure 0 0 0) 0x7f91880bcc60 con 0x7f9190106f10 2024-09-12T03:26:18.012 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:26:18.012 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"btime":"2024-09-12T03:24:43:270632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:24:43.042444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:26:18.012 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-09-12T03:26:18.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 msgr2=0x7f917c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 0x7f917c0a50c0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f918c009fc0 tx=0x7f918c005f90 comp rx=0 tx=0).stop 2024-09-12T03:26:18.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 msgr2=0x7f919019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:18.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9188031a20 tx=0x7f9188031a50 comp rx=0 tx=0).stop 2024-09-12T03:26:18.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 shutdown_connections 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f917c0a2c00 0x7f917c0a50c0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9190106f10 0x7f919019efa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9190102790 0x7f919019ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 --2- 172.21.15.19:0/1978323219 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9190101de0 0x7f919019a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 >> 172.21.15.19:0/1978323219 conn(0x7f91900fd8c0 msgr2=0x7f91900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 shutdown_connections 2024-09-12T03:26:18.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:18.014+0000 7f919829e640 1 -- 172.21.15.19:0/1978323219 wait complete. 2024-09-12T03:26:18.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:18 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:18.261 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:18 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1978323219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:18.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:18 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:18.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:18 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1978323219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:18 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:18.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:18 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1978323219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:18.702 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id e82c68e9 --release --await' 2024-09-12T03:26:18.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id e82c68e9 --release --await 2024-09-12T03:26:18.958 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:19.361 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:19 smithi019 ceph-mon[30953]: pgmap v555: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2024-09-12T03:26:19.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:19 smithi121 ceph-mon[34963]: pgmap v555: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2024-09-12T03:26:19.507 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:19 smithi135 ceph-mon[35474]: pgmap v555: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 444 op/s 2024-09-12T03:26:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 -- 172.21.15.19:0/1373378041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 msgr2=0x7f1be01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 --2- 172.21.15.19:0/1373378041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be01021c0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009900 tx=0x7f1bd002a410 comp rx=0 tx=0).stop 2024-09-12T03:26:19.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 -- 172.21.15.19:0/1373378041 shutdown_connections 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 --2- 172.21.15.19:0/1373378041 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0106f10 0x7f1be010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 --2- 172.21.15.19:0/1373378041 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 0x7f1be01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 --2- 172.21.15.19:0/1373378041 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be01021c0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 -- 172.21.15.19:0/1373378041 >> 172.21.15.19:0/1373378041 conn(0x7f1be00fd8c0 msgr2=0x7f1be00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 -- 172.21.15.19:0/1373378041 shutdown_connections 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 -- 172.21.15.19:0/1373378041 wait complete. 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.592+0000 7f1be5c67640 1 Processor -- start 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 -- start start 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 0x7f1be0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0106f10 0x7f1be0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0111f30 con 0x7f1be0101de0 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0111db0 con 0x7f1be0102790 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1be5c67640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01120b0 con 0x7f1be0106f10 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35346/0 (socket says 172.21.15.19:35346) 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3552320016 learned_addr learned my addr 172.21.15.19:0/3552320016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3552320016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0106f10 msgr2=0x7f1be0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0106f10 0x7f1be0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 0x7f1be0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3552320016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 msgr2=0x7f1be0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 0x7f1be0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3552320016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be01a6870 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.593+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006010 tx=0x7f1bd00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.594+0000 7f1bdcff9640 1 -- 172.21.15.19:0/3552320016 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd003f030 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.594+0000 7f1bdcff9640 1 -- 172.21.15.19:0/3552320016 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd0003ce0 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.594+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be01a6b00 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.594+0000 7f1bdcff9640 1 -- 172.21.15.19:0/3552320016 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0035070 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.594+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01a7030 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.596+0000 7f1bdcff9640 1 -- 172.21.15.19:0/3552320016 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bd0042030 con 0x7f1be0101de0 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.596+0000 7f1be8040740 1 Processor -- start 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.596+0000 7f1be8040740 1 -- start start 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdcff9640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 0x7f1bc80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d715620 con 0x55cd8d7bb240 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d6bce50 con 0x55cd8d7b0980 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1be8040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d7115e0 con 0x55cd8d7b0060 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 0x7f1bc80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35358/0 (socket says 172.21.15.19:35358) 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 -- 172.21.15.19:0/2595334297 learned_addr learned my addr 172.21.15.19:0/2595334297 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdcff9640 1 -- 172.21.15.19:0/3552320016 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bd00ef690 con 0x7f1be0101de0 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 -- 172.21.15.19:0/2595334297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0060 msgr2=0x55cd8d7b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 -- 172.21.15.19:0/2595334297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 msgr2=0x55cd8d7bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 -- 172.21.15.19:0/2595334297 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8d6add40 con 0x55cd8d7bb240 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdffff640 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8013a20 tx=0x7f1bd8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997826f5d8ad8e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.597+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 0x7f1bc80a50c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005fd0 tx=0x7f1bd4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1bc5ffb640 1 -- 172.21.15.19:0/2595334297 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004d20 con 0x55cd8d7bb240 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1bc5ffb640 1 -- 172.21.15.19:0/2595334297 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8003e50 con 0x55cd8d7bb240 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1bc5ffb640 1 -- 172.21.15.19:0/2595334297 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8004020 con 0x55cd8d7bb240 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 -- 172.21.15.19:0/2595334297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 msgr2=0x55cd8d7bd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8013a20 tx=0x7f1bd8009590 comp rx=0 tx=0).stop 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 -- 172.21.15.19:0/2595334297 shutdown_connections 2024-09-12T03:26:19.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7bb240 0x55cd8d7bd630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 --2- 172.21.15.19:0/2595334297 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 -- 172.21.15.19:0/2595334297 >> 172.21.15.19:0/2595334297 conn(0x55cd8d7ac5f0 msgr2=0x55cd8d7ac9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 -- 172.21.15.19:0/2595334297 shutdown_connections 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.598+0000 7f1be8040740 1 -- 172.21.15.19:0/2595334297 wait complete. 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 -- start start 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 Processor -- start 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7b8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7b8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d7bde10 con 0x55cd8d7b0060 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d69df10 con 0x55cd8d7b0980 2024-09-12T03:26:19.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55cd8d7bdf90 con 0x55cd8d7bb240 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1be8040740 10 client.0 _ll_register_callbacks cb 0x55cd8d6c1db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35360/0 (socket says 172.21.15.19:35360) 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.599+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3238855950 learned_addr learned my addr 172.21.15.19:0/3238855950 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7b8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7b8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 msgr2=0x55cd8d7b8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7b8c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3238855950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 msgr2=0x55cd8d7b8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7b8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3238855950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8d424580 con 0x55cd8d7b0060 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:19.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00bbcc0 tx=0x7f1bd00bbe10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5893e6c449d8a3c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0049070 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bd00bc220 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd8003e50 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1bd003bcb0 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1be8040740 10 client.16689 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd8d4ee960 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.600+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd8003e50 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1bd00be030 con 0x55cd8d7b0060 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 10 client.16689 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 10 client.16689 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 20 client.16689 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:19.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d7cae20 0x55cd8d7cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 4 client.16689 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd8d722b90 con 0x55cd8d7cae20 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d7cae20 0x55cd8d7cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d7cae20 0x55cd8d7cd2a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b42a18a3a07382 server_cookie=b20721041bf75af5 in_seq=0 out_seq=0 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.601+0000 7f1bbeffd640 10 client.16689 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3238855950 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd8d722b90 con 0x55cd8d7cae20 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1bbeffd640 10 client.16689 handle_command_reply: tid=0 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 client.16689 shutdown 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 msgr2=0x55cd8d7b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00bbcc0 tx=0x7f1bd00bbe10 comp rx=0 tx=0).stop 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 shutdown_connections 2024-09-12T03:26:19.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7b8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d7cae20 0x55cd8d7cd2a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7b8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 --2- 172.21.15.19:0/3238855950 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b81f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 >> 172.21.15.19:0/3238855950 conn(0x55cd8d7ac5f0 msgr2=0x55cd8d7b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1bbeffd640 0 client.16689 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 shutdown_connections 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.602+0000 7f1be8040740 1 -- 172.21.15.19:0/3238855950 wait complete. 2024-09-12T03:26:19.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.603+0000 7f1be8040740 20 client.16689 trim_cache size 0 max 16384 2024-09-12T03:26:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.619+0000 7f1be8040740 1 Processor -- start 2024-09-12T03:26:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 -- start start 2024-09-12T03:26:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0980 0x55cd8d7d2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 0x55cd8d7d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01af720 con 0x55cd8d7b0060 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b13e0 con 0x55cd8d7bb240 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1be8040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01afdf0 con 0x55cd8d7b0980 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35366/0 (socket says 172.21.15.19:35366) 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/275084552 learned_addr learned my addr 172.21.15.19:0/275084552 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0980 0x55cd8d7d2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdffff640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 0x55cd8d7d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/275084552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0980 msgr2=0x55cd8d7d2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0980 0x55cd8d7d2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/275084552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 msgr2=0x55cd8d7d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 0x55cd8d7d30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/275084552 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd8d7b07b0 con 0x55cd8d7b0060 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.620+0000 7f1bdffff640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 0x55cd8d7d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00b8990 tx=0x7f1bd00ba4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181d6ce4073a804b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1bc57fa640 1 -- 172.21.15.19:0/275084552 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0049070 con 0x55cd8d7b0060 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1bc57fa640 1 -- 172.21.15.19:0/275084552 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd00bc6c0 con 0x55cd8d7b0060 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1bc57fa640 1 -- 172.21.15.19:0/275084552 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00bc480 con 0x55cd8d7b0060 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 -- 172.21.15.19:0/275084552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 msgr2=0x55cd8d7b8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00b8990 tx=0x7f1bd00ba4d0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 -- 172.21.15.19:0/275084552 shutdown_connections 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7bb240 0x55cd8d7d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7b0980 0x55cd8d7d2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 --2- 172.21.15.19:0/275084552 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7b8820 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.621+0000 7f1be8040740 1 -- 172.21.15.19:0/275084552 >> 172.21.15.19:0/275084552 conn(0x55cd8d7ac5f0 msgr2=0x55cd8d7ba270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.622+0000 7f1be8040740 1 -- 172.21.15.19:0/275084552 shutdown_connections 2024-09-12T03:26:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.622+0000 7f1be8040740 1 -- 172.21.15.19:0/275084552 wait complete. 2024-09-12T03:26:19.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.622+0000 7f1be8040740 1 -- start start 2024-09-12T03:26:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.622+0000 7f1be8040740 1 Processor -- start 2024-09-12T03:26:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b0c50 con 0x55cd8d7b0060 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01b0580 con 0x55cd8d7b0980 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01ad640 con 0x55cd8d7bb240 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1be8040740 10 client.0 _ll_register_callbacks cb 0x7f1bd0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35380/0 (socket says 172.21.15.19:35380) 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3697896374 learned_addr learned my addr 172.21.15.19:0/3697896374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdffff640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 msgr2=0x55cd8d7c00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7c00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3697896374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 msgr2=0x55cd8d7bfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:19.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bfb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 -- 172.21.15.19:0/3697896374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd8d7c0790 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bdf7fe640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1bd003d040 tx=0x7f1bd00c4da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65a61e9763d923c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.623+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00aa070 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1bd00bc220 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd00bc6c0 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1bd00b6050 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 10 client.16695 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cd8d7c0790 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bd00bc6c0 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f1bd00be070 con 0x55cd8d7b0060 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 10 client.16695 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:19.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 10 client.16695 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 20 client.16695 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d836d00 0x55cd8d839120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 4 client.16695 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e82c68e9", "release": true, "await": true}] 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.624+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e82c68e9", "release": true, "await": true}) -- 0x55cd8d71d0a0 con 0x55cd8d836d00 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.625+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d836d00 0x55cd8d839120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.625+0000 7f1bbeffd640 10 client.16695 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:19.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:19.625+0000 7f1bdeffd640 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d836d00 0x55cd8d839120 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a937e82a85ff7c8 server_cookie=d5792f20b828ee00 in_seq=0 out_seq=0 2024-09-12T03:26:20.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.703+0000 7f1bbeffd640 1 -- 172.21.15.19:0/3697896374 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cd8d71d0a0 con 0x55cd8d836d00 2024-09-12T03:26:20.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.703+0000 7f1bbeffd640 10 client.16695 handle_command_reply: tid=0 2024-09-12T03:26:20.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.703+0000 7f1be8040740 1 client.16695 shutdown 2024-09-12T03:26:20.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.703+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 msgr2=0x55cd8d7c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.703+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f1bd003d040 tx=0x7f1bd00c4da0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 shutdown_connections 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55cd8d7bb240 0x55cd8d7c00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55cd8d836d00 0x55cd8d839120 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55cd8d7b0980 0x55cd8d7bfb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 --2- 172.21.15.19:0/3697896374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55cd8d7b0060 0x55cd8d7c3820 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1bbeffd640 0 client.16695 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 >> 172.21.15.19:0/3697896374 conn(0x55cd8d7ac5f0 msgr2=0x55cd8d7ab750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 shutdown_connections 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 1 -- 172.21.15.19:0/3697896374 wait complete. 2024-09-12T03:26:20.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.704+0000 7f1be8040740 20 client.16695 trim_cache size 0 max 16384 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 36, 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 24, 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "e82c68e9": { 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 24, 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:26:20.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 140.1, 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:20.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:20.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:20.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:20.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:20.708 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 msgr2=0x7f1bc80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 0x7f1bc80a50c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005fd0 tx=0x7f1bd4005850 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 msgr2=0x7f1be00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0006010 tx=0x7f1bd00042f0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 shutdown_connections 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1be0106f10 0x7f1be0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1be0102790 0x7f1be0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f1bc80a2c00 0x7f1bc80a50c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 --2- 172.21.15.19:0/3552320016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1be0101de0 0x7f1be00744f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:20.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 >> 172.21.15.19:0/3552320016 conn(0x7f1be00fd8c0 msgr2=0x7f1be00febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 shutdown_connections 2024-09-12T03:26:20.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:20.705+0000 7f1be5c67640 1 -- 172.21.15.19:0/3552320016 wait complete. 2024-09-12T03:26:21.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:21 smithi121 ceph-mon[34963]: pgmap v556: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 486 op/s 2024-09-12T03:26:21.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:21 smithi019 ceph-mon[30953]: pgmap v556: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 486 op/s 2024-09-12T03:26:21.536 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e82c68e9', total seconds elapsed: 58.5 2024-09-12T03:26:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:21 smithi135 ceph-mon[35474]: pgmap v556: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.5 MiB/s wr, 486 op/s 2024-09-12T03:26:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:22 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:22 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:22 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:22 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:22 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:22 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:22 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:22 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:22.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:22 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:22 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:22 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:22 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:22 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:22 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:22.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:22 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:23.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:23 smithi121 ceph-mon[34963]: pgmap v557: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-12T03:26:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:23 smithi019 ceph-mon[30953]: pgmap v557: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-12T03:26:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:23 smithi135 ceph-mon[35474]: pgmap v557: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.1 MiB/s wr, 437 op/s 2024-09-12T03:26:25.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:25 smithi121 ceph-mon[34963]: pgmap v558: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 897 KiB/s wr, 419 op/s 2024-09-12T03:26:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:25 smithi019 ceph-mon[30953]: pgmap v558: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 897 KiB/s wr, 419 op/s 2024-09-12T03:26:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:25 smithi135 ceph-mon[35474]: pgmap v558: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 897 KiB/s wr, 419 op/s 2024-09-12T03:26:27.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:27 smithi121 ceph-mon[34963]: pgmap v559: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 928 KiB/s wr, 420 op/s 2024-09-12T03:26:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:27 smithi019 ceph-mon[30953]: pgmap v559: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 928 KiB/s wr, 420 op/s 2024-09-12T03:26:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:27 smithi135 ceph-mon[35474]: pgmap v559: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 928 KiB/s wr, 420 op/s 2024-09-12T03:26:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:29 smithi121 ceph-mon[34963]: pgmap v560: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 493 KiB/s wr, 196 op/s 2024-09-12T03:26:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:29 smithi019 ceph-mon[30953]: pgmap v560: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 493 KiB/s wr, 196 op/s 2024-09-12T03:26:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:29 smithi135 ceph-mon[35474]: pgmap v560: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 493 KiB/s wr, 196 op/s 2024-09-12T03:26:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:31 smithi121 ceph-mon[34963]: pgmap v561: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 585 KiB/s wr, 137 op/s 2024-09-12T03:26:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:31 smithi019 ceph-mon[30953]: pgmap v561: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 585 KiB/s wr, 137 op/s 2024-09-12T03:26:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:31 smithi135 ceph-mon[35474]: pgmap v561: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 585 KiB/s wr, 137 op/s 2024-09-12T03:26:33.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:33 smithi121 ceph-mon[34963]: pgmap v562: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 KiB/s wr, 95 op/s 2024-09-12T03:26:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:33 smithi019 ceph-mon[30953]: pgmap v562: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 KiB/s wr, 95 op/s 2024-09-12T03:26:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:33 smithi135 ceph-mon[35474]: pgmap v562: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 773 KiB/s wr, 95 op/s 2024-09-12T03:26:34.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:34 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:26:34.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:34 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:26:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:34 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:26:34.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:34 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:26:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:34 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:26:34.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:34 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:26:34.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:26:34.996 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:35 smithi121 ceph-mon[34963]: pgmap v563: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 999 KiB/s wr, 94 op/s 2024-09-12T03:26:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:35 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:26:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:35 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:26:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:35 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:26:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:35 smithi019 ceph-mon[30953]: pgmap v563: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 999 KiB/s wr, 94 op/s 2024-09-12T03:26:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:35 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:26:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:35 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:26:35.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:35 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:26:35.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 -- 172.21.15.19:0/1118456773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc102ef0 msgr2=0x7f32cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 --2- 172.21.15.19:0/1118456773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc102ef0 0x7f32cc10da40 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009ca0 tx=0x7f32c402a950 comp rx=0 tx=0).stop 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 -- 172.21.15.19:0/1118456773 shutdown_connections 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 --2- 172.21.15.19:0/1118456773 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc102ef0 0x7f32cc10da40 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 --2- 172.21.15.19:0/1118456773 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102570 0x7f32cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 --2- 172.21.15.19:0/1118456773 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc100e20 0x7f32cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 -- 172.21.15.19:0/1118456773 >> 172.21.15.19:0/1118456773 conn(0x7f32cc0fc7d0 msgr2=0x7f32cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 -- 172.21.15.19:0/1118456773 shutdown_connections 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.506+0000 7f32d35d1640 1 -- 172.21.15.19:0/1118456773 wait complete. 2024-09-12T03:26:35.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 Processor -- start 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 -- start start 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc102570 0x7f32cc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 0x7f32cc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110f80 con 0x7f32cc100e20 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc110e00 con 0x7f32cc102570 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d35d1640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc111100 con 0x7f32cc102ef0 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.507+0000 7f32d136d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51832/0 (socket says 172.21.15.19:51832) 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 -- 172.21.15.19:0/3844533054 learned_addr learned my addr 172.21.15.19:0/3844533054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:35.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc102570 0x7f32cc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 0x7f32cc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 -- 172.21.15.19:0/3844533054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 msgr2=0x7f32cc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 0x7f32cc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 -- 172.21.15.19:0/3844533054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc102570 msgr2=0x7f32cc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc102570 0x7f32cc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 -- 172.21.15.19:0/3844533054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc1a6990 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 0x7f32cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d136d640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d950 tx=0x7f32c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32c27fc640 1 -- 172.21.15.19:0/3844533054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8017070 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a6c80 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32c27fc640 1 -- 172.21.15.19:0/3844533054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c8009c30 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32c27fc640 1 -- 172.21.15.19:0/3844533054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c800f6b0 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.508+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a71c0 con 0x7f32cc100e20 2024-09-12T03:26:35.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.511+0000 7f32c27fc640 1 -- 172.21.15.19:0/3844533054 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32c80109d0 con 0x7f32cc100e20 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.511+0000 7f32c27fc640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.512+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.512+0000 7f32c27fc640 1 -- 172.21.15.19:0/3844533054 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32c80cece0 con 0x7f32cc100e20 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.512+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0fe710 tx=0x7f32bc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.512+0000 7f32d59aa740 1 Processor -- start 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.512+0000 7f32d59aa740 1 -- start start 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff124a0 0x55e67ff12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e67fe782e0 con 0x55e67ff12dc0 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e67fe1fce0 con 0x55e67ff1d680 2024-09-12T03:26:35.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d59aa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e67fe742a0 con 0x55e67ff124a0 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51844/0 (socket says 172.21.15.19:51844) 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2046785745 learned_addr learned my addr 172.21.15.19:0/2046785745 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d136d640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff124a0 0x55e67ff12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2046785745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff124a0 msgr2=0x55e67ff12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff124a0 0x55e67ff12880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2046785745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 msgr2=0x55e67ff1fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2046785745 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e67fe04440 con 0x55e67ff12dc0 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.513+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:35.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.514+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f32bc010f80 tx=0x7f32bc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54409dbf6b9d8d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.514+0000 7f32b77fe640 1 -- 172.21.15.19:0/2046785745 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc01e620 con 0x55e67ff12dc0 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.514+0000 7f32b77fe640 1 -- 172.21.15.19:0/2046785745 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc01eca0 con 0x55e67ff12dc0 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.514+0000 7f32b77fe640 1 -- 172.21.15.19:0/2046785745 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc026d30 con 0x55e67ff12dc0 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- 172.21.15.19:0/2046785745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 msgr2=0x55e67ff1d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f32bc010f80 tx=0x7f32bc013000 comp rx=0 tx=0).stop 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- 172.21.15.19:0/2046785745 shutdown_connections 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1d140 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2046785745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff124a0 0x55e67ff12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- 172.21.15.19:0/2046785745 >> 172.21.15.19:0/2046785745 conn(0x55e67ff0ea30 msgr2=0x55e67ff0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- 172.21.15.19:0/2046785745 shutdown_connections 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- 172.21.15.19:0/2046785745 wait complete. 2024-09-12T03:26:35.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 -- start start 2024-09-12T03:26:35.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.515+0000 7f32d59aa740 1 Processor -- start 2024-09-12T03:26:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 0x55e67ff1a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e67ff20250 con 0x55e67ff124a0 2024-09-12T03:26:35.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e67fe33050 con 0x55e67ff1d680 2024-09-12T03:26:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e67ff203d0 con 0x55e67ff12dc0 2024-09-12T03:26:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d59aa740 10 client.0 _ll_register_callbacks cb 0x55e67fe73160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51860/0 (socket says 172.21.15.19:51860) 2024-09-12T03:26:35.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 -- 172.21.15.19:0/3729001440 learned_addr learned my addr 172.21.15.19:0/3729001440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 0x55e67ff1a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 -- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 msgr2=0x55e67ff1a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 0x55e67ff1a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 -- 172.21.15.19:0/3729001440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 msgr2=0x55e67ff1a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d136d640 1 -- 172.21.15.19:0/3729001440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e67fbe93c0 con 0x55e67ff124a0 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 0x55e67ff1a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:35.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.516+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d136d640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f32c809b1f0 tx=0x7f32c809af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5ea1c0b423f960 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8020040 con 0x55e67ff124a0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32c809a940 con 0x55e67ff124a0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc01eca0 con 0x55e67ff124a0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32c809b4f0 con 0x55e67ff124a0 2024-09-12T03:26:35.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 10 client.16707 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e67ff2bc70 con 0x55e67ff124a0 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c809f8a0 con 0x55e67ff124a0 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f32c809a3b0 con 0x55e67ff124a0 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 10 client.16707 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 20 client.16707 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff2d260 0x55e67ff2f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 4 client.16707 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.517+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32bc01eca0 con 0x55e67ff2d260 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.518+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff2d260 0x55e67ff2f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.518+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff2d260 0x55e67ff2f680 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e4b9a6151dce230 server_cookie=74ddb6163c45e620 in_seq=0 out_seq=0 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.518+0000 7f32b4ff9640 10 client.16707 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:35.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32b4ff9640 1 -- 172.21.15.19:0/3729001440 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f32bc01eca0 con 0x55e67ff2d260 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32b4ff9640 10 client.16707 handle_command_reply: tid=0 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 client.16707 shutdown 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 msgr2=0x55e67ff1d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f32c809b1f0 tx=0x7f32c809af20 comp rx=0 tx=0).stop 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 shutdown_connections 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff2d260 0x55e67ff2f680 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff1d680 0x55e67ff1a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32b4ff9640 0 client.16707 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff12dc0 0x55e67ff1a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 --2- 172.21.15.19:0/3729001440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff124a0 0x55e67ff1d010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 >> 172.21.15.19:0/3729001440 conn(0x55e67ff0ea30 msgr2=0x55e67ff14710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 shutdown_connections 2024-09-12T03:26:35.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 1 -- 172.21.15.19:0/3729001440 wait complete. 2024-09-12T03:26:35.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.519+0000 7f32d59aa740 20 client.16707 trim_cache size 0 max 16384 2024-09-12T03:26:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 Processor -- start 2024-09-12T03:26:35.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 -- start start 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff294e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff1c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1af940 con 0x55e67ff12dc0 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b1600 con 0x55e67ff124a0 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.537+0000 7f32d59aa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b0010 con 0x55e67ff1d680 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51874/0 (socket says 172.21.15.19:51874) 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2679427409 learned_addr learned my addr 172.21.15.19:0/2679427409 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff1c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d136d640 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff294e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2679427409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 msgr2=0x55e67ff1c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff1c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2679427409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 msgr2=0x55e67ff294e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff294e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 -- 172.21.15.19:0/2679427409 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e67ff0f630 con 0x55e67ff12dc0 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32bc011ca0 tx=0x7f32bc026690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c263274a6dc79145 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32b6ffd640 1 -- 172.21.15.19:0/2679427409 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc026c10 con 0x55e67ff12dc0 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.538+0000 7f32b6ffd640 1 -- 172.21.15.19:0/2679427409 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc01ee50 con 0x55e67ff12dc0 2024-09-12T03:26:35.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32b6ffd640 1 -- 172.21.15.19:0/2679427409 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc032440 con 0x55e67ff12dc0 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 -- 172.21.15.19:0/2679427409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 msgr2=0x55e67ff1c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f32bc011ca0 tx=0x7f32bc026690 comp rx=0 tx=0).stop 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 -- 172.21.15.19:0/2679427409 shutdown_connections 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff1c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff1c230 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 --2- 172.21.15.19:0/2679427409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff294e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 -- 172.21.15.19:0/2679427409 >> 172.21.15.19:0/2679427409 conn(0x55e67ff0ea30 msgr2=0x55e67ff1bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 -- 172.21.15.19:0/2679427409 shutdown_connections 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.539+0000 7f32d59aa740 1 -- 172.21.15.19:0/2679427409 wait complete. 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 -- start start 2024-09-12T03:26:35.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 Processor -- start 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff25f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff239d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b0e70 con 0x55e67ff12dc0 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1b07a0 con 0x55e67ff124a0 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc1ad860 con 0x55e67ff1d680 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d59aa740 10 client.0 _ll_register_callbacks cb 0x7f32c809b300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.540+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51886/0 (socket says 172.21.15.19:51886) 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 -- 172.21.15.19:0/1258345789 learned_addr learned my addr 172.21.15.19:0/1258345789 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d1b6e640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff239d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d136d640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff25f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 -- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 msgr2=0x55e67ff239d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff239d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 -- 172.21.15.19:0/1258345789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 msgr2=0x55e67ff25f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff25f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 -- 172.21.15.19:0/1258345789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e67ff96f30 con 0x55e67ff12dc0 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d0b6c640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f32bc021040 tx=0x7f32bc010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a1537b8e5d1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc026ce0 con 0x55e67ff12dc0 2024-09-12T03:26:35.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f32bc01e420 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32bc01ee50 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.541+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f32bc005850 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 10 client.16713 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e67ff980e0 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32bc01ee50 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f32bc032710 con 0x55e67ff12dc0 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 10 client.16713 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 20 client.16713 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff98720 0x55e67ff9ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 4 client.16713 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:26:35.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e67fe85850 con 0x55e67ff98720 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.542+0000 7f32d136d640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff98720 0x55e67ff9ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d136d640 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff98720 0x55e67ff9ab40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c77bf44c17e5a45 server_cookie=6a75228bb68b1444 in_seq=0 out_seq=0 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32b4ff9640 10 client.16713 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32b4ff9640 1 -- 172.21.15.19:0/1258345789 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e67fe85850 con 0x55e67ff98720 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32b4ff9640 10 client.16713 handle_command_reply: tid=0 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 client.16713 shutdown 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 msgr2=0x55e67ff23290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f32bc021040 tx=0x7f32bc010980 comp rx=0 tx=0).stop 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 shutdown_connections 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e67ff1d680 0x55e67ff239d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e67ff12dc0 0x55e67ff23290 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55e67ff98720 0x55e67ff9ab40 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 --2- 172.21.15.19:0/1258345789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e67ff124a0 0x55e67ff25f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32b4ff9640 0 client.16713 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 >> 172.21.15.19:0/1258345789 conn(0x55e67ff0ea30 msgr2=0x55e67ff1c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.543+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 shutdown_connections 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d59aa740 1 -- 172.21.15.19:0/1258345789 wait complete. 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d59aa740 20 client.16713 trim_cache size 0 max 16384 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:26:35.549 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 msgr2=0x7f32b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f32cc0fe710 tx=0x7f32bc00b410 comp rx=0 tx=0).stop 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 msgr2=0x7f32cc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f32c800d950 tx=0x7f32c800de10 comp rx=0 tx=0).stop 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 shutdown_connections 2024-09-12T03:26:35.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32cc102ef0 0x7f32cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32cc102570 0x7f32cc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 --2- 172.21.15.19:0/3844533054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32cc100e20 0x7f32cc0fcc80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.544+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 >> 172.21.15.19:0/3844533054 conn(0x7f32cc0fc7d0 msgr2=0x7f32cc06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.545+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 shutdown_connections 2024-09-12T03:26:35.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:35.545+0000 7f32d35d1640 1 -- 172.21.15.19:0/3844533054 wait complete. 2024-09-12T03:26:35.551 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:35 smithi135 ceph-mon[35474]: pgmap v563: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 999 KiB/s wr, 94 op/s 2024-09-12T03:26:35.551 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:35 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:26:35.551 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:35 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:26:35.552 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:35 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:26:36.421 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:26:36.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:36.682 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 -- 172.21.15.19:0/3533571975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c10e4f0 msgr2=0x7f259c110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 --2- 172.21.15.19:0/3533571975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c10e4f0 0x7f259c110900 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f258c009ca0 tx=0x7f258c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 -- 172.21.15.19:0/3533571975 shutdown_connections 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 --2- 172.21.15.19:0/3533571975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c10e4f0 0x7f259c110900 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 --2- 172.21.15.19:0/3533571975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c101380 0x7f259c10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 --2- 172.21.15.19:0/3533571975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c1009d0 0x7f259c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 -- 172.21.15.19:0/3533571975 >> 172.21.15.19:0/3533571975 conn(0x7f259c0fc7d0 msgr2=0x7f259c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 -- 172.21.15.19:0/3533571975 shutdown_connections 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 -- 172.21.15.19:0/3533571975 wait complete. 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.189+0000 7f25a1145640 1 Processor -- start 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 -- start start 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c1009d0 0x7f259c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 0x7f259c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f259c114250 con 0x7f259c101380 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1140d0 con 0x7f259c1009d0 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f25a1145640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f259c1143d0 con 0x7f259c10e4f0 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51898/0 (socket says 172.21.15.19:51898) 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 -- 172.21.15.19:0/1388978696 learned_addr learned my addr 172.21.15.19:0/1388978696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259b59e640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 0x7f259c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259ad9d640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c1009d0 0x7f259c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 -- 172.21.15.19:0/1388978696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 msgr2=0x7f259c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 0x7f259c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 -- 172.21.15.19:0/1388978696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c1009d0 msgr2=0x7f259c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c1009d0 0x7f259c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259a59c640 1 -- 172.21.15.19:0/1388978696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f259c19f6a0 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.190+0000 7f259b59e640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 0x7f259c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f259a59c640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f258400c920 tx=0x7f258400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584007c70 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f259c19f990 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2584007e10 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2584018890 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.191+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f259c1a7270 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.192+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f259c06a7d0 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.193+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2584018a70 con 0x7f259c101380 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.194+0000 7f2593fff640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 0x7f25880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:37.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.194+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f25840cf130 con 0x7f259c101380 2024-09-12T03:26:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.194+0000 7f259ad9d640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 0x7f25880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.194+0000 7f259ad9d640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 0x7f25880a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25940058c0 tx=0x7f259400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:37.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.198+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2584014030 con 0x7f259c101380 2024-09-12T03:26:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:37 smithi121 ceph-mon[34963]: pgmap v564: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-12T03:26:37.410 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.410+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f259c109680 con 0x7f259c101380 2024-09-12T03:26:37.410 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:37 smithi019 ceph-mon[30953]: pgmap v564: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-12T03:26:37.411 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.411+0000 7f2593fff640 1 -- 172.21.15.19:0/1388978696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12002 (secure 0 0 0) 0x7f258409c0a0 con 0x7f259c101380 2024-09-12T03:26:37.411 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:26:37.411 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"btime":"2024-09-12T03:24:43:270632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:24:43.042444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:26:37.412 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-09-12T03:26:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 msgr2=0x7f25880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 0x7f25880a5040 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f25940058c0 tx=0x7f259400a250 comp rx=0 tx=0).stop 2024-09-12T03:26:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 msgr2=0x7f259c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f258400c920 tx=0x7f258400cde0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 shutdown_connections 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f259c10e4f0 0x7f259c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f25880a2b80 0x7f25880a5040 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f259c101380 0x7f259c19ac30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 --2- 172.21.15.19:0/1388978696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f259c1009d0 0x7f259c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.413+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 >> 172.21.15.19:0/1388978696 conn(0x7f259c0fc7d0 msgr2=0x7f259c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.414+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 shutdown_connections 2024-09-12T03:26:37.414 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:37.414+0000 7f25a1145640 1 -- 172.21.15.19:0/1388978696 wait complete. 2024-09-12T03:26:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:37 smithi135 ceph-mon[35474]: pgmap v564: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 121 op/s 2024-09-12T03:26:38.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:26:38.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1388978696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:38 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1388978696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:38 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.492 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1388978696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:26:38.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:38 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:26:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 -- 172.21.15.19:0/3392214551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 msgr2=0x7f99201021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 --2- 172.21.15.19:0/3392214551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99201021b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9910009900 tx=0x7f991002a410 comp rx=0 tx=0).stop 2024-09-12T03:26:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 -- 172.21.15.19:0/3392214551 shutdown_connections 2024-09-12T03:26:39.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 --2- 172.21.15.19:0/3392214551 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f992010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 --2- 172.21.15.19:0/3392214551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f99201067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 --2- 172.21.15.19:0/3392214551 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99201021b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 -- 172.21.15.19:0/3392214551 >> 172.21.15.19:0/3392214551 conn(0x7f99200fd890 msgr2=0x7f99200ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 -- 172.21.15.19:0/3392214551 shutdown_connections 2024-09-12T03:26:39.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.984+0000 7f9925214640 1 -- 172.21.15.19:0/3392214551 wait complete. 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 Processor -- start 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 -- start start 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f9920074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f9920074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9920111fb0 con 0x7f9920101dd0 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9920111e30 con 0x7f9920102780 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9925214640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9920112130 con 0x7f9920106f00 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51908/0 (socket says 172.21.15.19:51908) 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 -- 172.21.15.19:0/1017472860 learned_addr learned my addr 172.21.15.19:0/1017472860 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991f59e640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f9920074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 -- 172.21.15.19:0/1017472860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 msgr2=0x7f9920074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f9920074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f991ed9d640 1 -- 172.21.15.19:0/1017472860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 msgr2=0x7f9920074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.985+0000 7f9917fff640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f9920074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991ed9d640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f9920074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991ed9d640 1 -- 172.21.15.19:0/1017472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99201a68b0 con 0x7f9920101dd0 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991f59e640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f9920074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f9917fff640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f9920074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991ed9d640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9910006010 tx=0x7f9910003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991cd99640 1 -- 172.21.15.19:0/1017472860 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991003f070 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99201a6b40 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991cd99640 1 -- 172.21.15.19:0/1017472860 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9910031c90 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f991cd99640 1 -- 172.21.15.19:0/1017472860 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910035030 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.986+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99201a7020 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.988+0000 7f991cd99640 1 -- 172.21.15.19:0/1017472860 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9910042070 con 0x7f9920101dd0 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.988+0000 7f99275ed740 1 Processor -- start 2024-09-12T03:26:39.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.988+0000 7f99275ed740 1 -- start start 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d13d8182f0 con 0x55d13d8b25d0 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d13d7bfcf0 con 0x55d13d8bd370 2024-09-12T03:26:39.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f99275ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d13d8142b0 con 0x55d13d8b3b40 2024-09-12T03:26:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51910/0 (socket says 172.21.15.19:51910) 2024-09-12T03:26:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 -- 172.21.15.19:0/3426182278 learned_addr learned my addr 172.21.15.19:0/3426182278 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f9917fff640 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991f59e640 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991cd99640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 0x7f990c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 -- 172.21.15.19:0/3426182278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 msgr2=0x55d13d8b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 -- 172.21.15.19:0/3426182278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 msgr2=0x55d13d8bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.989+0000 7f991ed9d640 1 -- 172.21.15.19:0/3426182278 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13d7a43a0 con 0x55d13d8b25d0 2024-09-12T03:26:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f991f59e640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 0x7f990c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f991cd99640 1 -- 172.21.15.19:0/1017472860 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f99100ef990 con 0x7f9920101dd0 2024-09-12T03:26:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f99100eb960 tx=0x7f99100e6d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47e82304932e7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f99157fa640 1 -- 172.21.15.19:0/3426182278 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100bbb60 con 0x55d13d8b25d0 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f991f59e640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 0x7f990c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9918005fd0 tx=0x7f9918004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f99157fa640 1 -- 172.21.15.19:0/3426182278 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99100ea8d0 con 0x55d13d8b25d0 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.990+0000 7f99157fa640 1 -- 172.21.15.19:0/3426182278 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100e9490 con 0x55d13d8b25d0 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- 172.21.15.19:0/3426182278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 msgr2=0x55d13d8b2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f99100eb960 tx=0x7f99100e6d90 comp rx=0 tx=0).stop 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- 172.21.15.19:0/3426182278 shutdown_connections 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8b2a10 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 --2- 172.21.15.19:0/3426182278 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- 172.21.15.19:0/3426182278 >> 172.21.15.19:0/3426182278 conn(0x55d13d87b310 msgr2=0x55d13d8aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- 172.21.15.19:0/3426182278 shutdown_connections 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- 172.21.15.19:0/3426182278 wait complete. 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 -- start start 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.991+0000 7f99275ed740 1 Processor -- start 2024-09-12T03:26:39.039 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 0x55d13d8ba5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d13d8bfe30 con 0x55d13d8b25d0 2024-09-12T03:26:39.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d13d7d3060 con 0x55d13d8b3b40 2024-09-12T03:26:39.040 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d13d8bffb0 con 0x55d13d8bd370 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 10 client.0 _ll_register_callbacks cb 0x55d13d5268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51914/0 (socket says 172.21.15.19:51914) 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 -- 172.21.15.19:0/3817937411 learned_addr learned my addr 172.21.15.19:0/3817937411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f9917fff640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991f59e640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 0x55d13d8ba5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 -- 172.21.15.19:0/3817937411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 msgr2=0x55d13d8ba5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 0x55d13d8ba5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 -- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 msgr2=0x55d13d8ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 -- 172.21.15.19:0/3817937411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13d5893c0 con 0x55d13d8b25d0 2024-09-12T03:26:39.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991f59e640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 0x55d13d8ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f991ed9d640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f99100c4940 tx=0x7f9910005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef27c3107a59c35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99067fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910049030 con 0x55d13d8b25d0 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f99100ba2a0 con 0x55d13d8b25d0 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.992+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d13d825860 con 0x55d13d8b25d0 2024-09-12T03:26:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99100bbb60 con 0x55d13d8b25d0 2024-09-12T03:26:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 10 client.16731 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d13d8bfe30 con 0x55d13d8b25d0 2024-09-12T03:26:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55d13d825860 con 0x55d13d8b25d0 2024-09-12T03:26:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f99100e9120 con 0x55d13d8b25d0 2024-09-12T03:26:39.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 10 client.16731 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 10 client.16731 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 20 client.16731 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d8ccae0 0x55d13d8cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 4 client.16731 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d13d81fd70 con 0x55d13d8ccae0 2024-09-12T03:26:39.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.993+0000 7f9917fff640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d8ccae0 0x55d13d8cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.994+0000 7f99067fc640 10 client.16731 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:26:39.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.994+0000 7f9917fff640 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d8ccae0 0x55d13d8cef20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9ede813bc8757dd9 server_cookie=dad0b1510b13255a in_seq=0 out_seq=0 2024-09-12T03:26:39.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.994+0000 7f99067fc640 1 -- 172.21.15.19:0/3817937411 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d13d81fd70 con 0x55d13d8ccae0 2024-09-12T03:26:39.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.994+0000 7f99067fc640 10 client.16731 handle_command_reply: tid=0 2024-09-12T03:26:39.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.994+0000 7f99275ed740 1 client.16731 shutdown 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 msgr2=0x55d13d8bcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f99100c4940 tx=0x7f9910005d50 comp rx=0 tx=0).stop 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 shutdown_connections 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d8ccae0 0x55d13d8cef20 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8bd370 0x55d13d8ba5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99067fc640 0 client.16731 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:26:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 --2- 172.21.15.19:0/3817937411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8bcd50 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 >> 172.21.15.19:0/3817937411 conn(0x55d13d87b310 msgr2=0x55d13d8b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 shutdown_connections 2024-09-12T03:26:39.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 1 -- 172.21.15.19:0/3817937411 wait complete. 2024-09-12T03:26:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:38.995+0000 7f99275ed740 20 client.16731 trim_cache size 0 max 16384 2024-09-12T03:26:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 Processor -- start 2024-09-12T03:26:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 -- start start 2024-09-12T03:26:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8b5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99201af810 con 0x55d13d8b25d0 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99201b14d0 con 0x55d13d8bd370 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f99275ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99201afee0 con 0x55d13d8b3b40 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51926/0 (socket says 172.21.15.19:51926) 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991ed9d640 1 -- 172.21.15.19:0/579581668 learned_addr learned my addr 172.21.15.19:0/579581668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f9917fff640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991ed9d640 1 -- 172.21.15.19:0/579581668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 msgr2=0x55d13d8b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991ed9d640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.008+0000 7f991f59e640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f991ed9d640 1 -- 172.21.15.19:0/579581668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 msgr2=0x55d13d8b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f991ed9d640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8b5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f991ed9d640 1 -- 172.21.15.19:0/579581668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d13d8ad310 con 0x55d13d8b25d0 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f9917fff640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f991ed9d640 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9910003fd0 tx=0x7f991003b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3264e77a33d3e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f9914ff9640 1 -- 172.21.15.19:0/579581668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910049070 con 0x55d13d8b25d0 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f9914ff9640 1 -- 172.21.15.19:0/579581668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99100bc900 con 0x55d13d8b25d0 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f9914ff9640 1 -- 172.21.15.19:0/579581668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99100f5310 con 0x55d13d8b25d0 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f99275ed740 1 -- 172.21.15.19:0/579581668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 msgr2=0x55d13d8c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.009+0000 7f99275ed740 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9910003fd0 tx=0x7f991003b920 comp rx=0 tx=0).stop 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- 172.21.15.19:0/579581668 shutdown_connections 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8bd370 0x55d13d8b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b3b40 0x55d13d8b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- 172.21.15.19:0/579581668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8b25d0 0x55d13d8c9090 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- 172.21.15.19:0/579581668 >> 172.21.15.19:0/579581668 conn(0x55d13d87b310 msgr2=0x55d13d8b90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- 172.21.15.19:0/579581668 shutdown_connections 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- 172.21.15.19:0/579581668 wait complete. 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- start start 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 Processor -- start 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b25d0 0x55d13d8c5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f99201b0d40 con 0x55d13d8bd370 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99201b0670 con 0x55d13d8b3b40 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99201ad730 con 0x55d13d8b25d0 2024-09-12T03:26:39.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f99275ed740 10 client.0 _ll_register_callbacks cb 0x7f990800b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f991f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f991f59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51940/0 (socket says 172.21.15.19:51940) 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f991f59e640 1 -- 172.21.15.19:0/926470749 learned_addr learned my addr 172.21.15.19:0/926470749 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.010+0000 7f9917fff640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991ed9d640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b25d0 0x55d13d8c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 -- 172.21.15.19:0/926470749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b25d0 msgr2=0x55d13d8c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b25d0 0x55d13d8c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 -- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 msgr2=0x55d13d8c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 -- 172.21.15.19:0/926470749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d13d8af960 con 0x55d13d8bd370 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f991f59e640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9918018e00 tx=0x7f9918004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27caf0cbd9e26970 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:39.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99067fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918028060 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99100bc900 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9918023600 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f991800fe30 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99275ed740 10 client.16737 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d13d937640 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.011+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99100bc900 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f991802e940 con 0x55d13d8bd370 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 10 client.16737 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 10 client.16737 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 20 client.16737 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d937da0 0x55d13d93a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:39.054 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 4 client.16737 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d13d79c170 con 0x55d13d937da0 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f991ed9d640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d937da0 0x55d13d93a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f991ed9d640 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d937da0 0x55d13d93a1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23e1b7cc604d2d12 server_cookie=1bdc9cbff1fbedbb in_seq=0 out_seq=0 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99067fc640 10 client.16737 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99067fc640 1 -- 172.21.15.19:0/926470749 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d13d79c170 con 0x55d13d937da0 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99067fc640 10 client.16737 handle_command_reply: tid=0 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 1 client.16737 shutdown 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 msgr2=0x55d13d8c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.012+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f9918018e00 tx=0x7f9918004ab0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 shutdown_connections 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d13d8bd370 0x55d13d8c1240 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d13d8b3b40 0x55d13d8c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55d13d937da0 0x55d13d93a1c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 --2- 172.21.15.19:0/926470749 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d13d8b25d0 0x55d13d8c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 >> 172.21.15.19:0/926470749 conn(0x55d13d87b310 msgr2=0x55d13d8ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99067fc640 0 client.16737 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 shutdown_connections 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 1 -- 172.21.15.19:0/926470749 wait complete. 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f99275ed740 20 client.16737 trim_cache size 0 max 16384 2024-09-12T03:26:39.056 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 msgr2=0x7f990c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 0x7f990c0a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9918005fd0 tx=0x7f9918004a40 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 msgr2=0x7f99200744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9910006010 tx=0x7f9910003ba0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 shutdown_connections 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9920106f00 0x7f9920074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9920102780 0x7f9920074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f990c0a2b80 0x7f990c0a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 --2- 172.21.15.19:0/1017472860 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9920101dd0 0x7f99200744f0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 >> 172.21.15.19:0/1017472860 conn(0x7f99200fd890 msgr2=0x7f99200febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:39.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 shutdown_connections 2024-09-12T03:26:39.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:39.013+0000 7f9925214640 1 -- 172.21.15.19:0/1017472860 wait complete. 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: pgmap v565: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]': finished 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: Unable to set osd_memory_target on smithi121 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-09-12T03:26:39.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:39.166 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: pgmap v565: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]': finished 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: Unable to set osd_memory_target on smithi121 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:39.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:26:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: pgmap v565: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 977 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-09-12T03:26:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd='[{"prefix": "config rm", "who": "osd/host:smithi135", "name": "osd_memory_target"}]': finished 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: Adjusting osd_memory_target on smithi121 to 175.9M 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: Unable to set osd_memory_target on smithi121 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:26:39.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:26:39.546 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:26:39.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:26:40.075 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 -- 172.21.15.19:0/476610995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78101420 msgr2=0x7fad7810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/476610995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78101420 0x7fad7810d950 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fad70009cd0 tx=0x7fad7002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 -- 172.21.15.19:0/476610995 shutdown_connections 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/476610995 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78101420 0x7fad7810d950 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/476610995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78100a80 0x7fad78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/476610995 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78106a80 0x7fad78106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 -- 172.21.15.19:0/476610995 >> 172.21.15.19:0/476610995 conn(0x7fad780fc7a0 msgr2=0x7fad780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 -- 172.21.15.19:0/476610995 shutdown_connections 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.711+0000 7fad7eb4e640 1 -- 172.21.15.19:0/476610995 wait complete. 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 Processor -- start 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 -- start start 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 0x7fad78199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 0x7fad781a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad78110ea0 con 0x7fad78100a80 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad78110d20 con 0x7fad78106a80 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7eb4e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad78111020 con 0x7fad78101420 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7c8ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7c8ea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33338/0 (socket says 172.21.15.19:33338) 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.712+0000 7fad7c8ea640 1 -- 172.21.15.19:0/3422332936 learned_addr learned my addr 172.21.15.19:0/3422332936 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad77fff640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 0x7fad78199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 0x7fad781a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 -- 172.21.15.19:0/3422332936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 msgr2=0x7fad78199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 0x7fad78199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 -- 172.21.15.19:0/3422332936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 msgr2=0x7fad781a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 0x7fad781a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 -- 172.21.15.19:0/3422332936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad781a48b0 con 0x7fad78100a80 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad77fff640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 0x7fad78199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 0x7fad781a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b4a0 tx=0x7fad6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad75ffb640 1 -- 172.21.15.19:0/3422332936 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68014070 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad75ffb640 1 -- 172.21.15.19:0/3422332936 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad68010af0 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad781a4a80 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad75ffb640 1 -- 172.21.15.19:0/3422332936 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad68017d10 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.713+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad781a4f40 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.715+0000 7fad75ffb640 1 -- 172.21.15.19:0/3422332936 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad68017070 con 0x7fad78100a80 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 Processor -- start 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 -- start start 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad75ffb640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 0x7fad640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf56370 0x55d8ddf58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8ddeb12f0 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d8dde58cf0 con 0x55d8ddf56370 2024-09-12T03:26:40.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad80f27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8ddead2b0 con 0x55d8ddf4cb40 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 0x7fad640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.716+0000 7fad77fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33340/0 (socket says 172.21.15.19:33340) 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 -- 172.21.15.19:0/2951266726 learned_addr learned my addr 172.21.15.19:0/2951266726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad75ffb640 1 -- 172.21.15.19:0/3422332936 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fad680ce4b0 con 0x7fad78100a80 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf56370 0x55d8ddf58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 -- 172.21.15.19:0/2951266726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4cb40 msgr2=0x55d8ddf4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 -- 172.21.15.19:0/2951266726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf56370 msgr2=0x55d8ddf58760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf56370 0x55d8ddf58760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 -- 172.21.15.19:0/2951266726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8dde3d3a0 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 0x7fad640a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad70006010 tx=0x7fad70031820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad77fff640 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fad6000cc50 tx=0x7fad6000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a83e3e987f57e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad6effd640 1 -- 172.21.15.19:0/2951266726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60014070 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad6effd640 1 -- 172.21.15.19:0/2951266726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad60019b40 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.717+0000 7fad6effd640 1 -- 172.21.15.19:0/2951266726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad60010be0 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 -- 172.21.15.19:0/2951266726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 msgr2=0x55d8ddf4ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fad6000cc50 tx=0x7fad6000e720 comp rx=0 tx=0).stop 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 -- 172.21.15.19:0/2951266726 shutdown_connections 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf56370 0x55d8ddf58760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf4ba10 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 --2- 172.21.15.19:0/2951266726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 -- 172.21.15.19:0/2951266726 >> 172.21.15.19:0/2951266726 conn(0x55d8ddf14310 msgr2=0x55d8ddf47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 -- 172.21.15.19:0/2951266726 shutdown_connections 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.718+0000 7fad80f27740 1 -- 172.21.15.19:0/2951266726 wait complete. 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 -- start start 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 Processor -- start 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf55d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf52fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d8ddf59070 con 0x55d8ddf56370 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d8dde6c060 con 0x55d8ddf4cb40 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d8ddf591f0 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad80f27740 10 client.0 _ll_register_callbacks cb 0x55d8ddbbf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33352/0 (socket says 172.21.15.19:33352) 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad7d0eb640 1 -- 172.21.15.19:0/104434314 learned_addr learned my addr 172.21.15.19:0/104434314 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf55d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad77fff640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf52fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.719+0000 7fad7d0eb640 1 -- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 msgr2=0x55d8ddf55d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf55d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7d0eb640 1 -- 172.21.15.19:0/104434314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 msgr2=0x55d8ddf52fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf52fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7d0eb640 1 -- 172.21.15.19:0/104434314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ddc223c0 con 0x55d8ddf56370 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad77fff640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf52fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf55d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fad70003a60 tx=0x7fad7002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae4441994d767f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7004e030 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fad70004740 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad60019b40 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fad700305a0 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad80f27740 10 client.16749 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d8ddf64b20 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad60019b40 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.720+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7fad70030070 con 0x55d8ddf56370 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 10 client.16749 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 10 client.16749 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 20 client.16749 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddf68530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 4 client.16749 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8ddebe860 con 0x55d8ddf66070 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.721+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddf68530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.723+0000 7fad57fff640 10 client.16749 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.723+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddf68530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ecc824b8dcbf5cf server_cookie=d3748767c0fcc574 in_seq=0 out_seq=0 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad57fff640 1 -- 172.21.15.19:0/104434314 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8ddebe860 con 0x55d8ddf66070 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad57fff640 10 client.16749 handle_command_reply: tid=0 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 client.16749 shutdown 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 msgr2=0x55d8ddf53560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fad70003a60 tx=0x7fad7002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 shutdown_connections 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddf68530 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf53560 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.769 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad57fff640 0 client.16749 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf52fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 --2- 172.21.15.19:0/104434314 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf55d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 >> 172.21.15.19:0/104434314 conn(0x55d8ddf14310 msgr2=0x55d8ddf4d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 shutdown_connections 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 1 -- 172.21.15.19:0/104434314 wait complete. 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.724+0000 7fad80f27740 20 client.16749 trim_cache size 0 max 16384 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 Processor -- start 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 -- start start 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf62370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.770 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad781ad680 con 0x55d8ddf56370 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad781af340 con 0x55d8ddf4cb40 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad80f27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad781add50 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.737+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33360/0 (socket says 172.21.15.19:33360) 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3861930641 learned_addr learned my addr 172.21.15.19:0/3861930641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad77fff640 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf62370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3861930641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 msgr2=0x55d8ddf62370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf62370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3861930641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 msgr2=0x55d8ddf4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.771 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3861930641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8ddf58b20 con 0x55d8ddf56370 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad70004980 tx=0x7fad70005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f19063d2b74425 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad6e7fc640 1 -- 172.21.15.19:0/3861930641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7004e040 con 0x55d8ddf56370 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad6e7fc640 1 -- 172.21.15.19:0/3861930641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70036040 con 0x55d8ddf56370 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.738+0000 7fad6e7fc640 1 -- 172.21.15.19:0/3861930641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad70030e40 con 0x55d8ddf56370 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- 172.21.15.19:0/3861930641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 msgr2=0x55d8ddf4e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad70004980 tx=0x7fad70005f70 comp rx=0 tx=0).stop 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- 172.21.15.19:0/3861930641 shutdown_connections 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf4e660 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 --2- 172.21.15.19:0/3861930641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf62370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- 172.21.15.19:0/3861930641 >> 172.21.15.19:0/3861930641 conn(0x55d8ddf14310 msgr2=0x55d8ddf51890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.772 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- 172.21.15.19:0/3861930641 shutdown_connections 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- 172.21.15.19:0/3861930641 wait complete. 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 -- start start 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.739+0000 7fad80f27740 1 Processor -- start 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf6d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fad781aebb0 con 0x55d8ddf56370 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fad781ae4e0 con 0x55d8ddf4cb40 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fad781ab5a0 con 0x55d8ddf4b5d0 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 10 client.0 _ll_register_callbacks cb 0x7fad70009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33372/0 (socket says 172.21.15.19:33372) 2024-09-12T03:26:40.773 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3913871090 learned_addr learned my addr 172.21.15.19:0/3913871090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad77fff640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf6d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 msgr2=0x55d8ddf6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf6cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3913871090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 msgr2=0x55d8ddf6d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf6d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 -- 172.21.15.19:0/3913871090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8ddf5d9f0 con 0x55d8ddf56370 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad7d0eb640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad7002a920 tx=0x7fad70009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161c8aceff48652b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad57fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad7004e040 con 0x55d8ddf56370 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fad70030bc0 con 0x55d8ddf56370 2024-09-12T03:26:40.774 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.740+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad70036040 con 0x55d8ddf56370 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fad70030dc0 con 0x55d8ddf56370 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 10 client.16755 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d8ddf5ecb0 con 0x55d8ddf56370 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad70036040 con 0x55d8ddf56370 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7fad70071070 con 0x55d8ddf56370 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 10 client.16755 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 10 client.16755 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 20 client.16755 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:40.775 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddfd2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 4 client.16755 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8ddeb8d70 con 0x55d8ddf66070 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.741+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddfd2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad7c8ea640 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddfd2e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f04a4637776a7b28 server_cookie=5775db1a173103d3 in_seq=0 out_seq=0 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad57fff640 10 client.16755 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad57fff640 1 -- 172.21.15.19:0/3913871090 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8ddeb8d70 con 0x55d8ddf66070 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad57fff640 10 client.16755 handle_command_reply: tid=0 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 client.16755 shutdown 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 msgr2=0x55d8ddf5d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.776 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fad7002a920 tx=0x7fad70009c20 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 shutdown_connections 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d8ddf66070 0x55d8ddfd2e20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d8ddf56370 0x55d8ddf5d4b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d8ddf4cb40 0x55d8ddf6d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad57fff640 0 client.16755 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 --2- 172.21.15.19:0/3913871090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d8ddf4b5d0 0x55d8ddf6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 >> 172.21.15.19:0/3913871090 conn(0x55d8ddf14310 msgr2=0x55d8ddf54820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 shutdown_connections 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 1 -- 172.21.15.19:0/3913871090 wait complete. 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.742+0000 7fad80f27740 20 client.16755 trim_cache size 0 max 16384 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 msgr2=0x7fad640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 0x7fad640a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fad70006010 tx=0x7fad70031820 comp rx=0 tx=0).stop 2024-09-12T03:26:40.777 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 msgr2=0x7fad781992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fad6800b4a0 tx=0x7fad6800b960 comp rx=0 tx=0).stop 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 shutdown_connections 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fad78106a80 0x7fad781a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fad78101420 0x7fad78199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fad640a2b80 0x7fad640a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 --2- 172.21.15.19:0/3422332936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fad78100a80 0x7fad781992f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 >> 172.21.15.19:0/3422332936 conn(0x7fad780fc7a0 msgr2=0x7fad780fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 shutdown_connections 2024-09-12T03:26:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:40.743+0000 7fad7eb4e640 1 -- 172.21.15.19:0/3422332936 wait complete. 2024-09-12T03:26:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:41 smithi121 ceph-mon[34963]: pgmap v566: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-09-12T03:26:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:41 smithi019 ceph-mon[30953]: pgmap v566: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-09-12T03:26:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:41 smithi135 ceph-mon[35474]: pgmap v566: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.2 MiB/s wr, 115 op/s 2024-09-12T03:26:41.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:26:41.832 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 -- 172.21.15.19:0/1936412654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 msgr2=0x7f9a0410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/1936412654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a0410da40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f99fc009ca0 tx=0x7f99fc02a950 comp rx=0 tx=0).stop 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 -- 172.21.15.19:0/1936412654 shutdown_connections 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/1936412654 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a0410da40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/1936412654 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04102570 0x7f9a041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/1936412654 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04100e20 0x7f9a04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 -- 172.21.15.19:0/1936412654 >> 172.21.15.19:0/1936412654 conn(0x7f9a040fc7d0 msgr2=0x7f9a040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 -- 172.21.15.19:0/1936412654 shutdown_connections 2024-09-12T03:26:42.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.321+0000 7f9a0a55e640 1 -- 172.21.15.19:0/1936412654 wait complete. 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 Processor -- start 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 -- start start 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 0x7f9a04199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 0x7f9a041997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04110f80 con 0x7f9a04102ef0 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04110e00 con 0x7f9a04100e20 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a0a55e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04111100 con 0x7f9a04102570 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33388/0 (socket says 172.21.15.19:33388) 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 -- 172.21.15.19:0/2231175687 learned_addr learned my addr 172.21.15.19:0/2231175687 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:42.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f99fbfff640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 0x7f9a041997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a03fff640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 0x7f9a04199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 -- 172.21.15.19:0/2231175687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 msgr2=0x7f9a041997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 0x7f9a041997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 -- 172.21.15.19:0/2231175687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 msgr2=0x7f9a04199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 0x7f9a04199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a08afb640 1 -- 172.21.15.19:0/2231175687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a041a4830 con 0x7f9a04102ef0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f99fbfff640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 0x7f9a041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.322+0000 7f9a03fff640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 0x7f9a04199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a08afb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006010 tx=0x7f99fc02f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a01ffb640 1 -- 172.21.15.19:0/2231175687 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc034040 con 0x7f9a04102ef0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a041a4a00 con 0x7f9a04102ef0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a01ffb640 1 -- 172.21.15.19:0/2231175687 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99fc003e00 con 0x7f9a04102ef0 2024-09-12T03:26:42.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a01ffb640 1 -- 172.21.15.19:0/2231175687 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc045480 con 0x7f9a04102ef0 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.323+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a041a4e50 con 0x7f9a04102ef0 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.325+0000 7f9a01ffb640 1 -- 172.21.15.19:0/2231175687 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f99fc037020 con 0x7f9a04102ef0 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.325+0000 7f9a0c937740 1 Processor -- start 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.325+0000 7f9a0c937740 1 -- start start 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795139a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x561795146760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56179509f2f0 con 0x56179513ab40 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561795046cf0 con 0x561795144370 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a0c937740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56179509b2b0 con 0x5617951395d0 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33398/0 (socket says 172.21.15.19:33398) 2024-09-12T03:26:42.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a01ffb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 0x7f99f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 -- 172.21.15.19:0/4214342341 learned_addr learned my addr 172.21.15.19:0/4214342341 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a03fff640 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x561795146760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a01ffb640 1 -- 172.21.15.19:0/2231175687 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f99fc0f0250 con 0x7f9a04102ef0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f9a08afb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 0x7f99f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 -- 172.21.15.19:0/4214342341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 msgr2=0x561795139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 -- 172.21.15.19:0/4214342341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 msgr2=0x561795146760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x561795146760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 -- 172.21.15.19:0/4214342341 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56179502b3a0 con 0x56179513ab40 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.326+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99ec005d90 tx=0x7f99ec00c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c9e777cbd0ba91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a08afb640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 0x7f99f00a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99fc02ae70 tx=0x7f99fc004480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f99fa7fc640 1 -- 172.21.15.19:0/4214342341 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec005020 con 0x56179513ab40 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f99fa7fc640 1 -- 172.21.15.19:0/4214342341 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec0047a0 con 0x56179513ab40 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f99fa7fc640 1 -- 172.21.15.19:0/4214342341 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec017750 con 0x56179513ab40 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 -- 172.21.15.19:0/4214342341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 msgr2=0x561795139090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f99ec005d90 tx=0x7f99ec00c600 comp rx=0 tx=0).stop 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 -- 172.21.15.19:0/4214342341 shutdown_connections 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x561795146760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4214342341 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795139090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 -- 172.21.15.19:0/4214342341 >> 172.21.15.19:0/4214342341 conn(0x561795102310 msgr2=0x561795135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 -- 172.21.15.19:0/4214342341 shutdown_connections 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.327+0000 7f9a0c937740 1 -- 172.21.15.19:0/4214342341 wait complete. 2024-09-12T03:26:42.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 -- start start 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 Processor -- start 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617951395d0 0x561795143d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 0x561795141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561795147070 con 0x56179513ab40 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56179505a060 con 0x5617951395d0 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5617951471f0 con 0x561795144370 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a0c937740 10 client.0 _ll_register_callbacks cb 0x561794dad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33408/0 (socket says 172.21.15.19:33408) 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.328+0000 7f9a03fff640 1 -- 172.21.15.19:0/4293739414 learned_addr learned my addr 172.21.15.19:0/4293739414 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617951395d0 0x561795143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 0x561795141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 -- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 msgr2=0x561795141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 0x561795141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 -- 172.21.15.19:0/4293739414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617951395d0 msgr2=0x561795143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617951395d0 0x561795143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 -- 172.21.15.19:0/4293739414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561794e103c0 con 0x56179513ab40 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 0x561795141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a03fff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99f4011850 tx=0x7f99f4011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3ff9937b34ffd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400f9a0 con 0x56179513ab40 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f99f400fb40 con 0x56179513ab40 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec0047a0 con 0x56179513ab40 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99f400fd40 con 0x56179513ab40 2024-09-12T03:26:42.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a0c937740 10 client.16767 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.329+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561795152b50 con 0x56179513ab40 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec0047a0 con 0x56179513ab40 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f99f4018030 con 0x56179513ab40 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 10 client.16767 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 10 client.16767 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 20 client.16767 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951540e0 0x561795156500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 4 client.16767 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617950ac860 con 0x5617951540e0 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951540e0 0x561795156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f99e37fe640 10 client.16767 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.330+0000 7f99fbfff640 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951540e0 0x561795156500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34037f60d3322db1 server_cookie=c579c35fe42fb327 in_seq=0 out_seq=0 2024-09-12T03:26:42.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f99e37fe640 1 -- 172.21.15.19:0/4293739414 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617950ac860 con 0x5617951540e0 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f99e37fe640 10 client.16767 handle_command_reply: tid=0 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 client.16767 shutdown 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 msgr2=0x561795140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f99f4011850 tx=0x7f99f4011d10 comp rx=0 tx=0).stop 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 shutdown_connections 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951540e0 0x561795156500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561795144370 0x561795141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x561795140fc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 --2- 172.21.15.19:0/4293739414 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617951395d0 0x561795143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f99e37fe640 0 client.16767 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 >> 172.21.15.19:0/4293739414 conn(0x561795102310 msgr2=0x56179513b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.331+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 shutdown_connections 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.332+0000 7f9a0c937740 1 -- 172.21.15.19:0/4293739414 wait complete. 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.332+0000 7f9a0c937740 20 client.16767 trim_cache size 0 max 16384 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 Processor -- start 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 -- start start 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795142f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56179513ab40 0x56179513c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ab300 con 0x561795144370 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041acfc0 con 0x56179513ab40 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a0c937740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ab9d0 con 0x5617951395d0 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33422/0 (socket says 172.21.15.19:33422) 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 -- 172.21.15.19:0/531068437 learned_addr learned my addr 172.21.15.19:0/531068437 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f9a03fff640 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56179513ab40 0x56179513c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 -- 172.21.15.19:0/531068437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 msgr2=0x561795142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 -- 172.21.15.19:0/531068437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56179513ab40 msgr2=0x56179513c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56179513ab40 0x56179513c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.345+0000 7f99fbfff640 1 -- 172.21.15.19:0/531068437 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561795136960 con 0x561795144370 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f99fbfff640 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99ec019f30 tx=0x7f99ec01b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce48577ca5e6fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f99f9ffb640 1 -- 172.21.15.19:0/531068437 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec034020 con 0x561795144370 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f99f9ffb640 1 -- 172.21.15.19:0/531068437 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec004020 con 0x561795144370 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f99f9ffb640 1 -- 172.21.15.19:0/531068437 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec004660 con 0x561795144370 2024-09-12T03:26:42.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 -- 172.21.15.19:0/531068437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 msgr2=0x56179513c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f99ec019f30 tx=0x7f99ec01b040 comp rx=0 tx=0).stop 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 -- 172.21.15.19:0/531068437 shutdown_connections 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561795144370 0x56179513c660 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56179513ab40 0x56179513c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 --2- 172.21.15.19:0/531068437 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x561795142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.346+0000 7f9a0c937740 1 -- 172.21.15.19:0/531068437 >> 172.21.15.19:0/531068437 conn(0x561795102310 msgr2=0x561795142820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- 172.21.15.19:0/531068437 shutdown_connections 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- 172.21.15.19:0/531068437 wait complete. 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- start start 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 Processor -- start 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x56179514cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x56179513e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ac830 con 0x56179513ab40 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041ac160 con 0x561795144370 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9a041a9240 con 0x5617951395d0 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.347+0000 7f9a0c937740 10 client.0 _ll_register_callbacks cb 0x7f99f4011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33424/0 (socket says 172.21.15.19:33424) 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 -- 172.21.15.19:0/1913288722 learned_addr learned my addr 172.21.15.19:0/1913288722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99fbfff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x56179514cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99fbfff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x56179513e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 -- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 msgr2=0x56179514cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x56179514cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 -- 172.21.15.19:0/1913288722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 msgr2=0x56179513e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x56179513e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 -- 172.21.15.19:0/1913288722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617951bdde0 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a03fff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99f4011570 tx=0x7f99f400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847de475384e3c44 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4033060 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f99f4024940 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec004020 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f99f4029070 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a0c937740 10 client.16773 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.348+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5617951befa0 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99ec004020 con 0x56179513ab40 2024-09-12T03:26:42.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f99f4018050 con 0x56179513ab40 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 10 client.16773 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 10 client.16773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 20 client.16773 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951bf7e0 0x5617951c1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 4 client.16773 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617950a6d70 con 0x5617951bf7e0 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.349+0000 7f99fbfff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951bf7e0 0x5617951c1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f99fbfff640 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951bf7e0 0x5617951c1c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81f079039d7299b server_cookie=93f3f40c1a6912c6 in_seq=0 out_seq=0 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f99e37fe640 10 client.16773 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f99e37fe640 1 -- 172.21.15.19:0/1913288722 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617950a6d70 con 0x5617951bf7e0 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f99e37fe640 10 client.16773 handle_command_reply: tid=0 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 client.16773 shutdown 2024-09-12T03:26:42.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 msgr2=0x56179513deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f99f4011570 tx=0x7f99f400fcb0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 shutdown_connections 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5617951bf7e0 0x5617951c1c00 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561795144370 0x56179513e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56179513ab40 0x56179513deb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 --2- 172.21.15.19:0/1913288722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617951395d0 0x56179514cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f99e37fe640 0 client.16773 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 >> 172.21.15.19:0/1913288722 conn(0x561795102310 msgr2=0x561795143040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 shutdown_connections 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 1 -- 172.21.15.19:0/1913288722 wait complete. 2024-09-12T03:26:42.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.350+0000 7f9a0c937740 20 client.16773 trim_cache size 0 max 16384 2024-09-12T03:26:42.368 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:26:42.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 msgr2=0x7f99f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 0x7f99f00a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99fc02ae70 tx=0x7f99fc004480 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 msgr2=0x7f9a041a42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f99fc006010 tx=0x7f99fc02f6f0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 shutdown_connections 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99f00a2b80 0x7f99f00a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9a04102ef0 0x7f9a041a42f0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9a04102570 0x7f9a041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 --2- 172.21.15.19:0/2231175687 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9a04100e20 0x7f9a04199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 >> 172.21.15.19:0/2231175687 conn(0x7f9a040fc7d0 msgr2=0x7f9a041067b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 shutdown_connections 2024-09-12T03:26:42.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:42.351+0000 7f9a0a55e640 1 -- 172.21.15.19:0/2231175687 wait complete. 2024-09-12T03:26:43.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:26:43.356 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:43.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:43 smithi121 ceph-mon[34963]: pgmap v567: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-12T03:26:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:43 smithi019 ceph-mon[30953]: pgmap v567: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-12T03:26:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:43 smithi135 ceph-mon[35474]: pgmap v567: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 828 KiB/s rd, 1.1 MiB/s wr, 109 op/s 2024-09-12T03:26:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 -- 172.21.15.19:0/4070140120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1002f0 msgr2=0x7f3ffc1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 --2- 172.21.15.19:0/4070140120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1006d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009cd0 tx=0x7f3fec02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 -- 172.21.15.19:0/4070140120 shutdown_connections 2024-09-12T03:26:43.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 --2- 172.21.15.19:0/4070140120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc106f50 0x7f3ffc10eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 --2- 172.21.15.19:0/4070140120 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 --2- 172.21.15.19:0/4070140120 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1006d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 -- 172.21.15.19:0/4070140120 >> 172.21.15.19:0/4070140120 conn(0x7f3ffc077210 msgr2=0x7f3ffc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 -- 172.21.15.19:0/4070140120 shutdown_connections 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.822+0000 7f4000cfc640 1 -- 172.21.15.19:0/4070140120 wait complete. 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.823+0000 7f4000cfc640 1 Processor -- start 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.823+0000 7f4000cfc640 1 -- start start 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111fa0 con 0x7f3ffc106f50 2024-09-12T03:26:43.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc111e20 con 0x7f3ffc100ca0 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc112120 con 0x7f3ffc1002f0 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33430/0 (socket says 172.21.15.19:33430) 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 -- 172.21.15.19:0/3616936565 learned_addr learned my addr 172.21.15.19:0/3616936565 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 -- 172.21.15.19:0/3616936565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc1002f0 msgr2=0x7f3ffc1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 -- 172.21.15.19:0/3616936565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 msgr2=0x7f3ffc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 -- 172.21.15.19:0/3616936565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc19d490 con 0x7f3ffc106f50 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3ffad9d640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e970 tx=0x7f3ff400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3feb7fe640 1 -- 172.21.15.19:0/3616936565 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff400ccc0 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc19d780 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3feb7fe640 1 -- 172.21.15.19:0/3616936565 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff400ce60 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.824+0000 7f3feb7fe640 1 -- 172.21.15.19:0/3616936565 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4010490 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.825+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a5060 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f3feb7fe640 1 -- 172.21.15.19:0/3616936565 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ff4005020 con 0x7f3ffc106f50 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 Processor -- start 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 -- start start 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x558876438090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876438a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55887639e2f0 con 0x558876447780 2024-09-12T03:26:43.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558876345cf0 con 0x558876439b40 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.827+0000 7f40030d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55887639a2b0 con 0x5588764385d0 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33438/0 (socket says 172.21.15.19:33438) 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ffa59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876438a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/2038481710 learned_addr learned my addr 172.21.15.19:0/2038481710 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x558876438090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3feb7fe640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 0x7f3fe40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/2038481710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 msgr2=0x558876438a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876438a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/2038481710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 msgr2=0x558876438090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x558876438090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/2038481710 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55887632a3a0 con 0x558876447780 2024-09-12T03:26:43.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3feb7fe640 1 -- 172.21.15.19:0/3616936565 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3ff40cf500 con 0x7f3ffc106f50 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ffad9d640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 0x7f3fe40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b620 tx=0x7f3ff000bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cc666e9981da28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3fe8ff9640 1 -- 172.21.15.19:0/2038481710 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0004020 con 0x558876447780 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3fe8ff9640 1 -- 172.21.15.19:0/2038481710 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff0005500 con 0x558876447780 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.828+0000 7f3fe8ff9640 1 -- 172.21.15.19:0/2038481710 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010e10 con 0x558876447780 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 -- 172.21.15.19:0/2038481710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 msgr2=0x558876449b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f3ffad9d640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 0x7f3fe40a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e940 tx=0x7f3ff40cf800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b620 tx=0x7f3ff000bb70 comp rx=0 tx=0).stop 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 -- 172.21.15.19:0/2038481710 shutdown_connections 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x558876449b70 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.843 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876438a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 --2- 172.21.15.19:0/2038481710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x558876438090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 -- 172.21.15.19:0/2038481710 >> 172.21.15.19:0/2038481710 conn(0x558876401310 msgr2=0x558876434c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 -- 172.21.15.19:0/2038481710 shutdown_connections 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.829+0000 7f40030d5740 1 -- 172.21.15.19:0/2038481710 wait complete. 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 -- start start 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 Processor -- start 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876447190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x558876444a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55887644a480 con 0x558876439b40 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558876359060 con 0x558876447780 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55887644a600 con 0x5588764385d0 2024-09-12T03:26:43.844 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f40030d5740 10 client.0 _ll_register_callbacks cb 0x5588760ac8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f3ffa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f3ffa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33444/0 (socket says 172.21.15.19:33444) 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f3ffa59c640 1 -- 172.21.15.19:0/2113113245 learned_addr learned my addr 172.21.15.19:0/2113113245 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.830+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876447190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x558876444a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 -- 172.21.15.19:0/2113113245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 msgr2=0x558876447190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876447190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 -- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 msgr2=0x558876444a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x558876444a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 -- 172.21.15.19:0/2113113245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887610f3c0 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876447190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02ae00 tx=0x7f3fec031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8a9fb903cfa085 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec048020 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3fec003970 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff0005500 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3fec003b70 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f40030d5740 10 client.16785 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558876455f50 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.831+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ff0005500 con 0x558876439b40 2024-09-12T03:26:43.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f3fec043070 con 0x558876439b40 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 10 client.16785 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 10 client.16785 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 20 client.16785 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x558876457460 0x558876459880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 4 client.16785 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588763ab860 con 0x558876457460 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x558876457460 0x558876459880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x558876457460 0x558876459880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3a4c0101be7d0fad server_cookie=7cee279dfb1f1610 in_seq=0 out_seq=0 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.832+0000 7f3fe1ffb640 10 client.16785 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:26:43.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/2113113245 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5588763ab860 con 0x558876457460 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f3fe1ffb640 10 client.16785 handle_command_reply: tid=0 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 client.16785 shutdown 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 msgr2=0x558876444440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3fec02ae00 tx=0x7f3fec031af0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 shutdown_connections 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x558876457460 0x558876459880 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x558876444a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x558876444440 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f3fe1ffb640 0 client.16785 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 --2- 172.21.15.19:0/2113113245 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876447190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 >> 172.21.15.19:0/2113113245 conn(0x558876401310 msgr2=0x55887643a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 shutdown_connections 2024-09-12T03:26:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.833+0000 7f40030d5740 1 -- 172.21.15.19:0/2113113245 wait complete. 2024-09-12T03:26:43.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.834+0000 7f40030d5740 20 client.16785 trim_cache size 0 max 16384 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 Processor -- start 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 -- start start 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876453720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x55887645f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1ab440 con 0x558876439b40 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1ad100 con 0x558876447780 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f40030d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1abb10 con 0x5588764385d0 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f3ffa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f3ffa59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33460/0 (socket says 172.21.15.19:33460) 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f3ffa59c640 1 -- 172.21.15.19:0/72112365 learned_addr learned my addr 172.21.15.19:0/72112365 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:43.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.847+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876453720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x55887645f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 -- 172.21.15.19:0/72112365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 msgr2=0x558876453720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876453720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 -- 172.21.15.19:0/72112365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 msgr2=0x55887645f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x55887645f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 -- 172.21.15.19:0/72112365 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588764453e0 con 0x558876439b40 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x55887645f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fec030880 tx=0x7f3fec02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c924b6fbf1fca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3fe3fff640 1 -- 172.21.15.19:0/72112365 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec048070 con 0x558876439b40 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3fe3fff640 1 -- 172.21.15.19:0/72112365 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fec036040 con 0x558876439b40 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.848+0000 7f3fe3fff640 1 -- 172.21.15.19:0/72112365 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec030b20 con 0x558876439b40 2024-09-12T03:26:43.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- 172.21.15.19:0/72112365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 msgr2=0x55887645f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3fec030880 tx=0x7f3fec02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- 172.21.15.19:0/72112365 shutdown_connections 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876447780 0x55887645f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876439b40 0x55887645f150 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 --2- 172.21.15.19:0/72112365 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x558876453720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- 172.21.15.19:0/72112365 >> 172.21.15.19:0/72112365 conn(0x558876401310 msgr2=0x558876443550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- 172.21.15.19:0/72112365 shutdown_connections 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- 172.21.15.19:0/72112365 wait complete. 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 -- start start 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.849+0000 7f40030d5740 1 Processor -- start 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x55887644f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x55887644fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1ac970 con 0x558876447780 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1ac2a0 con 0x558876439b40 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1a9360 con 0x5588764385d0 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 10 client.0 _ll_register_callbacks cb 0x7f3fec009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33472/0 (socket says 172.21.15.19:33472) 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/454399660 learned_addr learned my addr 172.21.15.19:0/454399660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x55887644fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x55887644f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/454399660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 msgr2=0x55887644f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x55887644f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 msgr2=0x55887644fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x55887644fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 -- 172.21.15.19:0/454399660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55887643b120 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3ff9d9b640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b620 tx=0x7f3fec0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373e69f268427390 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3fe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0010cd0 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3ff0028040 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.850+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3fec036040 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3ff0010e70 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 10 client.16791 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5588764c6910 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fec036040 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f3ff002c3a0 con 0x558876447780 2024-09-12T03:26:43.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 10 client.16791 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 10 client.16791 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 20 client.16791 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5588764c6cc0 0x5588764c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 4 client.16791 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588763a5d70 con 0x5588764c6cc0 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.851+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5588764c6cc0 0x5588764c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f3ffa59c640 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5588764c6cc0 0x5588764c90b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22805d5b28e133fe server_cookie=aa0cff55ce70a05 in_seq=0 out_seq=0 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f3fe1ffb640 10 client.16791 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f3fe1ffb640 1 -- 172.21.15.19:0/454399660 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588763a5d70 con 0x5588764c6cc0 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f3fe1ffb640 10 client.16791 handle_command_reply: tid=0 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 client.16791 shutdown 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 msgr2=0x55887644c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000b620 tx=0x7f3fec0098b0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 shutdown_connections 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5588764c6cc0 0x5588764c90b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558876447780 0x55887644c860 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558876439b40 0x55887644fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 --2- 172.21.15.19:0/454399660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5588764385d0 0x55887644f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 >> 172.21.15.19:0/454399660 conn(0x558876401310 msgr2=0x558876445ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f3fe1ffb640 0 client.16791 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 shutdown_connections 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 1 -- 172.21.15.19:0/454399660 wait complete. 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.852+0000 7f40030d5740 20 client.16791 trim_cache size 0 max 16384 2024-09-12T03:26:43.860 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:26:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 msgr2=0x7f3fe40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 0x7f3fe40a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e940 tx=0x7f3ff40cf800 comp rx=0 tx=0).stop 2024-09-12T03:26:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 msgr2=0x7f3ffc19cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:43.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400e970 tx=0x7f3ff400ed70 comp rx=0 tx=0).stop 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 shutdown_connections 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3fe40a2b80 0x7f3fe40a5040 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3ffc106f50 0x7f3ffc19cdb0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3ffc100ca0 0x7f3ffc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 --2- 172.21.15.19:0/3616936565 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3ffc1002f0 0x7f3ffc1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 >> 172.21.15.19:0/3616936565 conn(0x7f3ffc077210 msgr2=0x7f3ffc1052f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 shutdown_connections 2024-09-12T03:26:43.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:43.853+0000 7f4000cfc640 1 -- 172.21.15.19:0/3616936565 wait complete. 2024-09-12T03:26:44.724 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:26:44.982 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:45.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:45 smithi121 ceph-mon[34963]: pgmap v568: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 924 KiB/s wr, 84 op/s 2024-09-12T03:26:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:45 smithi019 ceph-mon[30953]: pgmap v568: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 924 KiB/s wr, 84 op/s 2024-09-12T03:26:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:45 smithi135 ceph-mon[35474]: pgmap v568: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 924 KiB/s wr, 84 op/s 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- 172.21.15.19:0/2067897127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478103740 msgr2=0x7fc478107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 --2- 172.21.15.19:0/2067897127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478103740 0x7fc478107b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009930 tx=0x7fc46c02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- 172.21.15.19:0/2067897127 shutdown_connections 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 --2- 172.21.15.19:0/2067897127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478108290 0x7fc47810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 --2- 172.21.15.19:0/2067897127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478103740 0x7fc478107b80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 --2- 172.21.15.19:0/2067897127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478102d90 0x7fc478103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- 172.21.15.19:0/2067897127 >> 172.21.15.19:0/2067897127 conn(0x7fc4780fc7d0 msgr2=0x7fc4780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- 172.21.15.19:0/2067897127 shutdown_connections 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- 172.21.15.19:0/2067897127 wait complete. 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 Processor -- start 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.534+0000 7fc480627640 1 -- start start 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478102d90 0x7fc478198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478103740 0x7fc478198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc478112090 con 0x7fc478108290 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc478111f10 con 0x7fc478102d90 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc480627640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc478112210 con 0x7fc478103740 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33488/0 (socket says 172.21.15.19:33488) 2024-09-12T03:26:45.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 -- 172.21.15.19:0/1339560124 learned_addr learned my addr 172.21.15.19:0/1339560124 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478102d90 0x7fc478198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478103740 0x7fc478198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 -- 172.21.15.19:0/1339560124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478103740 msgr2=0x7fc478198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478103740 0x7fc478198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 -- 172.21.15.19:0/1339560124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478102d90 msgr2=0x7fc478198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478102d90 0x7fc478198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 -- 172.21.15.19:0/1339560124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47819d600 con 0x7fc478108290 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.535+0000 7fc47ebc4640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc46800eaa0 tx=0x7fc46800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.536+0000 7fc4777fe640 1 -- 172.21.15.19:0/1339560124 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46800cdb0 con 0x7fc478108290 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.536+0000 7fc4777fe640 1 -- 172.21.15.19:0/1339560124 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc468005020 con 0x7fc478108290 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.536+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47819d8f0 con 0x7fc478108290 2024-09-12T03:26:45.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.536+0000 7fc4777fe640 1 -- 172.21.15.19:0/1339560124 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4680116b0 con 0x7fc478108290 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.536+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4781a51d0 con 0x7fc478108290 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.538+0000 7fc4777fe640 1 -- 172.21.15.19:0/1339560124 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc468005560 con 0x7fc478108290 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.538+0000 7fc482a00740 1 Processor -- start 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.538+0000 7fc482a00740 1 -- start start 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6420090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee642d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee63862f0 con 0x564ee64205d0 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564ee632dcf0 con 0x564ee642b370 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc482a00740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ee63822b0 con 0x564ee6421b40 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33500/0 (socket says 172.21.15.19:33500) 2024-09-12T03:26:45.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1193586800 learned_addr learned my addr 172.21.15.19:0/1193586800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee642d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc4777fe640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 0x7fc4600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6420090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1193586800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 msgr2=0x564ee6420090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6420090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1193586800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 msgr2=0x564ee642d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee642d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1193586800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee63123a0 con 0x564ee64205d0 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6420090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47ebc4640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 0x7fc4600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc47000bd10 tx=0x7fc4700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a381f99ac8f096 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.539+0000 7fc4777fe640 1 -- 172.21.15.19:0/1339560124 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc4680cf220 con 0x7fc478108290 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc474ff9640 1 -- 172.21.15.19:0/1193586800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470004030 con 0x564ee64205d0 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc474ff9640 1 -- 172.21.15.19:0/1193586800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4700041d0 con 0x564ee64205d0 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc474ff9640 1 -- 172.21.15.19:0/1193586800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470011840 con 0x564ee64205d0 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 -- 172.21.15.19:0/1193586800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 msgr2=0x564ee6420a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc47ebc4640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 0x7fc4600a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc46809b790 tx=0x7fc468066710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc47000bd10 tx=0x7fc4700107a0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 -- 172.21.15.19:0/1193586800 shutdown_connections 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee642d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee6420a10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 --2- 172.21.15.19:0/1193586800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6420090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.540+0000 7fc482a00740 1 -- 172.21.15.19:0/1193586800 >> 172.21.15.19:0/1193586800 conn(0x564ee63e9310 msgr2=0x564ee641cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- 172.21.15.19:0/1193586800 shutdown_connections 2024-09-12T03:26:45.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- 172.21.15.19:0/1193586800 wait complete. 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- start start 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 Processor -- start 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6428000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee64285d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564ee642e070 con 0x564ee64205d0 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564ee6341060 con 0x564ee642b370 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564ee642e1f0 con 0x564ee6421b40 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc482a00740 10 client.0 _ll_register_callbacks cb 0x564ee60948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc47dbc2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc47dbc2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33516/0 (socket says 172.21.15.19:33516) 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc47dbc2640 1 -- 172.21.15.19:0/1677809192 learned_addr learned my addr 172.21.15.19:0/1677809192 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:45.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.541+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee64285d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6428000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 -- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 msgr2=0x564ee6428000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6428000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 -- 172.21.15.19:0/1677809192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 msgr2=0x564ee64285d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee64285d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 -- 172.21.15.19:0/1677809192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee60f73c0 con 0x564ee64205d0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6428000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005ec0 tx=0x7fc46c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735af9d66626ba13 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c035040 con 0x564ee64205d0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc46c005040 con 0x564ee64205d0 2024-09-12T03:26:45.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4700041d0 con 0x564ee64205d0 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc46c051050 con 0x564ee64205d0 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc482a00740 10 client.16803 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ee6439a60 con 0x564ee64205d0 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4700041d0 con 0x564ee64205d0 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7fc46c03a040 con 0x564ee64205d0 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc482a00740 10 client.16803 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.542+0000 7fc482a00740 10 client.16803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc482a00740 20 client.16803 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee643af10 0x564ee643d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc482a00740 4 client.16803 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee6393860 con 0x564ee643af10 2024-09-12T03:26:45.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee643af10 0x564ee643d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee643af10 0x564ee643d3b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb87260add8b6dba server_cookie=74744f7eae4eb769 in_seq=0 out_seq=0 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.543+0000 7fc45dffb640 10 client.16803 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc45dffb640 1 -- 172.21.15.19:0/1677809192 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee6393860 con 0x564ee643af10 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc45dffb640 10 client.16803 handle_command_reply: tid=0 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 client.16803 shutdown 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 msgr2=0x564ee642ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc46c005ec0 tx=0x7fc46c02a420 comp rx=0 tx=0).stop 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 shutdown_connections 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee643af10 0x564ee643d3b0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee642b370 0x564ee64285d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee6428000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 --2- 172.21.15.19:0/1677809192 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee64205d0 0x564ee642ad50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 >> 172.21.15.19:0/1677809192 conn(0x564ee63e9310 msgr2=0x564ee6422960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc45dffb640 0 client.16803 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 shutdown_connections 2024-09-12T03:26:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 1 -- 172.21.15.19:0/1677809192 wait complete. 2024-09-12T03:26:45.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.544+0000 7fc482a00740 20 client.16803 trim_cache size 0 max 16384 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 Processor -- start 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 -- start start 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6437310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee642b370 0x564ee6423660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781ad6b0 con 0x564ee6421b40 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781af370 con 0x564ee64205d0 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc482a00740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781add80 con 0x564ee642b370 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33528/0 (socket says 172.21.15.19:33528) 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1222597738 learned_addr learned my addr 172.21.15.19:0/1222597738 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6437310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee642b370 0x564ee6423660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1222597738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee642b370 msgr2=0x564ee6423660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee642b370 0x564ee6423660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1222597738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 msgr2=0x564ee6437310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6437310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 -- 172.21.15.19:0/1222597738 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee6426570 con 0x564ee6421b40 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6437310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.558+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc470010ca0 tx=0x7fc470003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6527a5df8759011c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc45ffff640 1 -- 172.21.15.19:0/1222597738 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700037f0 con 0x564ee6421b40 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc45ffff640 1 -- 172.21.15.19:0/1222597738 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470003d10 con 0x564ee6421b40 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc45ffff640 1 -- 172.21.15.19:0/1222597738 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700268c0 con 0x564ee6421b40 2024-09-12T03:26:45.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 -- 172.21.15.19:0/1222597738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 msgr2=0x564ee6423120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fc470010ca0 tx=0x7fc470003ef0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 -- 172.21.15.19:0/1222597738 shutdown_connections 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee642b370 0x564ee6423660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee6421b40 0x564ee6423120 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 --2- 172.21.15.19:0/1222597738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6437310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 -- 172.21.15.19:0/1222597738 >> 172.21.15.19:0/1222597738 conn(0x564ee63e9310 msgr2=0x564ee6427110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 -- 172.21.15.19:0/1222597738 shutdown_connections 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.559+0000 7fc482a00740 1 -- 172.21.15.19:0/1222597738 wait complete. 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 -- start start 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 Processor -- start 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6433480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee64339c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781aebe0 con 0x564ee642b370 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781ae510 con 0x564ee64205d0 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4781ab5f0 con 0x564ee6421b40 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc482a00740 10 client.0 _ll_register_callbacks cb 0x7fc46c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc47dbc2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc47dbc2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33532/0 (socket says 172.21.15.19:33532) 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc47dbc2640 1 -- 172.21.15.19:0/2534794348 learned_addr learned my addr 172.21.15.19:0/2534794348 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.560+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6433480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee64339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 -- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 msgr2=0x564ee64339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee64339c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 -- 172.21.15.19:0/2534794348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 msgr2=0x564ee6433480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6433480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 -- 172.21.15.19:0/2534794348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee6424c30 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc47dbc2640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009850 tx=0x7fc46c0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cab05c26d9774d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc45dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c035040 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc470003d10 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc46c004ad0 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc46c0039f0 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc482a00740 10 client.16809 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564ee64a5510 con 0x564ee642b370 2024-09-12T03:26:45.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.561+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc470003d10 con 0x564ee642b370 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7fc46c03d480 con 0x564ee642b370 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 10 client.16809 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 10 client.16809 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 20 client.16809 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee64a5cd0 0x564ee64a80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 4 client.16809 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ee638dd70 con 0x564ee64a5cd0 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee64a5cd0 0x564ee64a80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc45dffb640 10 client.16809 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.562+0000 7fc47e3c3640 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee64a5cd0 0x564ee64a80f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=430c5841b2bc52ce server_cookie=59933defbd4319db in_seq=0 out_seq=0 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc45dffb640 1 -- 172.21.15.19:0/2534794348 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ee638dd70 con 0x564ee64a5cd0 2024-09-12T03:26:45.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc45dffb640 10 client.16809 handle_command_reply: tid=0 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 client.16809 shutdown 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 msgr2=0x564ee6424670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fc46c009850 tx=0x7fc46c0315b0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 shutdown_connections 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564ee642b370 0x564ee6424670 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564ee6421b40 0x564ee64339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564ee64a5cd0 0x564ee64a80f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 --2- 172.21.15.19:0/2534794348 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564ee64205d0 0x564ee6433480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 >> 172.21.15.19:0/2534794348 conn(0x564ee63e9310 msgr2=0x564ee641b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc45dffb640 0 client.16809 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 shutdown_connections 2024-09-12T03:26:45.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 1 -- 172.21.15.19:0/2534794348 wait complete. 2024-09-12T03:26:45.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.563+0000 7fc482a00740 20 client.16809 trim_cache size 0 max 16384 2024-09-12T03:26:45.572 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 msgr2=0x7fc4600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 0x7fc4600a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc46809b790 tx=0x7fc468066710 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 msgr2=0x7fc47819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc46800eaa0 tx=0x7fc46800ef60 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 shutdown_connections 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc4600a2b80 0x7fc4600a5040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc478108290 0x7fc47819cf20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc478103740 0x7fc478198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 --2- 172.21.15.19:0/1339560124 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc478102d90 0x7fc478198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 >> 172.21.15.19:0/1339560124 conn(0x7fc4780fc7d0 msgr2=0x7fc478105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 shutdown_connections 2024-09-12T03:26:45.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:45.564+0000 7fc480627640 1 -- 172.21.15.19:0/1339560124 wait complete. 2024-09-12T03:26:47.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:47 smithi121 ceph-mon[34963]: pgmap v569: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 412 KiB/s wr, 51 op/s 2024-09-12T03:26:47.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:47 smithi019 ceph-mon[30953]: pgmap v569: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 412 KiB/s wr, 51 op/s 2024-09-12T03:26:47.495 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:26:47.496 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:26:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:47 smithi135 ceph-mon[35474]: pgmap v569: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 412 KiB/s wr, 51 op/s 2024-09-12T03:26:47.759 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 -- 172.21.15.19:0/792920099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 msgr2=0x7f75901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 --2- 172.21.15.19:0/792920099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75901021c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7588009900 tx=0x7f758802a410 comp rx=0 tx=0).stop 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 -- 172.21.15.19:0/792920099 shutdown_connections 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 --2- 172.21.15.19:0/792920099 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f759010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 --2- 172.21.15.19:0/792920099 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 0x7f75901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 --2- 172.21.15.19:0/792920099 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75901021c0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 -- 172.21.15.19:0/792920099 >> 172.21.15.19:0/792920099 conn(0x7f75900fd880 msgr2=0x7f75900ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 -- 172.21.15.19:0/792920099 shutdown_connections 2024-09-12T03:26:48.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.275+0000 7f7597c5f640 1 -- 172.21.15.19:0/792920099 wait complete. 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 Processor -- start 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 -- start start 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 0x7f7590074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f7590074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7590111f70 con 0x7f7590101de0 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7590111df0 con 0x7f7590106f10 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f7597c5f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75901120f0 con 0x7f7590102790 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f75959fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f75959fb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33550/0 (socket says 172.21.15.19:33550) 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f75959fb640 1 -- 172.21.15.19:0/3365706557 learned_addr learned my addr 172.21.15.19:0/3365706557 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f75961fc640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f7590074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.276+0000 7f75951fa640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 0x7f7590074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 -- 172.21.15.19:0/3365706557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 msgr2=0x7f7590074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 0x7f7590074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 -- 172.21.15.19:0/3365706557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 msgr2=0x7f7590074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f7590074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 -- 172.21.15.19:0/3365706557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75901a68a0 con 0x7f7590101de0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75961fc640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f7590074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f75959fb640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7588006010 tx=0x7f75880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f758effd640 1 -- 172.21.15.19:0/3365706557 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758803f030 con 0x7f7590101de0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f758effd640 1 -- 172.21.15.19:0/3365706557 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7588003ce0 con 0x7f7590101de0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f758effd640 1 -- 172.21.15.19:0/3365706557 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588035070 con 0x7f7590101de0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75901a6b30 con 0x7f7590101de0 2024-09-12T03:26:48.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.277+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75901a6fc0 con 0x7f7590101de0 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.279+0000 7f758effd640 1 -- 172.21.15.19:0/3365706557 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7588042030 con 0x7f7590101de0 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 Processor -- start 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 -- start start 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08cd480 0x55a6f08d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6f0832280 con 0x55a6f08d7d40 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6f07d9b10 con 0x55a6f08cd480 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f759a038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6f082e240 con 0x55a6f08ccb60 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33556/0 (socket says 172.21.15.19:33556) 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f75951fa640 1 -- 172.21.15.19:0/1078718463 learned_addr learned my addr 172.21.15.19:0/1078718463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:48.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f75961fc640 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.280+0000 7f75959fb640 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08cd480 0x55a6f08d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f758effd640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 0x7f75780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 -- 172.21.15.19:0/1078718463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 msgr2=0x55a6f08ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 -- 172.21.15.19:0/1078718463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08cd480 msgr2=0x55a6f08d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08cd480 0x55a6f08d7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 -- 172.21.15.19:0/1078718463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f07e61f0 con 0x55a6f08d7d40 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75961fc640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 0x7f75780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f758effd640 1 -- 172.21.15.19:0/3365706557 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f75880ef5a0 con 0x7f7590101de0 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f75951fa640 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f758400bc90 tx=0x7f758400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d30ced7c9f56691 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f7577fff640 1 -- 172.21.15.19:0/1078718463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584015070 con 0x55a6f08d7d40 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f7577fff640 1 -- 172.21.15.19:0/1078718463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758401a940 con 0x55a6f08d7d40 2024-09-12T03:26:48.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.281+0000 7f7577fff640 1 -- 172.21.15.19:0/1078718463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584011970 con 0x55a6f08d7d40 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- 172.21.15.19:0/1078718463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 msgr2=0x55a6f08da130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f75961fc640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 0x7f75780a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fd0 tx=0x7f7580004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f758400bc90 tx=0x7f758400f730 comp rx=0 tx=0).stop 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- 172.21.15.19:0/1078718463 shutdown_connections 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f08da130 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08cd480 0x55a6f08d7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 --2- 172.21.15.19:0/1078718463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- 172.21.15.19:0/1078718463 >> 172.21.15.19:0/1078718463 conn(0x55a6f08c90f0 msgr2=0x55a6f08c94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- 172.21.15.19:0/1078718463 shutdown_connections 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- 172.21.15.19:0/1078718463 wait complete. 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.282+0000 7f759a038740 1 -- start start 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 Processor -- start 2024-09-12T03:26:48.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08d56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a6f08da910 con 0x55a6f08cd480 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a6f07ecf10 con 0x55a6f08d7d40 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a6f08daa90 con 0x55a6f08ccb60 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f759a038740 10 client.0 _ll_register_callbacks cb 0x55a6f07b5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33560/0 (socket says 172.21.15.19:33560) 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 -- 172.21.15.19:0/460359340 learned_addr learned my addr 172.21.15.19:0/460359340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75959fb640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75961fc640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08d56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 -- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 msgr2=0x55a6f08d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 -- 172.21.15.19:0/460359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 msgr2=0x55a6f08d56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08d56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75951fa640 1 -- 172.21.15.19:0/460359340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f05a36a0 con 0x55a6f08cd480 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.283+0000 7f75961fc640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75951fa640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f758400b9d0 tx=0x7f758400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9246a0606c7e1efb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584015070 con 0x55a6f08cd480 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f758401a460 con 0x55a6f08cd480 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758401a940 con 0x55a6f08cd480 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f758401a6a0 con 0x55a6f08cd480 2024-09-12T03:26:48.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 10 client.16821 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a6f08e6540 con 0x55a6f08cd480 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758401a940 con 0x55a6f08cd480 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f7584011b30 con 0x55a6f08cd480 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 10 client.16821 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 10 client.16821 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 20 client.16821 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f08e7a00 0x55a6f08e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 4 client.16821 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.284+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f083f7f0 con 0x55a6f08e7a00 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.285+0000 7f75959fb640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f08e7a00 0x55a6f08e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.285+0000 7f75757fa640 10 client.16821 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:48.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.285+0000 7f75959fb640 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f08e7a00 0x55a6f08e9df0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=248384395aae0c3a server_cookie=a71b120b281190c7 in_seq=0 out_seq=0 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f75757fa640 1 -- 172.21.15.19:0/460359340 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a6f083f7f0 con 0x55a6f08e7a00 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f75757fa640 10 client.16821 handle_command_reply: tid=0 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 client.16821 shutdown 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 msgr2=0x55a6f08d5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f758400b9d0 tx=0x7f758400bee0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 shutdown_connections 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f08e7a00 0x55a6f08e9df0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08d56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08d5190 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 --2- 172.21.15.19:0/460359340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f75757fa640 0 client.16821 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 >> 172.21.15.19:0/460359340 conn(0x55a6f08c90f0 msgr2=0x55a6f08cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 shutdown_connections 2024-09-12T03:26:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 1 -- 172.21.15.19:0/460359340 wait complete. 2024-09-12T03:26:48.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.286+0000 7f759a038740 20 client.16821 trim_cache size 0 max 16384 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 Processor -- start 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 -- start start 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08efb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75901af740 con 0x55a6f08cd480 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b1400 con 0x55a6f08d7d40 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f759a038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75901afe10 con 0x55a6f08ccb60 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33568/0 (socket says 172.21.15.19:33568) 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 -- 172.21.15.19:0/3137510467 learned_addr learned my addr 172.21.15.19:0/3137510467 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75959fb640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75961fc640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08efb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 -- 172.21.15.19:0/3137510467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 msgr2=0x55a6f08cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 -- 172.21.15.19:0/3137510467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 msgr2=0x55a6f08efb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08efb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75951fa640 1 -- 172.21.15.19:0/3137510467 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f08c3680 con 0x55a6f08cd480 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.304+0000 7f75961fc640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08efb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f75951fa640 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f758400ba20 tx=0x7f7584005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e21be86ba41f1a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f75777fe640 1 -- 172.21.15.19:0/3137510467 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584015040 con 0x55a6f08cd480 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f75777fe640 1 -- 172.21.15.19:0/3137510467 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758402a030 con 0x55a6f08cd480 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f75777fe640 1 -- 172.21.15.19:0/3137510467 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584005520 con 0x55a6f08cd480 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f759a038740 1 -- 172.21.15.19:0/3137510467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 msgr2=0x55a6f08ef620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.305+0000 7f759a038740 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f758400ba20 tx=0x7f7584005d20 comp rx=0 tx=0).stop 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- 172.21.15.19:0/3137510467 shutdown_connections 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08d7d40 0x55a6f08efb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08cd480 0x55a6f08ef620 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- 172.21.15.19:0/3137510467 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08ccb60 0x55a6f08cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- 172.21.15.19:0/3137510467 >> 172.21.15.19:0/3137510467 conn(0x55a6f08c90f0 msgr2=0x55a6f08ce5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- 172.21.15.19:0/3137510467 shutdown_connections 2024-09-12T03:26:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- 172.21.15.19:0/3137510467 wait complete. 2024-09-12T03:26:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- start start 2024-09-12T03:26:48.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 Processor -- start 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08ccb60 0x55a6f08dbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08cd480 0x55a6f08dc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b0c70 con 0x55a6f08d7d40 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75901b05a0 con 0x55a6f08ccb60 2024-09-12T03:26:48.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.306+0000 7f759a038740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f75901ad660 con 0x55a6f08cd480 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f759a038740 10 client.0 _ll_register_callbacks cb 0x7f758400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33580/0 (socket says 172.21.15.19:33580) 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 -- 172.21.15.19:0/2858470663 learned_addr learned my addr 172.21.15.19:0/2858470663 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75951fa640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08cd480 0x55a6f08dc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75959fb640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08ccb60 0x55a6f08dbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 -- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08cd480 msgr2=0x55a6f08dc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08cd480 0x55a6f08dc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 -- 172.21.15.19:0/2858470663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08ccb60 msgr2=0x55a6f08dbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08ccb60 0x55a6f08dbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 -- 172.21.15.19:0/2858470663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f08dcbd0 con 0x55a6f08d7d40 2024-09-12T03:26:48.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75961fc640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7580012ae0 tx=0x7f7580004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3a44d484126b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75757fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758002d060 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f758001fca0 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.307+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758402a030 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f75800283f0 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 10 client.16827 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a6f0952ce0 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f758402a030 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f7580027070 con 0x55a6f08d7d40 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 10 client.16827 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 10 client.16827 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 20 client.16827 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:48.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f0953490 0x55a6f09558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 4 client.16827 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6f0839d00 con 0x55a6f0953490 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.308+0000 7f75959fb640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f0953490 0x55a6f09558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.309+0000 7f75959fb640 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f0953490 0x55a6f09558b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4995bd2de5188b60 server_cookie=5d117a9bd8ccab28 in_seq=0 out_seq=0 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.309+0000 7f75757fa640 10 client.16827 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f75757fa640 1 -- 172.21.15.19:0/2858470663 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6f0839d00 con 0x55a6f0953490 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f75757fa640 10 client.16827 handle_command_reply: tid=0 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 client.16827 shutdown 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 msgr2=0x55a6f09515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f7580012ae0 tx=0x7f7580004a40 comp rx=0 tx=0).stop 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 shutdown_connections 2024-09-12T03:26:48.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a6f08d7d40 0x55a6f09515f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a6f08cd480 0x55a6f08dc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a6f0953490 0x55a6f09558b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 --2- 172.21.15.19:0/2858470663 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a6f08ccb60 0x55a6f08dbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f75757fa640 0 client.16827 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 >> 172.21.15.19:0/2858470663 conn(0x55a6f08c90f0 msgr2=0x55a6f08c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.310+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 shutdown_connections 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f759a038740 1 -- 172.21.15.19:0/2858470663 wait complete. 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f759a038740 20 client.16827 trim_cache size 0 max 16384 2024-09-12T03:26:48.316 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:26:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3e6f4c14-b810-4893-8ed4-81cec1c1d24d", 2024-09-12T03:26:48.317 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:26:48.317 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:48.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 msgr2=0x7f75780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 0x7f75780a50c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7580009fd0 tx=0x7f7580004ab0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 msgr2=0x7f75900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f7588006010 tx=0x7f75880042f0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 shutdown_connections 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7590106f10 0x7f7590074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7590102790 0x7f7590074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75780a2c00 0x7f75780a50c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 --2- 172.21.15.19:0/3365706557 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7590101de0 0x7f75900744f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 >> 172.21.15.19:0/3365706557 conn(0x7f75900fd880 msgr2=0x7f75900fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 shutdown_connections 2024-09-12T03:26:48.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:48.311+0000 7f7597c5f640 1 -- 172.21.15.19:0/3365706557 wait complete. 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: pgmap v570: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.162 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.163 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:49 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:49.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:26:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: pgmap v570: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-12T03:26:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:26:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.402 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:49 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:49.508 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: pgmap v570: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:49.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:49 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:50.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/2343402700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 msgr2=0x7f8c541124f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/2343402700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541124f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8c48009860 tx=0x7f8c4802a340 comp rx=0 tx=0).stop 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/2343402700 shutdown_connections 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/2343402700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54112a30 0x7f8c54114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/2343402700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541124f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/2343402700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54076720 0x7f8c54076b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/2343402700 >> 172.21.15.19:0/2343402700 conn(0x7f8c540ffab0 msgr2=0x7f8c54101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/2343402700 shutdown_connections 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.065+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/2343402700 wait complete. 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.066+0000 7f8c5a5a4640 1 Processor -- start 2024-09-12T03:26:50.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.066+0000 7f8c5a5a4640 1 -- start start 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54076720 0x7f8c541a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 0x7f8c5419e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541185a0 con 0x7f8c54112a30 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54118420 con 0x7f8c54077040 2024-09-12T03:26:50.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c5a5a4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c54118720 con 0x7f8c54076720 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54076720 0x7f8c541a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40714/0 (socket says 172.21.15.19:40714) 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c537fe640 1 -- 172.21.15.19:0/3554085767 learned_addr learned my addr 172.21.15.19:0/3554085767 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.067+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 0x7f8c5419e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.068+0000 7f8c58b41640 1 -- 172.21.15.19:0/3554085767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54076720 msgr2=0x7f8c541a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.068+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54076720 0x7f8c541a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.068+0000 7f8c58b41640 1 -- 172.21.15.19:0/3554085767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 msgr2=0x7f8c541a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.068+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.068+0000 7f8c58b41640 1 -- 172.21.15.19:0/3554085767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5419efe0 con 0x7f8c54112a30 2024-09-12T03:26:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.069+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 0x7f8c5419e8b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8c5407aa10 tx=0x7f8c4c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.069+0000 7f8c517fa640 1 -- 172.21.15.19:0/3554085767 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00cdf0 con 0x7f8c54112a30 2024-09-12T03:26:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.070+0000 7f8c517fa640 1 -- 172.21.15.19:0/3554085767 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c4c005020 con 0x7f8c54112a30 2024-09-12T03:26:50.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.070+0000 7f8c517fa640 1 -- 172.21.15.19:0/3554085767 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c010670 con 0x7f8c54112a30 2024-09-12T03:26:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.070+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5419f2d0 con 0x7f8c54112a30 2024-09-12T03:26:50.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.070+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c541ab5c0 con 0x7f8c54112a30 2024-09-12T03:26:50.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.074+0000 7f8c5c97d740 1 Processor -- start 2024-09-12T03:26:50.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.074+0000 7f8c517fa640 1 -- 172.21.15.19:0/3554085767 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c4c0054d0 con 0x7f8c54112a30 2024-09-12T03:26:50.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.074+0000 7f8c5c97d740 1 -- start start 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9b0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9bb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9bda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56298d9162e0 con 0x56298d9bb680 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56298d8bdce0 con 0x56298d9b04a0 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56298d9122a0 con 0x56298d9b0dc0 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9bb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c53fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9bb140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:35780/0 (socket says 172.21.15.19:35780) 2024-09-12T03:26:50.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.075+0000 7f8c53fff640 1 -- 172.21.15.19:0/2132079235 learned_addr learned my addr 172.21.15.19:0/2132079235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9bda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 -- 172.21.15.19:0/2132079235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 msgr2=0x56298d9bb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9bb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 -- 172.21.15.19:0/2132079235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 msgr2=0x56298d9bda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9bda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c517fa640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 0x7f8c3c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 -- 172.21.15.19:0/2132079235 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298d8a2440 con 0x56298d9b04a0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c517fa640 1 -- 172.21.15.19:0/3554085767 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c4c0cf590 con 0x7f8c54112a30 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c537fe640 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9b0880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000c00 tx=0x7f8c48005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7feef09440717e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.076+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 0x7f8c3c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.077+0000 7f8c3a7fc640 1 -- 172.21.15.19:0/2132079235 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48047070 con 0x56298d9b04a0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.077+0000 7f8c3a7fc640 1 -- 172.21.15.19:0/2132079235 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c480043d0 con 0x56298d9b04a0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.077+0000 7f8c3a7fc640 1 -- 172.21.15.19:0/2132079235 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037040 con 0x56298d9b04a0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 -- 172.21.15.19:0/2132079235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 msgr2=0x56298d9b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9b0880 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f8c48000c00 tx=0x7f8c48005d20 comp rx=0 tx=0).stop 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c58b41640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 0x7f8c3c0a5190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009fd0 tx=0x7f8c4c00cf90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 -- 172.21.15.19:0/2132079235 shutdown_connections 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9bda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9bb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/2132079235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9b0880 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 -- 172.21.15.19:0/2132079235 >> 172.21.15.19:0/2132079235 conn(0x56298d9aca30 msgr2=0x56298d9ace10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 -- 172.21.15.19:0/2132079235 shutdown_connections 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.078+0000 7f8c5c97d740 1 -- 172.21.15.19:0/2132079235 wait complete. 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 -- start start 2024-09-12T03:26:50.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 Processor -- start 2024-09-12T03:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9bb680 0x56298d9b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56298d9be250 con 0x56298d9b04a0 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56298d8d1050 con 0x56298d9b0dc0 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.079+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56298d9be3d0 con 0x56298d9bb680 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c5c97d740 10 client.0 _ll_register_callbacks cb 0x56298d911160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40734/0 (socket says 172.21.15.19:40734) 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 -- 172.21.15.19:0/1661901167 learned_addr learned my addr 172.21.15.19:0/1661901167 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33608/0 (socket says 172.21.15.19:33608) 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 -- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9bb680 msgr2=0x56298d9b8b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:26:50.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9bb680 0x56298d9b8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 -- 172.21.15.19:0/1661901167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 msgr2=0x56298d9bb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c53fff640 1 -- 172.21.15.19:0/1661901167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298d6873c0 con 0x56298d9b0dc0 2024-09-12T03:26:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.080+0000 7f8c537fe640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:26:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.081+0000 7f8c53fff640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400b940 tx=0x7f8c4400be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce7ddf6819db93e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.081+0000 7f8c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:26:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4401b440 con 0x56298d9b0dc0 2024-09-12T03:26:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c480043d0 con 0x56298d9b0dc0 2024-09-12T03:26:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c4401b5e0 con 0x56298d9b0dc0 2024-09-12T03:26:50.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8c4401b7e0 con 0x56298d9b0dc0 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c5c97d740 10 client.26300 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.082+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56298d9c9d60 con 0x56298d9b0dc0 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c44011920 con 0x56298d9b0dc0 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f8c44019020 con 0x56298d9b0dc0 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 10 client.26300 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 10 client.26300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 20 client.26300 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298d9cb6f0 0x56298d9cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 4 client.26300 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:26:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.083+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8c480043d0 con 0x56298d9cb6f0 2024-09-12T03:26:50.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.084+0000 7f8c537fe640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298d9cb6f0 0x56298d9cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.085+0000 7f8c537fe640 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298d9cb6f0 0x56298d9cdb70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b4594425d0e6ba server_cookie=5639f598156f24fb in_seq=0 out_seq=0 2024-09-12T03:26:50.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.085+0000 7f8c337fe640 10 client.26300 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c337fe640 1 -- 172.21.15.19:0/1661901167 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f8c480043d0 con 0x56298d9cb6f0 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c337fe640 10 client.26300 handle_command_reply: tid=0 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 client.26300 shutdown 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 msgr2=0x56298d9b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400b940 tx=0x7f8c4400be00 comp rx=0 tx=0).stop 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 shutdown_connections 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9bb680 0x56298d9b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298d9cb6f0 0x56298d9cdb70 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b0dc0 0x56298d9b8370 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/1661901167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 >> 172.21.15.19:0/1661901167 conn(0x56298d9aca30 msgr2=0x56298d9b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.088+0000 7f8c337fe640 0 client.26300 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.089+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 shutdown_connections 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.089+0000 7f8c5c97d740 1 -- 172.21.15.19:0/1661901167 wait complete. 2024-09-12T03:26:50.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.089+0000 7f8c5c97d740 20 client.26300 trim_cache size 0 max 16384 2024-09-12T03:26:50.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.116+0000 7f8c5c97d740 1 Processor -- start 2024-09-12T03:26:50.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.116+0000 7f8c5c97d740 1 -- start start 2024-09-12T03:26:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9bb680 0x56298d9d3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b3c70 con 0x56298d9b04a0 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b5930 con 0x56298d9bb680 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c5c97d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b4340 con 0x56298d9b0dc0 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:33624/0 (socket says 172.21.15.19:33624) 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 -- 172.21.15.19:0/291861849 learned_addr learned my addr 172.21.15.19:0/291861849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c53fff640 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9bb680 0x56298d9d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 -- 172.21.15.19:0/291861849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 msgr2=0x56298d9cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 -- 172.21.15.19:0/291861849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9bb680 msgr2=0x56298d9d3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9bb680 0x56298d9d3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.117+0000 7f8c537fe640 1 -- 172.21.15.19:0/291861849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56298d9b0bf0 con 0x56298d9b04a0 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.118+0000 7f8c537fe640 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035040 tx=0x7f8c480303c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9a7c1590293d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.118+0000 7f8c39ffb640 1 -- 172.21.15.19:0/291861849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48047070 con 0x56298d9b04a0 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.118+0000 7f8c39ffb640 1 -- 172.21.15.19:0/291861849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48048040 con 0x56298d9b04a0 2024-09-12T03:26:50.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.118+0000 7f8c39ffb640 1 -- 172.21.15.19:0/291861849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4804ce80 con 0x56298d9b04a0 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 -- 172.21.15.19:0/291861849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 msgr2=0x56298d9b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8c48035040 tx=0x7f8c480303c0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 -- 172.21.15.19:0/291861849 shutdown_connections 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9bb680 0x56298d9d3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/291861849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9b04a0 0x56298d9b8340 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 -- 172.21.15.19:0/291861849 >> 172.21.15.19:0/291861849 conn(0x56298d9aca30 msgr2=0x56298d9b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 -- 172.21.15.19:0/291861849 shutdown_connections 2024-09-12T03:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.120+0000 7f8c5c97d740 1 -- 172.21.15.19:0/291861849 wait complete. 2024-09-12T03:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.121+0000 7f8c5c97d740 1 -- start start 2024-09-12T03:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.121+0000 7f8c5c97d740 1 Processor -- start 2024-09-12T03:26:50.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.122+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.122+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c5c97d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:40754/0 (socket says 172.21.15.19:40754) 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 -- 172.21.15.19:0/3146000080 learned_addr learned my addr 172.21.15.19:0/3146000080 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b51a0 con 0x56298d9bb680 2024-09-12T03:26:50.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b4ad0 con 0x56298d9b04a0 2024-09-12T03:26:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8c541b1b90 con 0x56298d9b0dc0 2024-09-12T03:26:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c5c97d740 10 client.0 _ll_register_callbacks cb 0x7f8c44004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:26:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c53fff640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 -- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 msgr2=0x56298d9c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 -- 172.21.15.19:0/3146000080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 msgr2=0x56298d9c0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9c0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.123+0000 7f8c537fe640 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56298d9c1740 con 0x56298d9b04a0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c537fe640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006010 tx=0x7f8c48005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=265b95ebf4334ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48047040 con 0x56298d9b04a0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8c48005410 con 0x56298d9b04a0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c48048040 con 0x56298d9b04a0 2024-09-12T03:26:50.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8c48037040 con 0x56298d9b04a0 2024-09-12T03:26:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c5c97d740 10 client.26306 fetch_fsmap learned FSMap version 36 2024-09-12T03:26:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.124+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56298da360b0 con 0x56298d9b04a0 2024-09-12T03:26:50.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.125+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c48048040 con 0x56298d9b04a0 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.125+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f8c48004860 con 0x56298d9b04a0 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.125+0000 7f8c5c97d740 10 client.26306 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.125+0000 7f8c5c97d740 10 client.26306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.126+0000 7f8c5c97d740 20 client.26306 populate_metadata read hostname 'smithi019' 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.126+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298da362e0 0x56298da386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.126+0000 7f8c5c97d740 4 client.26306 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.126+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56298d923850 con 0x56298da362e0 2024-09-12T03:26:50.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.126+0000 7f8c53fff640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298da362e0 0x56298da386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:50.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.129+0000 7f8c53fff640 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298da362e0 0x56298da386d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d53e1afa6bd0f15 server_cookie=1a61db0a33600d40 in_seq=0 out_seq=0 2024-09-12T03:26:50.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.129+0000 7f8c337fe640 10 client.26306 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:50.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.130+0000 7f8c337fe640 1 -- 172.21.15.19:0/3146000080 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x56298d923850 con 0x56298da362e0 2024-09-12T03:26:50.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.130+0000 7f8c337fe640 10 client.26306 handle_command_reply: tid=0 2024-09-12T03:26:50.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.130+0000 7f8c5c97d740 1 client.26306 shutdown 2024-09-12T03:26:50.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 msgr2=0x56298d9c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006010 tx=0x7f8c48005780 comp rx=0 tx=0).stop 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 shutdown_connections 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56298d9bb680 0x56298d9c0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56298d9b0dc0 0x56298d9c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56298da362e0 0x56298da386d0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c337fe640 0 client.26306 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:26:50.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 --2- 172.21.15.19:0/3146000080 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56298d9b04a0 0x56298d9c39c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 >> 172.21.15.19:0/3146000080 conn(0x56298d9aca30 msgr2=0x56298d9b9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 shutdown_connections 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 1 -- 172.21.15.19:0/3146000080 wait complete. 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.131+0000 7f8c5c97d740 20 client.26306 trim_cache size 0 max 16384 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (8126 inodes in the stack)", 2024-09-12T03:26:50.133 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: "3e6f4c14-b810-4893-8ed4-81cec1c1d24d": { 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "3e6f4c14-b810-4893-8ed4-81cec1c1d24d", 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:26:50.134 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:26:50.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.132+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 msgr2=0x7f8c3c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.132+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 0x7f8c3c0a5190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009fd0 tx=0x7f8c4c00cf90 comp rx=0 tx=0).stop 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.132+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 msgr2=0x7f8c5419e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.132+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 0x7f8c5419e8b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8c5407aa10 tx=0x7f8c4c00ef10 comp rx=0 tx=0).stop 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 shutdown_connections 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8c3c0a2cd0 0x7f8c3c0a5190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8c54112a30 0x7f8c5419e8b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c54077040 0x7f8c541a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 --2- 172.21.15.19:0/3554085767 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8c54076720 0x7f8c541a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 >> 172.21.15.19:0/3554085767 conn(0x7f8c540ffab0 msgr2=0x7f8c54101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 shutdown_connections 2024-09-12T03:26:50.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:50.133+0000 7f8c5a5a4640 1 -- 172.21.15.19:0/3554085767 wait complete. 2024-09-12T03:26:50.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:50.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:26:50.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:50.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:26:50.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:50.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:26:51.178 INFO:tasks.cephfs.filesystem:scrub status for tag:3e6f4c14-b810-4893-8ed4-81cec1c1d24d - {'path': '/', 'tag': '3e6f4c14-b810-4893-8ed4-81cec1c1d24d', 'options': 'recursive,force'} 2024-09-12T03:26:51.401 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:51 smithi121 ceph-mon[34963]: pgmap v571: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T03:26:51.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:51 smithi019 ceph-mon[30953]: pgmap v571: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T03:26:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:51 smithi135 ceph-mon[35474]: pgmap v571: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-09-12T03:26:52.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:52 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:52.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:52 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:52.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:52 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:53 smithi019 ceph-mon[30953]: pgmap v572: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 31 op/s 2024-09-12T03:26:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:53 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:53 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:53 smithi135 ceph-mon[35474]: pgmap v572: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 31 op/s 2024-09-12T03:26:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:53 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:53 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:53 smithi121 ceph-mon[34963]: pgmap v572: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 22 KiB/s wr, 31 op/s 2024-09-12T03:26:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:53 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:53 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:54 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:54 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:54 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:54 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:54 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:54.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:54 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:26:55.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:55 smithi019 ceph-mon[30953]: pgmap v573: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 KiB/s wr, 37 op/s 2024-09-12T03:26:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:55 smithi135 ceph-mon[35474]: pgmap v573: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 KiB/s wr, 37 op/s 2024-09-12T03:26:55.551 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:55 smithi121 ceph-mon[34963]: pgmap v573: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 23 KiB/s wr, 37 op/s 2024-09-12T03:26:57.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:57 smithi019 ceph-mon[30953]: pgmap v574: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 KiB/s wr, 51 op/s 2024-09-12T03:26:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:57 smithi135 ceph-mon[35474]: pgmap v574: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 KiB/s wr, 51 op/s 2024-09-12T03:26:57.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:57 smithi121 ceph-mon[34963]: pgmap v574: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 19 KiB/s wr, 51 op/s 2024-09-12T03:26:58.866 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-12T03:26:58.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:26:59.139 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:26:59.427 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:26:59 smithi019 ceph-mon[30953]: pgmap v575: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 KiB/s wr, 54 op/s 2024-09-12T03:26:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:26:59 smithi135 ceph-mon[35474]: pgmap v575: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 KiB/s wr, 54 op/s 2024-09-12T03:26:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 -- 172.21.15.19:0/905143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100310 msgr2=0x7f63001006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 --2- 172.21.15.19:0/905143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100310 0x7f63001006f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f62fc009900 tx=0x7f62fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 -- 172.21.15.19:0/905143855 shutdown_connections 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 --2- 172.21.15.19:0/905143855 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300106f30 0x7f630010eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 --2- 172.21.15.19:0/905143855 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300100cc0 0x7f6300106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 --2- 172.21.15.19:0/905143855 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100310 0x7f63001006f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 -- 172.21.15.19:0/905143855 >> 172.21.15.19:0/905143855 conn(0x7f6300077210 msgr2=0x7f63000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 -- 172.21.15.19:0/905143855 shutdown_connections 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 -- 172.21.15.19:0/905143855 wait complete. 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.623+0000 7f6307f6e640 1 Processor -- start 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 -- start start 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300100310 0x7f6300198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.650 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300106f30 0x7f630019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6300111fb0 con 0x7f6300100cc0 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6300111e30 con 0x7f6300106f30 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6307f6e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6300112130 con 0x7f6300100310 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36230/0 (socket says 172.21.15.19:36230) 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 -- 172.21.15.19:0/1838352540 learned_addr learned my addr 172.21.15.19:0/1838352540 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f630650b640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300106f30 0x7f630019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305d0a640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300100310 0x7f6300198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 -- 172.21.15.19:0/1838352540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300100310 msgr2=0x7f6300198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300100310 0x7f6300198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.651 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 -- 172.21.15.19:0/1838352540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300106f30 msgr2=0x7f630019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300106f30 0x7f630019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.624+0000 7f6305509640 1 -- 172.21.15.19:0/1838352540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630019d3c0 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f6305509640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c920 tx=0x7f62f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0007c70 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f0007e10 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630019d6b0 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f0018890 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.625+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a4f90 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.626+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63001006f0 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.627+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62f0018a70 con 0x7f6300100cc0 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.628+0000 7f62f6ffd640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 0x7f62ec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:26:59.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.628+0000 7f6305d0a640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 0x7f62ec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:26:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.628+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f62f00cf200 con 0x7f6300100cc0 2024-09-12T03:26:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.628+0000 7f6305d0a640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 0x7f62ec0a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0096c0 tx=0x7f62fc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:26:59.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.632+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f62f0014030 con 0x7f6300100cc0 2024-09-12T03:26:59.653 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:26:59 smithi121 ceph-mon[34963]: pgmap v575: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 17 KiB/s wr, 54 op/s 2024-09-12T03:26:59.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.847+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f630002d7c0 con 0x7f6300100cc0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.848+0000 7f62f6ffd640 1 -- 172.21.15.19:0/1838352540 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+12002 (secure 0 0 0) 0x7f62f009c170 con 0x7f6300100cc0 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:26:59.849 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":36,"btime":"2024-09-12T03:24:43:270632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:24:43.042444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:26:59.850 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 36 2024-09-12T03:26:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.850+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 msgr2=0x7f62ec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.850+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 0x7f62ec0a50c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0096c0 tx=0x7f62fc000bc0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.850+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 msgr2=0x7f6300198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:26:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.850+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f62f000c920 tx=0x7f62f000cde0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 shutdown_connections 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6300106f30 0x7f630019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62ec0a2c00 0x7f62ec0a50c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6300100cc0 0x7f6300198950 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 --2- 172.21.15.19:0/1838352540 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6300100310 0x7f6300198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 >> 172.21.15.19:0/1838352540 conn(0x7f6300077210 msgr2=0x7f6300105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:26:59.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 shutdown_connections 2024-09-12T03:26:59.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:26:59.851+0000 7f6307f6e640 1 -- 172.21.15.19:0/1838352540 wait complete. 2024-09-12T03:27:00.529 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:00 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1838352540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:00 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1838352540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:00.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:00 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1838352540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:00.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-12T03:27:00.985 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-12T03:27:01.252 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:01.374 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:01 smithi019 ceph-mon[30953]: pgmap v576: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 KiB/s wr, 52 op/s 2024-09-12T03:27:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:01 smithi135 ceph-mon[35474]: pgmap v576: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 KiB/s wr, 52 op/s 2024-09-12T03:27:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:01 smithi121 ceph-mon[34963]: pgmap v576: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 16 KiB/s wr, 52 op/s 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- 172.21.15.19:0/3640680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 msgr2=0x7f97741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 --2- 172.21.15.19:0/3640680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f97741021c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f9764009900 tx=0x7f976402a410 comp rx=0 tx=0).stop 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- 172.21.15.19:0/3640680 shutdown_connections 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 --2- 172.21.15.19:0/3640680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774106f10 0x7f977410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 --2- 172.21.15.19:0/3640680 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774102790 0x7f97741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 --2- 172.21.15.19:0/3640680 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f97741021c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.778 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- 172.21.15.19:0/3640680 >> 172.21.15.19:0/3640680 conn(0x7f97740fd8c0 msgr2=0x7f97740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- 172.21.15.19:0/3640680 shutdown_connections 2024-09-12T03:27:01.779 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- 172.21.15.19:0/3640680 wait complete. 2024-09-12T03:27:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 Processor -- start 2024-09-12T03:27:01.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.778+0000 7f977a0e2640 1 -- start start 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774102790 0x7f9774198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774106f10 0x7f977419ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111fc0 con 0x7f9774101de0 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9774111e40 con 0x7f9774102790 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9774112140 con 0x7f9774106f10 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36734/0 (socket says 172.21.15.19:36734) 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 -- 172.21.15.19:0/1349383800 learned_addr learned my addr 172.21.15.19:0/1349383800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f9773fff640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774106f10 0x7f977419ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f976bfff640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774102790 0x7f9774198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 -- 172.21.15.19:0/1349383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774106f10 msgr2=0x7f977419ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774106f10 0x7f977419ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 -- 172.21.15.19:0/1349383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774102790 msgr2=0x7f9774198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774102790 0x7f9774198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 -- 172.21.15.19:0/1349383800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977419d3b0 con 0x7f9774101de0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97737fe640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9764004770 tx=0x7f9764003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97717fa640 1 -- 172.21.15.19:0/1349383800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976403f070 con 0x7f9774101de0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97717fa640 1 -- 172.21.15.19:0/1349383800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764031ca0 con 0x7f9774101de0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977419d640 con 0x7f9774101de0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.779+0000 7f97717fa640 1 -- 172.21.15.19:0/1349383800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764035030 con 0x7f9774101de0 2024-09-12T03:27:01.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.780+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97741a4f80 con 0x7f9774101de0 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.782+0000 7f97717fa640 1 -- 172.21.15.19:0/1349383800 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9764042070 con 0x7f9774101de0 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.782+0000 7f977c4bb740 1 Processor -- start 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.782+0000 7f977c4bb740 1 -- start start 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.782+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac1552a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1561d50 0x55fac1564140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f977c4bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fac14b82a0 con 0x55fac1552f40 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f977c4bb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fac145fb10 con 0x55fac1552620 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f977c4bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fac14b4260 con 0x55fac1561d50 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97717fa640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 0x7f97600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f9773fff640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 0x7f97600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36736/0 (socket says 172.21.15.19:36736) 2024-09-12T03:27:01.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 -- 172.21.15.19:0/3254650073 learned_addr learned my addr 172.21.15.19:0/3254650073 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f9773fff640 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac1552a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97717fa640 1 -- 172.21.15.19:0/1349383800 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97640ef830 con 0x7f9774101de0 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f976bfff640 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1561d50 0x55fac1564140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 -- 172.21.15.19:0/3254650073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1561d50 msgr2=0x55fac1564140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1561d50 0x55fac1564140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 -- 172.21.15.19:0/3254650073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 msgr2=0x55fac1552a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac1552a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 -- 172.21.15.19:0/3254650073 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac1229b90 con 0x55fac1552f40 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f9773fff640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 0x7f97600a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f976c007fd0 tx=0x7f976c007c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.783+0000 7f97737fe640 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f97640bc030 tx=0x7f976408f220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d750c11f902e5ef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f9769ffb640 1 -- 172.21.15.19:0/3254650073 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764049030 con 0x55fac1552f40 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f9769ffb640 1 -- 172.21.15.19:0/3254650073 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764031e50 con 0x55fac1552f40 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f9769ffb640 1 -- 172.21.15.19:0/3254650073 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640c4d50 con 0x55fac1552f40 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 -- 172.21.15.19:0/3254650073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 msgr2=0x55fac1561810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f97640bc030 tx=0x7f976408f220 comp rx=0 tx=0).stop 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 -- 172.21.15.19:0/3254650073 shutdown_connections 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1561d50 0x55fac1564140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac1561810 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3254650073 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac1552a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 -- 172.21.15.19:0/3254650073 >> 172.21.15.19:0/3254650073 conn(0x55fac154ecf0 msgr2=0x55fac154f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 -- 172.21.15.19:0/3254650073 shutdown_connections 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.784+0000 7f977c4bb740 1 -- 172.21.15.19:0/3254650073 wait complete. 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 -- start start 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 Processor -- start 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac155ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 0x55fac155f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fac15646e0 con 0x55fac1561d50 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fac14406e0 con 0x55fac1552620 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fac1564860 con 0x55fac1552f40 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f977c4bb740 10 client.0 _ll_register_callbacks cb 0x55fac1464a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36744/0 (socket says 172.21.15.19:36744) 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.785+0000 7f9773fff640 1 -- 172.21.15.19:0/3189443129 learned_addr learned my addr 172.21.15.19:0/3189443129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f976bfff640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 0x55fac155f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f97737fe640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac155ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 -- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 msgr2=0x55fac155f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 0x55fac155f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 -- 172.21.15.19:0/3189443129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 msgr2=0x55fac155ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac155ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 -- 172.21.15.19:0/3189443129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac156dc20 con 0x55fac1561d50 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f976bfff640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 0x55fac155f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9773fff640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f976c007f20 tx=0x7f976c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91e2633f046721b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c024030 con 0x55fac1561d50 2024-09-12T03:27:01.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f976c010cc0 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9764031e50 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f976c020460 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f977c4bb740 10 client.16860 fetch_fsmap learned FSMap version 36 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.786+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fac1570570 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9764031e50 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f976c01f580 con 0x55fac1561d50 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 10 client.16860 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 10 client.16860 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 20 client.16860 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1571aa0 0x55fac1573ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 4 client.16860 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:01.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac14c59d0 con 0x55fac1571aa0 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.787+0000 7f97737fe640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1571aa0 0x55fac1573ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.788+0000 7f9752ffd640 10 client.16860 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.788+0000 7f97737fe640 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1571aa0 0x55fac1573ee0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef902b8c05f90256 server_cookie=64ea501f92bcf75d in_seq=0 out_seq=0 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f9752ffd640 1 -- 172.21.15.19:0/3189443129 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fac14c59d0 con 0x55fac1571aa0 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f9752ffd640 10 client.16860 handle_command_reply: tid=0 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 client.16860 shutdown 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 msgr2=0x55fac155f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f976c007f20 tx=0x7f976c01a000 comp rx=0 tx=0).stop 2024-09-12T03:27:01.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 shutdown_connections 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1561d50 0x55fac155f7b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552f40 0x55fac155f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1571aa0 0x55fac1573ee0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 --2- 172.21.15.19:0/3189443129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac1552620 0x55fac155ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 >> 172.21.15.19:0/3189443129 conn(0x55fac154ecf0 msgr2=0x55fac1554a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f9752ffd640 0 client.16860 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 shutdown_connections 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 1 -- 172.21.15.19:0/3189443129 wait complete. 2024-09-12T03:27:01.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.789+0000 7f977c4bb740 20 client.16860 trim_cache size 0 max 16384 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 Processor -- start 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 -- start start 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac155a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac155f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97741ab3a0 con 0x55fac1552f40 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97741ad060 con 0x55fac154ecf0 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f977c4bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97741aba70 con 0x55fac1552620 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36760/0 (socket says 172.21.15.19:36760) 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.806+0000 7f9773fff640 1 -- 172.21.15.19:0/1239415887 learned_addr learned my addr 172.21.15.19:0/1239415887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f97737fe640 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac155a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 -- 172.21.15.19:0/1239415887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 msgr2=0x55fac155f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac155f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 -- 172.21.15.19:0/1239415887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 msgr2=0x55fac155a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac155a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 -- 172.21.15.19:0/1239415887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac154a260 con 0x55fac1552f40 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f97737fe640 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac155a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:01.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f9773fff640 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f976c013010 tx=0x7f976c007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f20de91d194dec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f97697fa640 1 -- 172.21.15.19:0/1239415887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c038030 con 0x55fac1552f40 2024-09-12T03:27:01.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f97697fa640 1 -- 172.21.15.19:0/1239415887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c010ca0 con 0x55fac1552f40 2024-09-12T03:27:01.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.807+0000 7f97697fa640 1 -- 172.21.15.19:0/1239415887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c024040 con 0x55fac1552f40 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- 172.21.15.19:0/1239415887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 msgr2=0x55fac15797f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f976c013010 tx=0x7f976c007800 comp rx=0 tx=0).stop 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- 172.21.15.19:0/1239415887 shutdown_connections 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15797f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac155f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 --2- 172.21.15.19:0/1239415887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac155a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- 172.21.15.19:0/1239415887 >> 172.21.15.19:0/1239415887 conn(0x55fac155f910 msgr2=0x55fac1560db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- 172.21.15.19:0/1239415887 shutdown_connections 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- 172.21.15.19:0/1239415887 wait complete. 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 -- start start 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.808+0000 7f977c4bb740 1 Processor -- start 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac1569c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac156a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97741ac8d0 con 0x55fac1552f40 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97741ac200 con 0x55fac154ecf0 2024-09-12T03:27:01.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97741a92e0 con 0x55fac1552620 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f977c4bb740 10 client.0 _ll_register_callbacks cb 0x7f976c013310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36774/0 (socket says 172.21.15.19:36774) 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 -- 172.21.15.19:0/111246929 learned_addr learned my addr 172.21.15.19:0/111246929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f97737fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac1569c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f976bfff640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac156a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 -- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 msgr2=0x55fac156a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac156a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 -- 172.21.15.19:0/111246929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 msgr2=0x55fac1569c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac1569c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.809+0000 7f9773fff640 1 -- 172.21.15.19:0/111246929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac1566a20 con 0x55fac1552f40 2024-09-12T03:27:01.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9773fff640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f976c014010 tx=0x7f976c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d560610bff93621 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c041030 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f976c01fb90 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f976c010ca0 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f976c024040 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 10 client.16866 fetch_fsmap learned FSMap version 36 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fac15dd380 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976c010ca0 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 36) ==== 8426+0+0 (secure 0 0 0) 0x7f976c020990 con 0x55fac1552f40 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 10 client.16866 fetch_fsmap finished waiting for FSMap version 36 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 10 client.16866 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 20 client.16866 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:01.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1561d50 0x55fac15dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 4 client.16866 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-12T03:27:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.810+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x55fac14bfd20 con 0x55fac1561d50 2024-09-12T03:27:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.811+0000 7f97737fe640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1561d50 0x55fac15dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.811+0000 7f9752ffd640 10 client.16866 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:01.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:01.811+0000 7f97737fe640 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1561d50 0x55fac15dfb20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1994e67897307e7 server_cookie=d03502ff4fe0ce0 in_seq=0 out_seq=0 2024-09-12T03:27:03.551 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:03 smithi135 ceph-mon[35474]: pgmap v577: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 KiB/s wr, 51 op/s 2024-09-12T03:27:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:03 smithi121 ceph-mon[34963]: pgmap v577: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 KiB/s wr, 51 op/s 2024-09-12T03:27:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:03 smithi019 ceph-mon[30953]: pgmap v577: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 15 KiB/s wr, 51 op/s 2024-09-12T03:27:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:04.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:04.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:04.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:04.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:04.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:05 smithi135 ceph-mon[35474]: pgmap v578: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 KiB/s wr, 43 op/s 2024-09-12T03:27:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:05 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:05 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:05 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:05 smithi121 ceph-mon[34963]: pgmap v578: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 KiB/s wr, 43 op/s 2024-09-12T03:27:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:05 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:05 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:05 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:05 smithi019 ceph-mon[30953]: pgmap v578: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 15 KiB/s wr, 43 op/s 2024-09-12T03:27:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:05 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:05.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:05 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:05.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:05 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:07.467 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:07 smithi019 ceph-mon[30953]: pgmap v579: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 KiB/s wr, 36 op/s 2024-09-12T03:27:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:07 smithi135 ceph-mon[35474]: pgmap v579: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 KiB/s wr, 36 op/s 2024-09-12T03:27:07.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:07 smithi121 ceph-mon[34963]: pgmap v579: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 15 KiB/s wr, 36 op/s 2024-09-12T03:27:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.995+0000 7f9752ffd640 1 -- 172.21.15.19:0/111246929 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fac14bfd20 con 0x55fac1561d50 2024-09-12T03:27:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.995+0000 7f9752ffd640 10 client.16866 handle_command_reply: tid=0 2024-09-12T03:27:08.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.995+0000 7f977c4bb740 1 client.16866 shutdown 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 msgr2=0x55fac15661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f976c014010 tx=0x7f976c004ab0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 shutdown_connections 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fac1552f40 0x55fac15661f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fac1552620 0x55fac156a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fac1561d50 0x55fac15dfb20 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 --2- 172.21.15.19:0/111246929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fac154ecf0 0x55fac1569c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f9752ffd640 0 client.16866 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 >> 172.21.15.19:0/111246929 conn(0x55fac155f910 msgr2=0x55fac1560590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 shutdown_connections 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 1 -- 172.21.15.19:0/111246929 wait complete. 2024-09-12T03:27:08.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.996+0000 7f977c4bb740 20 client.16866 trim_cache size 0 max 16384 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 36, 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 26, 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "1bfc07aa": { 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 26, 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:08.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 136.6, 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:09.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 msgr2=0x7f97600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 0x7f97600a50c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f976c007fd0 tx=0x7f976c007c60 comp rx=0 tx=0).stop 2024-09-12T03:27:09.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 msgr2=0x7f9774198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9764004770 tx=0x7f9764003c10 comp rx=0 tx=0).stop 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 shutdown_connections 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9774106f10 0x7f977419ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9774102790 0x7f9774198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97600a2c00 0x7f97600a50c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 --2- 172.21.15.19:0/1349383800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9774101de0 0x7f9774198400 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 >> 172.21.15.19:0/1349383800 conn(0x7f97740fd8c0 msgr2=0x7f97740fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 shutdown_connections 2024-09-12T03:27:09.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:08.997+0000 7f977a0e2640 1 -- 172.21.15.19:0/1349383800 wait complete. 2024-09-12T03:27:09.392 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:09 smithi019 ceph-mon[30953]: pgmap v580: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 25 op/s 2024-09-12T03:27:09.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:09 smithi135 ceph-mon[35474]: pgmap v580: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 25 op/s 2024-09-12T03:27:09.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:09 smithi121 ceph-mon[34963]: pgmap v580: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 14 KiB/s wr, 25 op/s 2024-09-12T03:27:09.837 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1bfc07aa', quiesce took 11.0 seconds. Will release after: 5.600000000000001 2024-09-12T03:27:10.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:10 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:10.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:10 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:10.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:10 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:11 smithi135 ceph-mon[35474]: pgmap v581: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-12T03:27:11.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:11 smithi121 ceph-mon[34963]: pgmap v581: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-12T03:27:11.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:11 smithi019 ceph-mon[30953]: pgmap v581: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-09-12T03:27:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:13 smithi135 ceph-mon[35474]: pgmap v582: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-09-12T03:27:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:13 smithi121 ceph-mon[34963]: pgmap v582: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-09-12T03:27:13.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:13 smithi019 ceph-mon[30953]: pgmap v582: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 21 op/s 2024-09-12T03:27:15.438 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:15.469 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:15 smithi019 ceph-mon[30953]: pgmap v583: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-12T03:27:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:15 smithi135 ceph-mon[35474]: pgmap v583: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-12T03:27:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:15 smithi121 ceph-mon[34963]: pgmap v583: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-09-12T03:27:15.715 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 -- 172.21.15.19:0/736302670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102ef0 msgr2=0x7ff72810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/736302670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102ef0 0x7ff72810da40 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff720009900 tx=0x7ff72002a480 comp rx=0 tx=0).stop 2024-09-12T03:27:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 -- 172.21.15.19:0/736302670 shutdown_connections 2024-09-12T03:27:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/736302670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102ef0 0x7ff72810da40 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/736302670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728102570 0x7ff7281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/736302670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728100e20 0x7ff728101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 -- 172.21.15.19:0/736302670 >> 172.21.15.19:0/736302670 conn(0x7ff7280fc7d0 msgr2=0x7ff7280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 -- 172.21.15.19:0/736302670 shutdown_connections 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.225+0000 7ff72dfb3640 1 -- 172.21.15.19:0/736302670 wait complete. 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 Processor -- start 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 -- start start 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100e20 0x7ff728198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728102ef0 0x7ff72819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff728110f80 con 0x7ff728102570 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff728110e00 con 0x7ff728102ef0 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff72dfb3640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff728111100 con 0x7ff728100e20 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff726ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff726ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53894/0 (socket says 172.21.15.19:53894) 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff726ffd640 1 -- 172.21.15.19:0/902773392 learned_addr learned my addr 172.21.15.19:0/902773392 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff7277fe640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100e20 0x7ff728198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.226+0000 7ff727fff640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728102ef0 0x7ff72819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 -- 172.21.15.19:0/902773392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100e20 msgr2=0x7ff728198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100e20 0x7ff728198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 -- 172.21.15.19:0/902773392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728102ef0 msgr2=0x7ff72819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728102ef0 0x7ff72819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 -- 172.21.15.19:0/902773392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72819d400 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff726ffd640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9f0 tx=0x7ff71c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00cd80 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff71c005500 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff71c010690 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72819d6f0 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.227+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7281a4fd0 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.228+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff728101200 con 0x7ff728102570 2024-09-12T03:27:16.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.229+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff71c0046c0 con 0x7ff728102570 2024-09-12T03:27:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.230+0000 7ff724ff9640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 0x7ff7100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.230+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff71c0cf640 con 0x7ff728102570 2024-09-12T03:27:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.230+0000 7ff7277fe640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 0x7ff7100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.231+0000 7ff7277fe640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 0x7ff7100a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff718009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:16.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.233+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff71c0541b0 con 0x7ff728102570 2024-09-12T03:27:16.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.445+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7281997b0 con 0x7ff728102570 2024-09-12T03:27:16.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.446+0000 7ff724ff9640 1 -- 172.21.15.19:0/902773392 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11996 (secure 0 0 0) 0x7ff71c09c5b0 con 0x7ff728102570 2024-09-12T03:27:16.446 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:27:16.447 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":38,"btime":"2024-09-12T03:27:16:214870+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:27:15.645446+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:27:16.447 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 38 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 msgr2=0x7ff7100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 0x7ff7100a5040 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff718005fd0 tx=0x7ff718009500 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 msgr2=0x7ff728198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00e9f0 tx=0x7ff71c00edf0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 shutdown_connections 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff728102ef0 0x7ff72819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7100a2b80 0x7ff7100a5040 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff728102570 0x7ff728198990 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 --2- 172.21.15.19:0/902773392 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff728100e20 0x7ff728198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:16.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.448+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 >> 172.21.15.19:0/902773392 conn(0x7ff7280fc7d0 msgr2=0x7ff72806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.449+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 shutdown_connections 2024-09-12T03:27:16.450 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:16.449+0000 7ff72dfb3640 1 -- 172.21.15.19:0/902773392 wait complete. 2024-09-12T03:27:17.126 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 1bfc07aa --release --await' 2024-09-12T03:27:17.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 1bfc07aa --release --await 2024-09-12T03:27:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:17 smithi019 ceph-mon[30953]: pgmap v584: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-09-12T03:27:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:17 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:17.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:17 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/902773392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.431 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:17 smithi135 ceph-mon[35474]: pgmap v584: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-09-12T03:27:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:17 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:17 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/902773392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:17 smithi121 ceph-mon[34963]: pgmap v584: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 21 op/s 2024-09-12T03:27:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:17 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:17 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/902773392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/2503246702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc1037a0 msgr2=0x7ff3bc10bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/2503246702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc10bc00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009900 tx=0x7ff3b802a410 comp rx=0 tx=0).stop 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/2503246702 shutdown_connections 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/2503246702 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/2503246702 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc10bc00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/2503246702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 0x7ff3bc1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/2503246702 >> 172.21.15.19:0/2503246702 conn(0x7ff3bc0fc7d0 msgr2=0x7ff3bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/2503246702 shutdown_connections 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.912+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/2503246702 wait complete. 2024-09-12T03:27:17.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 Processor -- start 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 -- start start 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 0x7ff3bc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1120b0 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc111f30 con 0x7ff3bc102df0 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c3fe0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc112230 con 0x7ff3bc1037a0 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53908/0 (socket says 172.21.15.19:53908) 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 -- 172.21.15.19:0/472209786 learned_addr learned my addr 172.21.15.19:0/472209786 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:17.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 0x7ff3bc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c157b640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 -- 172.21.15.19:0/472209786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc1037a0 msgr2=0x7ff3bc198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 -- 172.21.15.19:0/472209786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 msgr2=0x7ff3bc198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 0x7ff3bc198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.913+0000 7ff3c257d640 1 -- 172.21.15.19:0/472209786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc19d540 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3c257d640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cce0 tx=0x7ff3b400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3b2ffd640 1 -- 172.21.15.19:0/472209786 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b400f450 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc19d830 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3b2ffd640 1 -- 172.21.15.19:0/472209786 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b400fad0 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3b2ffd640 1 -- 172.21.15.19:0/472209786 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4018c10 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.914+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc1a5110 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.916+0000 7ff3b2ffd640 1 -- 172.21.15.19:0/472209786 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff3b400f5f0 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.916+0000 7ff3c63b9740 1 Processor -- start 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 -- start start 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7220d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7223630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570b717b620 con 0x5570b7216060 2024-09-12T03:27:17.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5570b7122e50 con 0x5570b7216980 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5570b71775e0 con 0x5570b7221240 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53924/0 (socket says 172.21.15.19:53924) 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 -- 172.21.15.19:0/2885916135 learned_addr learned my addr 172.21.15.19:0/2885916135 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7223630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7220d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 -- 172.21.15.19:0/2885916135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 msgr2=0x5570b7223630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7223630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3b2ffd640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 0x7ff3a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 -- 172.21.15.19:0/2885916135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 msgr2=0x5570b7220d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7220d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c157b640 1 -- 172.21.15.19:0/2885916135 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b7113d40 con 0x5570b7216060 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.917+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7220d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3c257d640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 0x7ff3a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3c157b640 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005ab0 tx=0x7ff3b8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb95585aa565ef81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3b2ffd640 1 -- 172.21.15.19:0/472209786 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff3b40cf660 con 0x7ff3bc10c2f0 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3a7fff640 1 -- 172.21.15.19:0/2885916135 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035050 con 0x5570b7216060 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3a7fff640 1 -- 172.21.15.19:0/2885916135 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8031800 con 0x5570b7216060 2024-09-12T03:27:17.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.918+0000 7ff3a7fff640 1 -- 172.21.15.19:0/2885916135 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8005650 con 0x5570b7216060 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- 172.21.15.19:0/2885916135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 msgr2=0x5570b7216440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c257d640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 0x7ff3a80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40988a0 tx=0x7ff3b40186b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005ab0 tx=0x7ff3b8005d70 comp rx=0 tx=0).stop 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- 172.21.15.19:0/2885916135 shutdown_connections 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7223630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7220d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/2885916135 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7216440 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- 172.21.15.19:0/2885916135 >> 172.21.15.19:0/2885916135 conn(0x5570b72125f0 msgr2=0x5570b72129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- 172.21.15.19:0/2885916135 shutdown_connections 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- 172.21.15.19:0/2885916135 wait complete. 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 -- start start 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.919+0000 7ff3c63b9740 1 Processor -- start 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7221240 0x5570b721ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5570b7223e10 con 0x5570b7216980 2024-09-12T03:27:17.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5570b7103f10 con 0x5570b7221240 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5570b7223f90 con 0x5570b7216060 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 10 client.0 _ll_register_callbacks cb 0x5570b7127db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53930/0 (socket says 172.21.15.19:53930) 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 -- 172.21.15.19:0/3228332288 learned_addr learned my addr 172.21.15.19:0/3228332288 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c157b640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c157b640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7221240 0x5570b721ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 -- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 msgr2=0x5570b721e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 -- 172.21.15.19:0/3228332288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7221240 msgr2=0x5570b721ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7221240 0x5570b721ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 -- 172.21.15.19:0/3228332288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b6e8a580 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c157b640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00baa0 tx=0x7ff3ac00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80189d599fd75dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac01b640 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff3ac011440 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.920+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8031800 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff3ac01b7e0 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 10 client.16884 fetch_fsmap learned FSMap version 38 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5570b6f54960 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b8031800 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7ff3ac019020 con 0x5570b7216980 2024-09-12T03:27:17.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 10 client.16884 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 10 client.16884 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 20 client.16884 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b7230fe0 0x5570b72333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 4 client.16884 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.921+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b7188b90 con 0x5570b7230fe0 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.922+0000 7ff3c157b640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b7230fe0 0x5570b72333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.922+0000 7ff3c157b640 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b7230fe0 0x5570b72333f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56856b4d487b0c40 server_cookie=4e30378dc65ece33 in_seq=0 out_seq=0 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.922+0000 7ff3a57fa640 10 client.16884 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3a57fa640 1 -- 172.21.15.19:0/3228332288 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570b7188b90 con 0x5570b7230fe0 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3a57fa640 10 client.16884 handle_command_reply: tid=0 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 client.16884 shutdown 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 msgr2=0x5570b721e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00baa0 tx=0x7ff3ac00bf60 comp rx=0 tx=0).stop 2024-09-12T03:27:17.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 shutdown_connections 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b7230fe0 0x5570b72333f0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7221240 0x5570b721ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216980 0x5570b721e730 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/3228332288 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3a57fa640 0 client.16884 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 >> 172.21.15.19:0/3228332288 conn(0x5570b72125f0 msgr2=0x5570b7218320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 shutdown_connections 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 1 -- 172.21.15.19:0/3228332288 wait complete. 2024-09-12T03:27:17.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.923+0000 7ff3c63b9740 20 client.16884 trim_cache size 0 max 16384 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 Processor -- start 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 -- start start 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7218ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1ad430 con 0x5570b7221240 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1af090 con 0x5570b7216980 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1adb00 con 0x5570b7216060 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53942/0 (socket says 172.21.15.19:53942) 2024-09-12T03:27:17.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c1d7c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7218ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 -- 172.21.15.19:0/628759530 learned_addr learned my addr 172.21.15.19:0/628759530 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 -- 172.21.15.19:0/628759530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 msgr2=0x5570b721cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 -- 172.21.15.19:0/628759530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 msgr2=0x5570b7218ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7218ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.941+0000 7ff3c157b640 1 -- 172.21.15.19:0/628759530 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b7219000 con 0x5570b7221240 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.942+0000 7ff3c157b640 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8032c00 tx=0x7ff3b8005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87244572a03001f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.942+0000 7ff3a77fe640 1 -- 172.21.15.19:0/628759530 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035040 con 0x5570b7221240 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.942+0000 7ff3a77fe640 1 -- 172.21.15.19:0/628759530 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b80055c0 con 0x5570b7221240 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.942+0000 7ff3a77fe640 1 -- 172.21.15.19:0/628759530 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8054e60 con 0x5570b7221240 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- 172.21.15.19:0/628759530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 msgr2=0x5570b7211640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8032c00 tx=0x7ff3b8005d70 comp rx=0 tx=0).stop 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- 172.21.15.19:0/628759530 shutdown_connections 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7221240 0x5570b7211640 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7218ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/628759530 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7216060 0x5570b721cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- 172.21.15.19:0/628759530 >> 172.21.15.19:0/628759530 conn(0x5570b72125f0 msgr2=0x5570b7210f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:17.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- 172.21.15.19:0/628759530 shutdown_connections 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- 172.21.15.19:0/628759530 wait complete. 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 -- start start 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.943+0000 7ff3c63b9740 1 Processor -- start 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7225a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7225f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1ae900 con 0x5570b7216060 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1ae230 con 0x5570b7216980 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1ab380 con 0x5570b7221240 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c63b9740 10 client.0 _ll_register_callbacks cb 0x7ff3ac004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:53946/0 (socket says 172.21.15.19:53946) 2024-09-12T03:27:17.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 -- 172.21.15.19:0/1487764729 learned_addr learned my addr 172.21.15.19:0/1487764729 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7225a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7225f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 -- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 msgr2=0x5570b7225f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7225f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 -- 172.21.15.19:0/1487764729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 msgr2=0x5570b7225a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7225a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 -- 172.21.15.19:0/1487764729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b7226580 con 0x5570b7216060 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3c157b640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005560 tx=0x7ff3b8032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7ee94e1a13ff9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.944+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8035040 con 0x5570b7216060 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff3b8031ad0 con 0x5570b7216060 2024-09-12T03:27:17.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b80055c0 con 0x5570b7216060 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ff3b8031cd0 con 0x5570b7216060 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 10 client.16890 fetch_fsmap learned FSMap version 38 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5570b72a0ed0 con 0x5570b7216060 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3b80055c0 con 0x5570b7216060 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 38) ==== 8414+0+0 (secure 0 0 0) 0x7ff3b80548a0 con 0x5570b7216060 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 10 client.16890 fetch_fsmap finished waiting for FSMap version 38 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 10 client.16890 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 20 client.16890 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b72a10f0 0x5570b72a3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 4 client.16890 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1bfc07aa", "release": true, "await": true}] 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1bfc07aa", "release": true, "await": true}) -- 0x5570b71830a0 con 0x5570b72a10f0 2024-09-12T03:27:17.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.945+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b72a10f0 0x5570b72a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.946+0000 7ff3c1d7c640 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b72a10f0 0x5570b72a3510 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22be7b34041bdff8 server_cookie=1653477091a19092 in_seq=0 out_seq=0 2024-09-12T03:27:17.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:17.946+0000 7ff3a57fa640 10 client.16890 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:19.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3a57fa640 1 -- 172.21.15.19:0/1487764729 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5570b71830a0 con 0x5570b72a10f0 2024-09-12T03:27:19.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3a57fa640 10 client.16890 handle_command_reply: tid=0 2024-09-12T03:27:19.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 client.16890 shutdown 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 msgr2=0x5570b7229710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005560 tx=0x7ff3b8032f30 comp rx=0 tx=0).stop 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 shutdown_connections 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5570b7221240 0x5570b7225f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5570b72a10f0 0x5570b72a3510 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5570b7216980 0x5570b7225a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 --2- 172.21.15.19:0/1487764729 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5570b7216060 0x5570b7229710 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3a57fa640 0 client.16890 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 >> 172.21.15.19:0/1487764729 conn(0x5570b72125f0 msgr2=0x5570b7211750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.064+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 shutdown_connections 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c63b9740 1 -- 172.21.15.19:0/1487764729 wait complete. 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c63b9740 20 client.16890 trim_cache size 0 max 16384 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:19.067 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 38, 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 28, 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "1bfc07aa": { 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 28, 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:27:19.068 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 136.6, 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:19.069 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:19.070 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:19.070 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:19.070 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:19.070 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:19.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 msgr2=0x7ff3a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 0x7ff3a80a5040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40988a0 tx=0x7ff3b40186b0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 msgr2=0x7ff3bc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400cce0 tx=0x7ff3b400a620 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.065+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 shutdown_connections 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff3a80a2b80 0x7ff3a80a5040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff3bc10c2f0 0x7ff3bc19ce60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff3bc1037a0 0x7ff3bc198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 --2- 172.21.15.19:0/472209786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3bc102df0 0x7ff3bc198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:19.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 >> 172.21.15.19:0/472209786 conn(0x7ff3bc0fc7d0 msgr2=0x7ff3bc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 shutdown_connections 2024-09-12T03:27:19.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:19.066+0000 7ff3c3fe0640 1 -- 172.21.15.19:0/472209786 wait complete. 2024-09-12T03:27:19.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:19 smithi019 ceph-mon[30953]: pgmap v585: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-12T03:27:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:19 smithi135 ceph-mon[35474]: pgmap v585: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-12T03:27:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:19 smithi121 ceph-mon[34963]: pgmap v585: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.2 KiB/s wr, 19 op/s 2024-09-12T03:27:19.791 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1bfc07aa', total seconds elapsed: 20.9 2024-09-12T03:27:20.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:20 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:20.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:20 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:20 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:21.179 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:27:21.455 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:21 smithi135 ceph-mon[35474]: pgmap v586: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.6 KiB/s wr, 15 op/s 2024-09-12T03:27:21.576 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:21 smithi019 ceph-mon[30953]: pgmap v586: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.6 KiB/s wr, 15 op/s 2024-09-12T03:27:21.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:21 smithi121 ceph-mon[34963]: pgmap v586: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.6 KiB/s wr, 15 op/s 2024-09-12T03:27:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.917+0000 7efebdbd5640 1 -- 172.21.15.19:0/3312583793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 msgr2=0x7efeb810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.917+0000 7efebdbd5640 1 --2- 172.21.15.19:0/3312583793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb810bb50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efeac009900 tx=0x7efeac02a480 comp rx=0 tx=0).stop 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 -- 172.21.15.19:0/3312583793 shutdown_connections 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 --2- 172.21.15.19:0/3312583793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb810c170 0x7efeb810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 --2- 172.21.15.19:0/3312583793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb810bb50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 --2- 172.21.15.19:0/3312583793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb8102d40 0x7efeb8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 -- 172.21.15.19:0/3312583793 >> 172.21.15.19:0/3312583793 conn(0x7efeb80fc7a0 msgr2=0x7efeb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 -- 172.21.15.19:0/3312583793 shutdown_connections 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 -- 172.21.15.19:0/3312583793 wait complete. 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.918+0000 7efebdbd5640 1 Processor -- start 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 -- start start 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8102d40 0x7efeb81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb810c170 0x7efeb819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8111f70 con 0x7efeb8103660 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8111df0 con 0x7efeb810c170 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efebdbd5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81120f0 con 0x7efeb8102d40 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50178/0 (socket says 172.21.15.19:50178) 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2020769854 learned_addr learned my addr 172.21.15.19:0/2020769854 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb77fe640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8102d40 0x7efeb81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb810c170 0x7efeb819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2020769854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8102d40 msgr2=0x7efeb81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8102d40 0x7efeb81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2020769854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb810c170 msgr2=0x7efeb819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb810c170 0x7efeb819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.919+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2020769854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb819d470 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efeb4ff9640 1 -- 172.21.15.19:0/2020769854 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac03f030 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb819d700 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efeb4ff9640 1 -- 172.21.15.19:0/2020769854 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac003ce0 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efeb4ff9640 1 -- 172.21.15.19:0/2020769854 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac035070 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.920+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb81a5040 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 Processor -- start 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 -- start start 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efeb4ff9640 1 -- 172.21.15.19:0/2020769854 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efeac042030 con 0x7efeb8103660 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed72b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed736140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585ed6912e0 con 0x5585ed736680 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5585ed638ce0 con 0x5585ed72b4a0 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efebffae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5585ed68d2a0 con 0x5585ed72bdc0 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50194/0 (socket says 172.21.15.19:50194) 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2760605398 learned_addr learned my addr 172.21.15.19:0/2760605398 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.923+0000 7efeb77fe640 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed736140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed72b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2760605398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 msgr2=0x5585ed736140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed736140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2760605398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 msgr2=0x5585ed72b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed72b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 -- 172.21.15.19:0/2760605398 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ed61d440 con 0x5585ed736680 2024-09-12T03:27:21.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb4ff9640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 0x7efea00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 0x7efea00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb4ff9640 1 -- 172.21.15.19:0/2020769854 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efeac0ef610 con 0x7efeb8103660 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efeac0f5c10 tx=0x7efeac0f5c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c5f8f5c1a03533 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efe9dffb640 1 -- 172.21.15.19:0/2760605398 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac049070 con 0x5585ed736680 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efe9dffb640 1 -- 172.21.15.19:0/2760605398 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac0bc640 con 0x5585ed736680 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.924+0000 7efe9dffb640 1 -- 172.21.15.19:0/2760605398 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0bbd10 con 0x5585ed736680 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- 172.21.15.19:0/2760605398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 msgr2=0x5585ed738a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efeac0f5c10 tx=0x7efeac0f5c40 comp rx=0 tx=0).stop 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 0x7efea00a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efeb00092f0 tx=0x7efeb0009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- 172.21.15.19:0/2760605398 shutdown_connections 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed738a70 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed736140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 --2- 172.21.15.19:0/2760605398 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed72b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- 172.21.15.19:0/2760605398 >> 172.21.15.19:0/2760605398 conn(0x5585ed727a30 msgr2=0x5585ed727e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- 172.21.15.19:0/2760605398 shutdown_connections 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- 172.21.15.19:0/2760605398 wait complete. 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.925+0000 7efebffae740 1 -- start start 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 Processor -- start 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72b4a0 0x5585ed735f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72bdc0 0x5585ed733200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5585ed739010 con 0x5585ed736680 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5585ed64c050 con 0x5585ed72bdc0 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5585ed739190 con 0x5585ed72b4a0 2024-09-12T03:27:21.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efebffae740 10 client.0 _ll_register_callbacks cb 0x5585ed68c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50204/0 (socket says 172.21.15.19:50204) 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 -- 172.21.15.19:0/2891614559 learned_addr learned my addr 172.21.15.19:0/2891614559 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72bdc0 0x5585ed733200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb77fe640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72b4a0 0x5585ed735f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 -- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72b4a0 msgr2=0x5585ed735f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72b4a0 0x5585ed735f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 -- 172.21.15.19:0/2891614559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72bdc0 msgr2=0x5585ed733200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72bdc0 0x5585ed733200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.926+0000 7efeb7fff640 1 -- 172.21.15.19:0/2891614559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ed4023c0 con 0x5585ed736680 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efeb7fff640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efeb0012870 tx=0x7efeb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d52d4b4fd03797 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:21.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb000ead0 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efeb000ec70 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac0bc640 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efeb001f420 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 10 client.16902 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5585ed744930 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac0bc640 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7efeb000f040 con 0x5585ed736680 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 10 client.16902 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 10 client.16902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 20 client.16902 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.927+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed745e90 0x5585ed7482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.928+0000 7efebffae740 4 client.16902 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:21.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.928+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585ed69e850 con 0x5585ed745e90 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.928+0000 7efeb77fe640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed745e90 0x5585ed7482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.928+0000 7efe96ffd640 10 client.16902 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.928+0000 7efeb77fe640 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed745e90 0x5585ed7482f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed96318f08ca462d server_cookie=e33cf98ce356c356 in_seq=0 out_seq=0 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efe96ffd640 1 -- 172.21.15.19:0/2891614559 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585ed69e850 con 0x5585ed745e90 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efe96ffd640 10 client.16902 handle_command_reply: tid=0 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 client.16902 shutdown 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 msgr2=0x5585ed733740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efeb0012870 tx=0x7efeb0004a40 comp rx=0 tx=0).stop 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 shutdown_connections 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed736680 0x5585ed733740 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed745e90 0x5585ed7482f0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72bdc0 0x5585ed733200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 --2- 172.21.15.19:0/2891614559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72b4a0 0x5585ed735f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 >> 172.21.15.19:0/2891614559 conn(0x5585ed727a30 msgr2=0x5585ed72d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efe96ffd640 0 client.16902 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 shutdown_connections 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 1 -- 172.21.15.19:0/2891614559 wait complete. 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.929+0000 7efebffae740 20 client.16902 trim_cache size 0 max 16384 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 Processor -- start 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 -- start start 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed7421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed736680 0x5585ed74dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81ad5d0 con 0x5585ed72bdc0 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81af200 con 0x5585ed72b4a0 2024-09-12T03:27:21.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efebffae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81adca0 con 0x5585ed736680 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50206/0 (socket says 172.21.15.19:50206) 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 -- 172.21.15.19:0/392912380 learned_addr learned my addr 172.21.15.19:0/392912380 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb7fff640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed736680 0x5585ed74dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb77fe640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed7421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 -- 172.21.15.19:0/392912380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed736680 msgr2=0x5585ed74dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed736680 0x5585ed74dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 -- 172.21.15.19:0/392912380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 msgr2=0x5585ed7421c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed7421c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 -- 172.21.15.19:0/392912380 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585ed727860 con 0x5585ed72bdc0 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb77fe640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed7421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.947+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efeac005d00 tx=0x7efeac005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a02e49f5fd60db7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efe9d7fa640 1 -- 172.21.15.19:0/392912380 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac049040 con 0x5585ed72bdc0 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efe9d7fa640 1 -- 172.21.15.19:0/392912380 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac0bc640 con 0x5585ed72bdc0 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efe9d7fa640 1 -- 172.21.15.19:0/392912380 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac0044e0 con 0x5585ed72bdc0 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 -- 172.21.15.19:0/392912380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 msgr2=0x5585ed72e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7efeac005d00 tx=0x7efeac005e50 comp rx=0 tx=0).stop 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 -- 172.21.15.19:0/392912380 shutdown_connections 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed736680 0x5585ed74dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72bdc0 0x5585ed72e7a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 --2- 172.21.15.19:0/392912380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed72b4a0 0x5585ed7421c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 -- 172.21.15.19:0/392912380 >> 172.21.15.19:0/392912380 conn(0x5585ed727a30 msgr2=0x5585ed72e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 -- 172.21.15.19:0/392912380 shutdown_connections 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.948+0000 7efebffae740 1 -- 172.21.15.19:0/392912380 wait complete. 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 -- start start 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 Processor -- start 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed739df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 0x5585ed73a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81aea70 con 0x5585ed72b4a0 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81ae3a0 con 0x5585ed736680 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81ab550 con 0x5585ed72bdc0 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efebffae740 10 client.0 _ll_register_callbacks cb 0x7efeb0009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:21.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50210/0 (socket says 172.21.15.19:50210) 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 -- 172.21.15.19:0/3712380781 learned_addr learned my addr 172.21.15.19:0/3712380781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed739df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 -- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 msgr2=0x5585ed739df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed739df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb7fff640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 0x5585ed73a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 -- 172.21.15.19:0/3712380781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 msgr2=0x5585ed73a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 0x5585ed73a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.949+0000 7efeb77fe640 1 -- 172.21.15.19:0/3712380781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585ed73ab30 con 0x5585ed72b4a0 2024-09-12T03:27:21.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed739df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efeb7fff640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 0x5585ed73a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efeb77fe640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efea8004540 tx=0x7efea800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0fc4f70c5d0b57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8005020 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efea80051c0 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeac0bc640 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7efea80053c0 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 10 client.16908 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5585ed7b0be0 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac0bc640 con 0x5585ed72b4a0 2024-09-12T03:27:21.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7efea8024030 con 0x5585ed72b4a0 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 10 client.16908 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 20 client.16908 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed7b1440 0x5585ed7b3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 4 client.16908 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.950+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585ed698d60 con 0x5585ed7b1440 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed7b1440 0x5585ed7b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efeb6ffd640 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed7b1440 0x5585ed7b3860 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa2c42f2f73d5fe server_cookie=a96f73f3828fd9e0 in_seq=0 out_seq=0 2024-09-12T03:27:21.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efe96ffd640 10 client.16908 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:21.976 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:21.976 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:27:21.976 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:27:21.976 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:21.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efe96ffd640 1 -- 172.21.15.19:0/3712380781 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5585ed698d60 con 0x5585ed7b1440 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efe96ffd640 10 client.16908 handle_command_reply: tid=0 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 client.16908 shutdown 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 msgr2=0x5585ed73eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efea8004540 tx=0x7efea800bd10 comp rx=0 tx=0).stop 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 shutdown_connections 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5585ed7b1440 0x5585ed7b3860 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5585ed736680 0x5585ed73a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5585ed72bdc0 0x5585ed739df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efe96ffd640 0 client.16908 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.951+0000 7efebffae740 1 --2- 172.21.15.19:0/3712380781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5585ed72b4a0 0x5585ed73eac0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 >> 172.21.15.19:0/3712380781 conn(0x5585ed727a30 msgr2=0x5585ed72e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 shutdown_connections 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebffae740 1 -- 172.21.15.19:0/3712380781 wait complete. 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebffae740 20 client.16908 trim_cache size 0 max 16384 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 msgr2=0x7efea00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 0x7efea00a5040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7efeb00092f0 tx=0x7efeb0009280 comp rx=0 tx=0).stop 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 msgr2=0x7efeb8198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efeac006010 tx=0x7efeac0042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 shutdown_connections 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb810c170 0x7efeb819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efea00a2b80 0x7efea00a5040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efeb8103660 0x7efeb8198a00 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 --2- 172.21.15.19:0/2020769854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efeb8102d40 0x7efeb81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 >> 172.21.15.19:0/2020769854 conn(0x7efeb80fc7a0 msgr2=0x7efeb80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 shutdown_connections 2024-09-12T03:27:21.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:21.952+0000 7efebdbd5640 1 -- 172.21.15.19:0/2020769854 wait complete. 2024-09-12T03:27:22.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:22 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:22.600 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:22 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:22.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:22 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:22.954 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:27:22.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:23.218 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:23.503 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:23 smithi019 ceph-mon[30953]: pgmap v587: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-12T03:27:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:23 smithi135 ceph-mon[35474]: pgmap v587: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-12T03:27:23.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:23 smithi121 ceph-mon[34963]: pgmap v587: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 3.2 KiB/s wr, 16 op/s 2024-09-12T03:27:23.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 -- 172.21.15.19:0/1741757153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 msgr2=0x7fb970110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 --2- 172.21.15.19:0/1741757153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb970110870 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb968009ca0 tx=0x7fb96802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 -- 172.21.15.19:0/1741757153 shutdown_connections 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 --2- 172.21.15.19:0/1741757153 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb970110870 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 --2- 172.21.15.19:0/1741757153 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb970101280 0x7fb97010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 --2- 172.21.15.19:0/1741757153 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb9701008d0 0x7fb970100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.717+0000 7fb97751c640 1 -- 172.21.15.19:0/1741757153 >> 172.21.15.19:0/1741757153 conn(0x7fb9700fc7a0 msgr2=0x7fb9700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- 172.21.15.19:0/1741757153 shutdown_connections 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- 172.21.15.19:0/1741757153 wait complete. 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 Processor -- start 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- start start 2024-09-12T03:27:23.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9701008d0 0x7fb97019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb970101280 0x7fb97019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb970114220 con 0x7fb97010e480 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701140a0 con 0x7fb9701008d0 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.718+0000 7fb97751c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb9701143a0 con 0x7fb970101280 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50234/0 (socket says 172.21.15.19:50234) 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 -- 172.21.15.19:0/2720557154 learned_addr learned my addr 172.21.15.19:0/2720557154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb974ab7640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb970101280 0x7fb97019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb9752b8640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9701008d0 0x7fb97019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 -- 172.21.15.19:0/2720557154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb970101280 msgr2=0x7fb97019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb970101280 0x7fb97019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 -- 172.21.15.19:0/2720557154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9701008d0 msgr2=0x7fb97019a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9701008d0 0x7fb97019a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 -- 172.21.15.19:0/2720557154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97019f690 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb975ab9640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb96802ade0 tx=0x7fb968003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680317d0 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb968031970 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97019f920 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96802fd30 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.719+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9701a7260 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.721+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb970100cb0 con 0x7fb97010e480 2024-09-12T03:27:23.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.722+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb968004b40 con 0x7fb97010e480 2024-09-12T03:27:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.722+0000 7fb9667fc640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 0x7fb95c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.723+0000 7fb9752b8640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 0x7fb95c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.723+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb9680f07a0 con 0x7fb97010e480 2024-09-12T03:27:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.723+0000 7fb9752b8640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 0x7fb95c0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009990 tx=0x7fb96c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:23.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.726+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb9680bd6b0 con 0x7fb97010e480 2024-09-12T03:27:23.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.937+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb97019ba40 con 0x7fb97010e480 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.938+0000 7fb9667fc640 1 -- 172.21.15.19:0/2720557154 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11992 (secure 0 0 0) 0x7fb97019ba40 con 0x7fb97010e480 2024-09-12T03:27:23.938 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":40,"btime":"2024-09-12T03:27:21:238471+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:27:20.643656+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:27:23.939 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 40 2024-09-12T03:27:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 msgr2=0x7fb95c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 0x7fb95c0a5040 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009990 tx=0x7fb96c009110 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 msgr2=0x7fb97019efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb96802ade0 tx=0x7fb968003f80 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 shutdown_connections 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb95c0a2b80 0x7fb95c0a5040 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb97010e480 0x7fb97019efb0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb970101280 0x7fb97019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 --2- 172.21.15.19:0/2720557154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb9701008d0 0x7fb97019a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 >> 172.21.15.19:0/2720557154 conn(0x7fb9700fc7a0 msgr2=0x7fb9700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 shutdown_connections 2024-09-12T03:27:23.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:23.940+0000 7fb97751c640 1 -- 172.21.15.19:0/2720557154 wait complete. 2024-09-12T03:27:24.256 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:24 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2720557154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:24.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:24 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2720557154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:24 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2720557154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:24.712 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:27:24.978 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:25.412 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:25 smithi019 ceph-mon[30953]: pgmap v588: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T03:27:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 -- 172.21.15.19:0/2853195590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64102570 msgr2=0x7ffb641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/2853195590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64102570 0x7ffb641029b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402a480 comp rx=0 tx=0).stop 2024-09-12T03:27:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 -- 172.21.15.19:0/2853195590 shutdown_connections 2024-09-12T03:27:25.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/2853195590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102ef0 0x7ffb6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/2853195590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64102570 0x7ffb641029b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/2853195590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64100e20 0x7ffb64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 -- 172.21.15.19:0/2853195590 >> 172.21.15.19:0/2853195590 conn(0x7ffb640fc7d0 msgr2=0x7ffb640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 -- 172.21.15.19:0/2853195590 shutdown_connections 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.496+0000 7ffb6b57b640 1 -- 172.21.15.19:0/2853195590 wait complete. 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 Processor -- start 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 -- start start 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 0x7ffb641989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64102ef0 0x7ffb6419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64110f50 con 0x7ffb64100e20 2024-09-12T03:27:25.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb64110dd0 con 0x7ffb64102570 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb6b57b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641110d0 con 0x7ffb64102ef0 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50256/0 (socket says 172.21.15.19:50256) 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 -- 172.21.15.19:0/3770421613 learned_addr learned my addr 172.21.15.19:0/3770421613 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 0x7ffb641989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69b18640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64102ef0 0x7ffb6419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 -- 172.21.15.19:0/3770421613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64102ef0 msgr2=0x7ffb6419cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64102ef0 0x7ffb6419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 -- 172.21.15.19:0/3770421613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 msgr2=0x7ffb641989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 0x7ffb641989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb69317640 1 -- 172.21.15.19:0/3770421613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6419d430 con 0x7ffb64100e20 2024-09-12T03:27:25.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.497+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 0x7ffb641989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb69317640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000cce0 tx=0x7ffb6000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb5a7fc640 1 -- 172.21.15.19:0/3770421613 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000f650 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb5a7fc640 1 -- 172.21.15.19:0/3770421613 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6000fcd0 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6419d720 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb5a7fc640 1 -- 172.21.15.19:0/3770421613 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60018c70 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.498+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb641a5000 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.500+0000 7ffb5a7fc640 1 -- 172.21.15.19:0/3770421613 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb6000f7f0 con 0x7ffb64100e20 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.500+0000 7ffb6d954740 1 Processor -- start 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.500+0000 7ffb6d954740 1 -- start start 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6885a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6896b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617b67eb2f0 con 0x5617b6886b40 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617b6792cf0 con 0x5617b68855d0 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb6d954740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5617b67e72b0 con 0x5617b6894780 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50264/0 (socket says 172.21.15.19:50264) 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 -- 172.21.15.19:0/36759211 learned_addr learned my addr 172.21.15.19:0/36759211 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69317640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6885a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb68b16640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6896b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 -- 172.21.15.19:0/36759211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 msgr2=0x5617b6896b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6896b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 -- 172.21.15.19:0/36759211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 msgr2=0x5617b6885a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6885a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69b18640 1 -- 172.21.15.19:0/36759211 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b67773a0 con 0x5617b6886b40 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb5a7fc640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 0x7ffb500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb69317640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6885a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.501+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 0x7ffb500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb5a7fc640 1 -- 172.21.15.19:0/3770421613 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffb600cf590 con 0x7ffb64100e20 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb69b18640 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009b40 tx=0x7ffb5c00e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c99b2654aefb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb4f7fe640 1 -- 172.21.15.19:0/36759211 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c016070 con 0x5617b6886b40 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb4f7fe640 1 -- 172.21.15.19:0/36759211 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c00ed20 con 0x5617b6886b40 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb4f7fe640 1 -- 172.21.15.19:0/36759211 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c0107f0 con 0x5617b6886b40 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb6d954740 1 -- 172.21.15.19:0/36759211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 msgr2=0x5617b6885090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb6d954740 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c009b40 tx=0x7ffb5c00e2f0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.502+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 0x7ffb500a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffb640fef00 tx=0x7ffb54005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- 172.21.15.19:0/36759211 shutdown_connections 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6896b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6885a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- 172.21.15.19:0/36759211 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6885090 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- 172.21.15.19:0/36759211 >> 172.21.15.19:0/36759211 conn(0x5617b684e310 msgr2=0x5617b6881c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- 172.21.15.19:0/36759211 shutdown_connections 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- 172.21.15.19:0/36759211 wait complete. 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- start start 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 Processor -- start 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6894180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6891a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617b6897480 con 0x5617b6886b40 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617b67a6060 con 0x5617b68855d0 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.503+0000 7ffb6d954740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5617b6897600 con 0x5617b6894780 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb6d954740 10 client.0 _ll_register_callbacks cb 0x5617b64f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50270/0 (socket says 172.21.15.19:50270) 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 -- 172.21.15.19:0/3636255899 learned_addr learned my addr 172.21.15.19:0/3636255899 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb69b18640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6891a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb69317640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6894180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 -- 172.21.15.19:0/3636255899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 msgr2=0x5617b6891a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6891a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 -- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 msgr2=0x5617b6894180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6894180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 -- 172.21.15.19:0/3636255899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b655c3c0 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb69b18640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6891a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb68b16640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40226b1b26c30f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5404c020 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffb54004d90 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.504+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb5c00ed20 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffb54035050 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 10 client.16926 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617b68a2f20 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb5c00ed20 con 0x5617b6886b40 2024-09-12T03:27:25.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7ffb54048620 con 0x5617b6886b40 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 10 client.16926 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 10 client.16926 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 20 client.16926 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b68a44b0 0x5617b68a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 4 client.16926 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b67f8860 con 0x5617b68a44b0 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.505+0000 7ffb69317640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b68a44b0 0x5617b68a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.506+0000 7ffb4cff9640 10 client.16926 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.506+0000 7ffb69317640 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b68a44b0 0x5617b68a68f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cb2b602a6cc2ecad server_cookie=29fb25c3a6eca35f in_seq=0 out_seq=0 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb4cff9640 1 -- 172.21.15.19:0/3636255899 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617b67f8860 con 0x5617b68a44b0 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb4cff9640 10 client.16926 handle_command_reply: tid=0 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 client.16926 shutdown 2024-09-12T03:27:25.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 msgr2=0x5617b6891430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009900 tx=0x7ffb5402a480 comp rx=0 tx=0).stop 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 shutdown_connections 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b68a44b0 0x5617b68a68f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b6894780 0x5617b6891a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6891430 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb4cff9640 0 client.16926 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 --2- 172.21.15.19:0/3636255899 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b68855d0 0x5617b6894180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 >> 172.21.15.19:0/3636255899 conn(0x5617b684e310 msgr2=0x5617b6887960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 shutdown_connections 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 1 -- 172.21.15.19:0/3636255899 wait complete. 2024-09-12T03:27:25.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.507+0000 7ffb6d954740 20 client.16926 trim_cache size 0 max 16384 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 Processor -- start 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 -- start start 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b68a0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b6890410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641ab340 con 0x5617b6886b40 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641acf70 con 0x5617b6894780 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.520+0000 7ffb6d954740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641aba10 con 0x5617b68855d0 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb69b18640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b6890410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb69317640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b68a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50280/0 (socket says 172.21.15.19:50280) 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 -- 172.21.15.19:0/1794924088 learned_addr learned my addr 172.21.15.19:0/1794924088 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 -- 172.21.15.19:0/1794924088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 msgr2=0x5617b68a0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b68a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 -- 172.21.15.19:0/1794924088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 msgr2=0x5617b6890410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b6890410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.521+0000 7ffb68b16640 1 -- 172.21.15.19:0/1794924088 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b6881510 con 0x5617b6886b40 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.522+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb54048b10 tx=0x7ffb54004b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d357515ec65b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.522+0000 7ffb4effd640 1 -- 172.21.15.19:0/1794924088 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb54061070 con 0x5617b6886b40 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.522+0000 7ffb4effd640 1 -- 172.21.15.19:0/1794924088 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb54032550 con 0x5617b6886b40 2024-09-12T03:27:25.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.522+0000 7ffb4effd640 1 -- 172.21.15.19:0/1794924088 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5404c040 con 0x5617b6886b40 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- 172.21.15.19:0/1794924088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 msgr2=0x5617b6890cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ffb54048b10 tx=0x7ffb54004b10 comp rx=0 tx=0).stop 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- 172.21.15.19:0/1794924088 shutdown_connections 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b6890410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b6890cf0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1794924088 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b68a0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- 172.21.15.19:0/1794924088 >> 172.21.15.19:0/1794924088 conn(0x5617b684e310 msgr2=0x5617b688fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- 172.21.15.19:0/1794924088 shutdown_connections 2024-09-12T03:27:25.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- 172.21.15.19:0/1794924088 wait complete. 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 -- start start 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.523+0000 7ffb6d954740 1 Processor -- start 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b689ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b688e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641ac7e0 con 0x5617b6886b40 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641ac110 con 0x5617b6894780 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ffb641a92c0 con 0x5617b68855d0 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb6d954740 10 client.0 _ll_register_callbacks cb 0x7ffb5403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50294/0 (socket says 172.21.15.19:50294) 2024-09-12T03:27:25.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 -- 172.21.15.19:0/1110694935 learned_addr learned my addr 172.21.15.19:0/1110694935 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb69317640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b689ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb69b18640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b688e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 -- 172.21.15.19:0/1110694935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 msgr2=0x5617b689ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b689ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 -- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 msgr2=0x5617b688e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b688e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb68b16640 1 -- 172.21.15.19:0/1110694935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b688e8d0 con 0x5617b6886b40 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.524+0000 7ffb69b18640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b688e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb68b16640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb540487f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e45b69d6cc140a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb5404c030 con 0x5617b6886b40 2024-09-12T03:27:25.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb54032550 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ffb54032bd0 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ffb54061040 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 10 client.16932 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5617b690eed0 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb54032550 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7ffb5405d550 con 0x5617b6886b40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 10 client.16932 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 10 client.16932 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 20 client.16932 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b690f560 0x5617b6911980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 4 client.16932 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:25.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.525+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617b67f2d70 con 0x5617b690f560 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb69317640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b690f560 0x5617b6911980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb4cff9640 10 client.16932 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb69317640 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b690f560 0x5617b6911980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8964f66de14bde46 server_cookie=227323a3e0859e65 in_seq=0 out_seq=0 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb4cff9640 1 -- 172.21.15.19:0/1110694935 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617b67f2d70 con 0x5617b690f560 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb4cff9640 10 client.16932 handle_command_reply: tid=0 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.526+0000 7ffb6d954740 1 client.16932 shutdown 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 msgr2=0x5617b688de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb540487f0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 shutdown_connections 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617b6894780 0x5617b688e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617b6886b40 0x5617b688de20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5617b690f560 0x5617b6911980 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 --2- 172.21.15.19:0/1110694935 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617b68855d0 0x5617b689ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb4cff9640 0 client.16932 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 >> 172.21.15.19:0/1110694935 conn(0x5617b684e310 msgr2=0x5617b68934b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 shutdown_connections 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 1 -- 172.21.15.19:0/1110694935 wait complete. 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.527+0000 7ffb6d954740 20 client.16932 trim_cache size 0 max 16384 2024-09-12T03:27:25.543 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 msgr2=0x7ffb500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 0x7ffb500a5040 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ffb640fef00 tx=0x7ffb54005a40 comp rx=0 tx=0).stop 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 msgr2=0x7ffb64198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000cce0 tx=0x7ffb6000a620 comp rx=0 tx=0).stop 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 shutdown_connections 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ffb64102ef0 0x7ffb6419cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb64102570 0x7ffb641989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ffb500a2b80 0x7ffb500a5040 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 --2- 172.21.15.19:0/3770421613 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ffb64100e20 0x7ffb64198480 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:25.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 >> 172.21.15.19:0/3770421613 conn(0x7ffb640fc7d0 msgr2=0x7ffb6406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:25.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 shutdown_connections 2024-09-12T03:27:25.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:25.528+0000 7ffb6b57b640 1 -- 172.21.15.19:0/3770421613 wait complete. 2024-09-12T03:27:25.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:25 smithi135 ceph-mon[35474]: pgmap v588: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T03:27:25.551 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:25 smithi121 ceph-mon[34963]: pgmap v588: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T03:27:26.346 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:27:26.603 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 -- 172.21.15.19:0/832002347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730106f10 msgr2=0x7f973010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 --2- 172.21.15.19:0/832002347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730106f10 0x7f973010ea90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9728009900 tx=0x7f972802a410 comp rx=0 tx=0).stop 2024-09-12T03:27:27.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 -- 172.21.15.19:0/832002347 shutdown_connections 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 --2- 172.21.15.19:0/832002347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730106f10 0x7f973010ea90 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 --2- 172.21.15.19:0/832002347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 0x7f97301067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 --2- 172.21.15.19:0/832002347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730101de0 0x7f97301021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 -- 172.21.15.19:0/832002347 >> 172.21.15.19:0/832002347 conn(0x7f97300fd8c0 msgr2=0x7f97300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 -- 172.21.15.19:0/832002347 shutdown_connections 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.107+0000 7f97348a5640 1 -- 172.21.15.19:0/832002347 wait complete. 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 Processor -- start 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 -- start start 2024-09-12T03:27:27.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 0x7f9730198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 0x7f973019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9730111f70 con 0x7f9730101de0 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9730111df0 con 0x7f9730102790 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f97348a5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97301120f0 con 0x7f9730106f10 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50314/0 (socket says 172.21.15.19:50314) 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 -- 172.21.15.19:0/2063191039 learned_addr learned my addr 172.21.15.19:0/2063191039 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972ed9d640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 0x7f973019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972dd9b640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 0x7f9730198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 -- 172.21.15.19:0/2063191039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 msgr2=0x7f973019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 0x7f973019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 -- 172.21.15.19:0/2063191039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 msgr2=0x7f9730198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 0x7f9730198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972e59c640 1 -- 172.21.15.19:0/2063191039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f973019d380 con 0x7f9730101de0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.108+0000 7f972ed9d640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 0x7f973019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f972e59c640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f972000eb90 tx=0x7f972000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f971f7fe640 1 -- 172.21.15.19:0/2063191039 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720004d20 con 0x7f9730101de0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f971f7fe640 1 -- 172.21.15.19:0/2063191039 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972000ee90 con 0x7f9730101de0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f973019d670 con 0x7f9730101de0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f971f7fe640 1 -- 172.21.15.19:0/2063191039 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9720010830 con 0x7f9730101de0 2024-09-12T03:27:27.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.109+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97301a4f50 con 0x7f9730101de0 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.111+0000 7f971f7fe640 1 -- 172.21.15.19:0/2063191039 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9720005020 con 0x7f9730101de0 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.111+0000 7f9736c7e740 1 Processor -- start 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 -- start start 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8054090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f971f7fe640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 0x7f97180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb8054a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563cb7fba2f0 con 0x563cb805f370 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563cb7f61cf0 con 0x563cb8055b40 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f9736c7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cb7fb62b0 con 0x563cb80545d0 2024-09-12T03:27:27.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f971f7fe640 1 -- 172.21.15.19:0/2063191039 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97200cf7e0 con 0x7f9730101de0 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972e59c640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 0x7f97180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50326/0 (socket says 172.21.15.19:50326) 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972ed9d640 1 -- 172.21.15.19:0/399826011 learned_addr learned my addr 172.21.15.19:0/399826011 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972dd9b640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb8054a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.112+0000 7f972ed9d640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8054090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 -- 172.21.15.19:0/399826011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 msgr2=0x563cb8054a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb8054a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 -- 172.21.15.19:0/399826011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 msgr2=0x563cb8054090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8054090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 -- 172.21.15.19:0/399826011 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb7f463a0 con 0x563cb805f370 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972dd9b640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb8054a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972e59c640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 0x7f97180a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9720098a20 tx=0x7f9720005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f972ed9d640 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9728005ec0 tx=0x7f97280056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce505ef7c3d77e28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f971cff9640 1 -- 172.21.15.19:0/399826011 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728035040 con 0x563cb805f370 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f971cff9640 1 -- 172.21.15.19:0/399826011 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97280035f0 con 0x563cb805f370 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.113+0000 7f971cff9640 1 -- 172.21.15.19:0/399826011 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972803d680 con 0x563cb805f370 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- 172.21.15.19:0/399826011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 msgr2=0x563cb8061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9728005ec0 tx=0x7f97280056d0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- 172.21.15.19:0/399826011 shutdown_connections 2024-09-12T03:27:27.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8061760 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb8054a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- 172.21.15.19:0/399826011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8054090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- 172.21.15.19:0/399826011 >> 172.21.15.19:0/399826011 conn(0x563cb801d310 msgr2=0x563cb8050c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- 172.21.15.19:0/399826011 shutdown_connections 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- 172.21.15.19:0/399826011 wait complete. 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- start start 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 Processor -- start 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb80545d0 0x563cb805ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb8055b40 0x563cb805c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563cb8066420 con 0x563cb805f370 2024-09-12T03:27:27.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563cb7f75060 con 0x563cb80545d0 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.114+0000 7f9736c7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cb80665a0 con 0x563cb8055b40 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9736c7e740 10 client.0 _ll_register_callbacks cb 0x563cb7cc88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50342/0 (socket says 172.21.15.19:50342) 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 -- 172.21.15.19:0/231938064 learned_addr learned my addr 172.21.15.19:0/231938064 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb80545d0 0x563cb805ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972dd9b640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb8055b40 0x563cb805c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 -- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb8055b40 msgr2=0x563cb805c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb8055b40 0x563cb805c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 -- 172.21.15.19:0/231938064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb80545d0 msgr2=0x563cb805ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb80545d0 0x563cb805ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 -- 172.21.15.19:0/231938064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb7d2b3c0 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f972ed9d640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f972802a930 tx=0x7f9728032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c19a2a63da9b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728052020 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9728003fb0 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97280035f0 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f9728035040 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9736c7e740 10 client.16944 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.115+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563cb8071ec0 con 0x563cb805f370 2024-09-12T03:27:27.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97280035f0 con 0x563cb805f370 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7f972803a040 con 0x563cb805f370 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 10 client.16944 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 10 client.16944 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 20 client.16944 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb8073210 0x563cb8075600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 4 client.16944 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cb7fc7860 con 0x563cb8073210 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.116+0000 7f972dd9b640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb8073210 0x563cb8075600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.117+0000 7f972dd9b640 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb8073210 0x563cb8075600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9eebc41b15833102 server_cookie=386edd1f0b338525 in_seq=0 out_seq=0 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.117+0000 7f9715ffb640 10 client.16944 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.117+0000 7f9715ffb640 1 -- 172.21.15.19:0/231938064 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cb7fc7860 con 0x563cb8073210 2024-09-12T03:27:27.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.117+0000 7f9715ffb640 10 client.16944 handle_command_reply: tid=0 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.117+0000 7f9736c7e740 1 client.16944 shutdown 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 msgr2=0x563cb805c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f972802a930 tx=0x7f9728032ef0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 shutdown_connections 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb8073210 0x563cb8075600 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb805c610 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb8055b40 0x563cb805c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 --2- 172.21.15.19:0/231938064 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb80545d0 0x563cb805ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 >> 172.21.15.19:0/231938064 conn(0x563cb801d310 msgr2=0x563cb8056960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9715ffb640 0 client.16944 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 shutdown_connections 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 1 -- 172.21.15.19:0/231938064 wait complete. 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.118+0000 7f9736c7e740 20 client.16944 trim_cache size 0 max 16384 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 Processor -- start 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 -- start start 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb805df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb80570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97301ab3d0 con 0x563cb805f370 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97301ad000 con 0x563cb8055b40 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f9736c7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97301abaa0 con 0x563cb80545d0 2024-09-12T03:27:27.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50356/0 (socket says 172.21.15.19:50356) 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f972ed9d640 1 -- 172.21.15.19:0/3913589062 learned_addr learned my addr 172.21.15.19:0/3913589062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f972dd9b640 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb80570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.132+0000 7f972ed9d640 1 -- 172.21.15.19:0/3913589062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 msgr2=0x563cb805df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb805df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972ed9d640 1 -- 172.21.15.19:0/3913589062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 msgr2=0x563cb80570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb80570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972ed9d640 1 -- 172.21.15.19:0/3913589062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb805e990 con 0x563cb805f370 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972dd9b640 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb80570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9728009850 tx=0x7f97280056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fadef0f099d1faa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f9717fff640 1 -- 172.21.15.19:0/3913589062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728052070 con 0x563cb805f370 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f9717fff640 1 -- 172.21.15.19:0/3913589062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9728050070 con 0x563cb805f370 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f9717fff640 1 -- 172.21.15.19:0/3913589062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728004010 con 0x563cb805f370 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f9736c7e740 1 -- 172.21.15.19:0/3913589062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 msgr2=0x563cb8057630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.133+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9728009850 tx=0x7f97280056d0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- 172.21.15.19:0/3913589062 shutdown_connections 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb805f370 0x563cb8057630 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb80570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3913589062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb80545d0 0x563cb805df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- 172.21.15.19:0/3913589062 >> 172.21.15.19:0/3913589062 conn(0x563cb801d310 msgr2=0x563cb805d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- 172.21.15.19:0/3913589062 shutdown_connections 2024-09-12T03:27:27.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- 172.21.15.19:0/3913589062 wait complete. 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- start start 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 Processor -- start 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8075ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb805f370 0x563cb80761e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97301ac870 con 0x563cb80545d0 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97301ac1a0 con 0x563cb8055b40 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97301a9350 con 0x563cb805f370 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f9736c7e740 10 client.0 _ll_register_callbacks cb 0x7f9728009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f972ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50370/0 (socket says 172.21.15.19:50370) 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f972ed9d640 1 -- 172.21.15.19:0/3335027843 learned_addr learned my addr 172.21.15.19:0/3335027843 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb805f370 0x563cb80761e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.134+0000 7f972dd9b640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8075ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 -- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb805f370 msgr2=0x563cb80761e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb805f370 0x563cb80761e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 -- 172.21.15.19:0/3335027843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 msgr2=0x563cb8075ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8075ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 -- 172.21.15.19:0/3335027843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb80769c0 con 0x563cb80545d0 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f972ed9d640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9728031bf0 tx=0x7f972804b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32473be27c3cedd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:27.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728052020 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f972803ddd0 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9728050070 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f972804b930 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 10 client.16950 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563cb80dda70 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9728050070 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7f9728062550 con 0x563cb80545d0 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 10 client.16950 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 10 client.16950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:27:27.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 20 client.16950 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.135+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb80ddee0 0x563cb80e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9736c7e740 4 client.16950 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cb7fc1d70 con 0x563cb80ddee0 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f972dd9b640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb80ddee0 0x563cb80e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f972dd9b640 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb80ddee0 0x563cb80e02d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5785bb03bb5c7b88 server_cookie=8f0f10a88fe3687e in_seq=0 out_seq=0 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9715ffb640 10 client.16950 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9715ffb640 1 -- 172.21.15.19:0/3335027843 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cb7fc1d70 con 0x563cb80ddee0 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9715ffb640 10 client.16950 handle_command_reply: tid=0 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.136+0000 7f9736c7e740 1 client.16950 shutdown 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 msgr2=0x563cb8078950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9728031bf0 tx=0x7f972804b550 comp rx=0 tx=0).stop 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 shutdown_connections 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x563cb80ddee0 0x563cb80e02d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cb805f370 0x563cb80761e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cb8055b40 0x563cb8075ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9715ffb640 0 client.16950 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 --2- 172.21.15.19:0/3335027843 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cb80545d0 0x563cb8078950 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 >> 172.21.15.19:0/3335027843 conn(0x563cb801d310 msgr2=0x563cb804fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 shutdown_connections 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 1 -- 172.21.15.19:0/3335027843 wait complete. 2024-09-12T03:27:27.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f9736c7e740 20 client.16950 trim_cache size 0 max 16384 2024-09-12T03:27:27.169 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:27:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 msgr2=0x7f97180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 0x7f97180a5040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9720098a20 tx=0x7f9720005960 comp rx=0 tx=0).stop 2024-09-12T03:27:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 msgr2=0x7f97301983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:27.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f972000eb90 tx=0x7f972000c6a0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 shutdown_connections 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9730106f10 0x7f973019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9730102790 0x7f9730198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97180a2b80 0x7f97180a5040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 --2- 172.21.15.19:0/2063191039 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9730101de0 0x7f97301983d0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 >> 172.21.15.19:0/2063191039 conn(0x7f97300fd8c0 msgr2=0x7f97300feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 shutdown_connections 2024-09-12T03:27:27.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:27.137+0000 7f97348a5640 1 -- 172.21.15.19:0/2063191039 wait complete. 2024-09-12T03:27:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:27 smithi019 ceph-mon[30953]: pgmap v589: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-12T03:27:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:27 smithi135 ceph-mon[35474]: pgmap v589: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-12T03:27:27.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:27 smithi121 ceph-mon[34963]: pgmap v589: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-09-12T03:27:27.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:27:28.142 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:28.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- 172.21.15.19:0/360193036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801008d0 msgr2=0x7fd080100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 --2- 172.21.15.19:0/360193036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801008d0 0x7fd080100d30 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd070009930 tx=0x7fd07002a420 comp rx=0 tx=0).stop 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- 172.21.15.19:0/360193036 shutdown_connections 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 --2- 172.21.15.19:0/360193036 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd080101270 0x7fd08010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 --2- 172.21.15.19:0/360193036 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801008d0 0x7fd080100d30 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 --2- 172.21.15.19:0/360193036 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd0801068d0 0x7fd080106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- 172.21.15.19:0/360193036 >> 172.21.15.19:0/360193036 conn(0x7fd0800fc630 msgr2=0x7fd0800fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- 172.21.15.19:0/360193036 shutdown_connections 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- 172.21.15.19:0/360193036 wait complete. 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 Processor -- start 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.601+0000 7fd08739f640 1 -- start start 2024-09-12T03:27:28.617 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 0x7fd0801982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 0x7fd080198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd080110cd0 con 0x7fd0801068d0 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd080110b50 con 0x7fd0801008d0 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08739f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd080110e50 con 0x7fd080101270 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50380/0 (socket says 172.21.15.19:50380) 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 -- 172.21.15.19:0/517888432 learned_addr learned my addr 172.21.15.19:0/517888432 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08639d640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 0x7fd0801982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd085b9c640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 0x7fd080198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 -- 172.21.15.19:0/517888432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 msgr2=0x7fd080198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 0x7fd080198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.618 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 -- 172.21.15.19:0/517888432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 msgr2=0x7fd0801982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 0x7fd0801982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 -- 172.21.15.19:0/517888432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08019d2a0 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd08639d640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 0x7fd0801982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd085b9c640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 0x7fd080198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd086b9e640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd078009fd0 tx=0x7fd07800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.602+0000 7fd0777fe640 1 -- 172.21.15.19:0/517888432 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078017070 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.603+0000 7fd0777fe640 1 -- 172.21.15.19:0/517888432 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd078003e50 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.603+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08019d590 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.603+0000 7fd0777fe640 1 -- 172.21.15.19:0/517888432 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780106c0 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.603+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0801a4e70 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.605+0000 7fd0777fe640 1 -- 172.21.15.19:0/517888432 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd078003970 con 0x7fd0801068d0 2024-09-12T03:27:28.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.605+0000 7fd08a9da740 1 Processor -- start 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.605+0000 7fd08a9da740 1 -- start start 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b685bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b696d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d2b5eb2f0 con 0x556d2b686d20 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d2b592cf0 con 0x556d2b6857b0 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08a9da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556d2b5e72b0 con 0x556d2b694960 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50396/0 (socket says 172.21.15.19:50396) 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd085b9c640 1 -- 172.21.15.19:0/2410788840 learned_addr learned my addr 172.21.15.19:0/2410788840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b685bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd0777fe640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 0x7fd06c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd085b9c640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b696d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd086b9e640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 0x7fd06c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 -- 172.21.15.19:0/2410788840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 msgr2=0x556d2b696d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b696d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 -- 172.21.15.19:0/2410788840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 msgr2=0x556d2b685270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 -- 172.21.15.19:0/2410788840 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2b5773a0 con 0x556d2b6857b0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd085b9c640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd0777fe640 1 -- 172.21.15.19:0/517888432 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd0780cee10 con 0x7fd0801068d0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.606+0000 7fd08639d640 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b685bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009bb0 tx=0x7fd07c0117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcaa9ba3d70e42a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd086b9e640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 0x7fd06c0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ccc90 tx=0x7fd07806e740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd074ff9640 1 -- 172.21.15.19:0/2410788840 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c012450 con 0x556d2b6857b0 2024-09-12T03:27:28.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd074ff9640 1 -- 172.21.15.19:0/2410788840 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c012ad0 con 0x556d2b6857b0 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd074ff9640 1 -- 172.21.15.19:0/2410788840 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00fac0 con 0x556d2b6857b0 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 -- 172.21.15.19:0/2410788840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 msgr2=0x556d2b685bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b685bf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009bb0 tx=0x7fd07c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 -- 172.21.15.19:0/2410788840 shutdown_connections 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b696d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b685bf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 --2- 172.21.15.19:0/2410788840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b685270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.607+0000 7fd08a9da740 1 -- 172.21.15.19:0/2410788840 >> 172.21.15.19:0/2410788840 conn(0x556d2b64e310 msgr2=0x556d2b681e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- 172.21.15.19:0/2410788840 shutdown_connections 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- 172.21.15.19:0/2410788840 wait complete. 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- start start 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 Processor -- start 2024-09-12T03:27:28.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b694280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b691a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556d2b697660 con 0x556d2b686d20 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556d2b5a6060 con 0x556d2b6857b0 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x556d2b6977e0 con 0x556d2b694960 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08a9da740 10 client.0 _ll_register_callbacks cb 0x556d2b2f98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08639d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08639d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50400/0 (socket says 172.21.15.19:50400) 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd08639d640 1 -- 172.21.15.19:0/1603337575 learned_addr learned my addr 172.21.15.19:0/1603337575 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd085b9c640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b694280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.608+0000 7fd085b9c640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b691a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 -- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 msgr2=0x556d2b691a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b691a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 -- 172.21.15.19:0/1603337575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 msgr2=0x556d2b694280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b694280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 -- 172.21.15.19:0/1603337575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b35c3c0 con 0x556d2b686d20 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08639d640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00f6c0 tx=0x7fd07c009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99faf34b6d0c5fd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c029020 con 0x556d2b686d20 2024-09-12T03:27:28.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd07c012660 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd07c012ad0 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd07c012880 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08a9da740 10 client.16962 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556d2b6a3050 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.609+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd07c012ad0 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7fd07c0046c0 con 0x556d2b686d20 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 10 client.16962 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:28.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 10 client.16962 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 20 client.16962 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b6a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 4 client.16962 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d2b5f8860 con 0x556d2b6a4400 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.610+0000 7fd085b9c640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b6a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd085b9c640 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b6a67f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be869e25f5d9f99b server_cookie=88a410bcf3ffe556 in_seq=0 out_seq=0 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd069ffb640 10 client.16962 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd069ffb640 1 -- 172.21.15.19:0/1603337575 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d2b5f8860 con 0x556d2b6a4400 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd069ffb640 10 client.16962 handle_command_reply: tid=0 2024-09-12T03:27:28.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd08a9da740 1 client.16962 shutdown 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 msgr2=0x556d2b691530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00f6c0 tx=0x7fd07c009850 comp rx=0 tx=0).stop 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 shutdown_connections 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.611+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b6a67f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b691a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd069ffb640 0 client.16962 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:27:28.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b686d20 0x556d2b691530 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 --2- 172.21.15.19:0/1603337575 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b6857b0 0x556d2b694280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 >> 172.21.15.19:0/1603337575 conn(0x556d2b64e310 msgr2=0x556d2b687b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 shutdown_connections 2024-09-12T03:27:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 1 -- 172.21.15.19:0/1603337575 wait complete. 2024-09-12T03:27:28.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.612+0000 7fd08a9da740 20 client.16962 trim_cache size 0 max 16384 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 Processor -- start 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 -- start start 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b686d20 0x556d2b6907b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b68c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ab2f0 con 0x556d2b6857b0 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801acf20 con 0x556d2b686d20 2024-09-12T03:27:28.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08a9da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ab9c0 con 0x556d2b694960 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50410/0 (socket says 172.21.15.19:50410) 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 -- 172.21.15.19:0/3079636742 learned_addr learned my addr 172.21.15.19:0/3079636742 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 -- 172.21.15.19:0/3079636742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 msgr2=0x556d2b68c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b68c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd08639d640 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b686d20 0x556d2b6907b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 -- 172.21.15.19:0/3079636742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b686d20 msgr2=0x556d2b6907b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b686d20 0x556d2b6907b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 -- 172.21.15.19:0/3079636742 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d2b697800 con 0x556d2b6857b0 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.625+0000 7fd085b9c640 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd07002a940 tx=0x7fd070041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728efb5c47f6bbfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd06bfff640 1 -- 172.21.15.19:0/3079636742 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070035040 con 0x556d2b6857b0 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd06bfff640 1 -- 172.21.15.19:0/3079636742 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070005530 con 0x556d2b6857b0 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd06bfff640 1 -- 172.21.15.19:0/3079636742 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0700035f0 con 0x556d2b6857b0 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 -- 172.21.15.19:0/3079636742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 msgr2=0x556d2b6a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fd07002a940 tx=0x7fd070041040 comp rx=0 tx=0).stop 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 -- 172.21.15.19:0/3079636742 shutdown_connections 2024-09-12T03:27:28.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b694960 0x556d2b68c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b686d20 0x556d2b6907b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 --2- 172.21.15.19:0/3079636742 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b6a0930 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 -- 172.21.15.19:0/3079636742 >> 172.21.15.19:0/3079636742 conn(0x556d2b64e310 msgr2=0x556d2b692d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 -- 172.21.15.19:0/3079636742 shutdown_connections 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.626+0000 7fd08a9da740 1 -- 172.21.15.19:0/3079636742 wait complete. 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 -- start start 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 Processor -- start 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b686d20 0x556d2b69caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b694960 0x556d2b69cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ac790 con 0x556d2b6857b0 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801ac0c0 con 0x556d2b694960 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd0801a9270 con 0x556d2b686d20 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08a9da740 10 client.0 _ll_register_callbacks cb 0x7fd07c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd085b9c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50422/0 (socket says 172.21.15.19:50422) 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd085b9c640 1 -- 172.21.15.19:0/844680833 learned_addr learned my addr 172.21.15.19:0/844680833 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.627+0000 7fd08639d640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b686d20 0x556d2b69caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 -- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b686d20 msgr2=0x556d2b69caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b686d20 0x556d2b69caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 -- 172.21.15.19:0/844680833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b694960 msgr2=0x556d2b69cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b694960 0x556d2b69cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 -- 172.21.15.19:0/844680833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d2b69d6d0 con 0x556d2b6857b0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd085b9c640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd0700096c0 tx=0x7fd070031b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8473cce10b52af72 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd069ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070035070 con 0x556d2b6857b0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd070004950 con 0x556d2b6857b0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd070005530 con 0x556d2b6857b0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd070004bc0 con 0x556d2b6857b0 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd08a9da740 10 client.16968 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:28.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x556d2b70eeb0 con 0x556d2b6857b0 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.628+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd070005530 con 0x556d2b6857b0 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7fd0700619e0 con 0x556d2b6857b0 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 10 client.16968 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 10 client.16968 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 20 client.16968 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b711620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 4 client.16968 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d2b5f2d70 con 0x556d2b6a4400 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08639d640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b711620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd069ffb640 10 client.16968 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08639d640 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b711620 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5ef55652fe4b9b7 server_cookie=df2e28caa23b2119 in_seq=0 out_seq=0 2024-09-12T03:27:28.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd069ffb640 1 -- 172.21.15.19:0/844680833 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d2b5f2d70 con 0x556d2b6a4400 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd069ffb640 10 client.16968 handle_command_reply: tid=0 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.629+0000 7fd08a9da740 1 client.16968 shutdown 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 msgr2=0x556d2b68ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fd0700096c0 tx=0x7fd070031b90 comp rx=0 tx=0).stop 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 shutdown_connections 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x556d2b6a4400 0x556d2b711620 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556d2b694960 0x556d2b69cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556d2b686d20 0x556d2b69caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd069ffb640 0 client.16968 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 --2- 172.21.15.19:0/844680833 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556d2b6857b0 0x556d2b68ef00 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 >> 172.21.15.19:0/844680833 conn(0x556d2b64e310 msgr2=0x556d2b6935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 shutdown_connections 2024-09-12T03:27:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 1 -- 172.21.15.19:0/844680833 wait complete. 2024-09-12T03:27:28.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08a9da740 20 client.16968 trim_cache size 0 max 16384 2024-09-12T03:27:28.643 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 msgr2=0x7fd06c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 0x7fd06c0a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd0780ccc90 tx=0x7fd07806e740 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 msgr2=0x7fd08019cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd078009fd0 tx=0x7fd07800b260 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 shutdown_connections 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd06c0a2c00 0x7fd06c0a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd0801068d0 0x7fd08019cbc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd080101270 0x7fd080198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 --2- 172.21.15.19:0/517888432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd0801008d0 0x7fd0801982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.630+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 >> 172.21.15.19:0/517888432 conn(0x7fd0800fc630 msgr2=0x7fd08006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:28.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.631+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 shutdown_connections 2024-09-12T03:27:28.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:28.631+0000 7fd08739f640 1 -- 172.21.15.19:0/517888432 wait complete. 2024-09-12T03:27:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:29 smithi019 ceph-mon[30953]: pgmap v590: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:27:29.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:27:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:29 smithi135 ceph-mon[35474]: pgmap v590: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:27:29.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:29 smithi121 ceph-mon[34963]: pgmap v590: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 2.7 KiB/s wr, 12 op/s 2024-09-12T03:27:29.793 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:30.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 -- 172.21.15.19:0/358082873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088102d30 msgr2=0x7f2088103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 --2- 172.21.15.19:0/358082873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088102d30 0x7f2088103110 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2080009930 tx=0x7f208002a420 comp rx=0 tx=0).stop 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 -- 172.21.15.19:0/358082873 shutdown_connections 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 --2- 172.21.15.19:0/358082873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20881082a0 0x7f208810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 --2- 172.21.15.19:0/358082873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 0x7f2088107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 --2- 172.21.15.19:0/358082873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088102d30 0x7f2088103110 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 -- 172.21.15.19:0/358082873 >> 172.21.15.19:0/358082873 conn(0x7f20880fc7d0 msgr2=0x7f20880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 -- 172.21.15.19:0/358082873 shutdown_connections 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.285+0000 7f208ea99640 1 -- 172.21.15.19:0/358082873 wait complete. 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 Processor -- start 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 -- start start 2024-09-12T03:27:30.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 0x7f20881a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 0x7f20881a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20881120a0 con 0x7f20881082a0 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2088111f20 con 0x7f2088102d30 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208ea99640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2088112220 con 0x7f20881036e0 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208c835640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 0x7f20881a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50434/0 (socket says 172.21.15.19:50434) 2024-09-12T03:27:30.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208d036640 1 -- 172.21.15.19:0/1772189976 learned_addr learned my addr 172.21.15.19:0/1772189976 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f208c835640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 0x7f20881a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59694/0 (socket says 172.21.15.19:59694) 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.286+0000 7f2087fff640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 0x7f20881a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 -- 172.21.15.19:0/1772189976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 msgr2=0x7f20881a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 0x7f20881a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 -- 172.21.15.19:0/1772189976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 msgr2=0x7f20881a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 0x7f20881a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 -- 172.21.15.19:0/1772189976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f208819ad20 con 0x7f20881082a0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208d036640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f207000c970 tx=0x7f207000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f2085ffb640 1 -- 172.21.15.19:0/1772189976 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070007bb0 con 0x7f20881082a0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f2085ffb640 1 -- 172.21.15.19:0/1772189976 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2070007d50 con 0x7f20881082a0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f208819b010 con 0x7f20881082a0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f2085ffb640 1 -- 172.21.15.19:0/1772189976 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070018800 con 0x7f20881082a0 2024-09-12T03:27:30.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.287+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20881a7300 con 0x7f20881082a0 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2085ffb640 1 -- 172.21.15.19:0/1772189976 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f207000fa50 con 0x7f20881082a0 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 Processor -- start 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 -- start start 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cdb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cdba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2085ffb640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 0x7f20740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047ce8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559047c412f0 con 0x559047cdcb40 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559047be8cf0 con 0x559047ce6370 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.290+0000 7f2090e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559047c3d2b0 con 0x559047cdb5d0 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cdb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2085ffb640 1 -- 172.21.15.19:0/1772189976 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f20700ceee0 con 0x7f20881082a0 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cdb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50440/0 (socket says 172.21.15.19:50440) 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2087fff640 1 -- 172.21.15.19:0/312413447 learned_addr learned my addr 172.21.15.19:0/312413447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047ce8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208c835640 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cdba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2087fff640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 0x7f20740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 -- 172.21.15.19:0/312413447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 msgr2=0x559047cdba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cdba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 -- 172.21.15.19:0/312413447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 msgr2=0x559047cdb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cdb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 -- 172.21.15.19:0/312413447 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559047bcd3a0 con 0x559047ce6370 2024-09-12T03:27:30.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f208d036640 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047ce8760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f207000ffd0 tx=0x7f207009b290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bef24977311dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f2087fff640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 0x7f20740a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2078005fd0 tx=0x7f2078009b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f207effd640 1 -- 172.21.15.19:0/312413447 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700b4950 con 0x559047ce6370 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f207effd640 1 -- 172.21.15.19:0/312413447 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207009be50 con 0x559047ce6370 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.291+0000 7f207effd640 1 -- 172.21.15.19:0/312413447 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070097dc0 con 0x559047ce6370 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- 172.21.15.19:0/312413447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 msgr2=0x559047ce8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047ce8760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f207000ffd0 tx=0x7f207009b290 comp rx=0 tx=0).stop 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- 172.21.15.19:0/312413447 shutdown_connections 2024-09-12T03:27:30.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047ce8760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cdba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 --2- 172.21.15.19:0/312413447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cdb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- 172.21.15.19:0/312413447 >> 172.21.15.19:0/312413447 conn(0x559047ca4310 msgr2=0x559047cd7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- 172.21.15.19:0/312413447 shutdown_connections 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- 172.21.15.19:0/312413447 wait complete. 2024-09-12T03:27:30.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.292+0000 7f2090e72740 1 -- start start 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 Processor -- start 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047ce5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047ce2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559047ce9070 con 0x559047ce6370 2024-09-12T03:27:30.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559047bfc060 con 0x559047cdb5d0 2024-09-12T03:27:30.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559047ce91f0 con 0x559047cdcb40 2024-09-12T03:27:30.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2090e72740 10 client.0 _ll_register_callbacks cb 0x55904794f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:30.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50446/0 (socket says 172.21.15.19:50446) 2024-09-12T03:27:30.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 -- 172.21.15.19:0/3500101621 learned_addr learned my addr 172.21.15.19:0/3500101621 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f2087fff640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047ce2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208c835640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047ce5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 -- 172.21.15.19:0/3500101621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 msgr2=0x559047ce2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047ce2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 -- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 msgr2=0x559047ce5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047ce5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.293+0000 7f208d036640 1 -- 172.21.15.19:0/3500101621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590479b23c0 con 0x559047ce6370 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f208d036640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2070098120 tx=0x7f2070099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc9021f4e327370 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700b9060 con 0x559047ce6370 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f207009aba0 con 0x559047ce6370 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207009be50 con 0x559047ce6370 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f20700b4950 con 0x559047ce6370 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 10 client.16980 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559047cf4ac0 con 0x559047ce6370 2024-09-12T03:27:30.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207009be50 con 0x559047ce6370 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7f2070084ce0 con 0x559047ce6370 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 10 client.16980 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 10 client.16980 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 20 client.16980 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.294+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047cf5e10 0x559047cf8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.295+0000 7f2090e72740 4 client.16980 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.295+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559047c4e860 con 0x559047cf5e10 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.295+0000 7f208c835640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047cf5e10 0x559047cf8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.295+0000 7f2067fff640 10 client.16980 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.295+0000 7f208c835640 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047cf5e10 0x559047cf8200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa1a3b1697d453ad server_cookie=feeb6bba28316b08 in_seq=0 out_seq=0 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2067fff640 1 -- 172.21.15.19:0/3500101621 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559047c4e860 con 0x559047cf5e10 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2067fff640 10 client.16980 handle_command_reply: tid=0 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 client.16980 shutdown 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 msgr2=0x559047ce3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f2070098120 tx=0x7f2070099960 comp rx=0 tx=0).stop 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 shutdown_connections 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047cf5e10 0x559047cf8200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047ce3570 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047ce2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 --2- 172.21.15.19:0/3500101621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047ce5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 >> 172.21.15.19:0/3500101621 conn(0x559047ca4310 msgr2=0x559047cdd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2067fff640 0 client.16980 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.296+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 shutdown_connections 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.297+0000 7f2090e72740 1 -- 172.21.15.19:0/3500101621 wait complete. 2024-09-12T03:27:30.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.297+0000 7f2090e72740 20 client.16980 trim_cache size 0 max 16384 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 Processor -- start 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 -- start start 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047cde120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047cde660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20881af640 con 0x559047ce6370 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20881b1270 con 0x559047cdb5d0 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2090e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20881afd10 con 0x559047cdcb40 2024-09-12T03:27:30.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f208d036640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50458/0 (socket says 172.21.15.19:50458) 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f208d036640 1 -- 172.21.15.19:0/3052832102 learned_addr learned my addr 172.21.15.19:0/3052832102 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.310+0000 7f2087fff640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047cde660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 -- 172.21.15.19:0/3052832102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 msgr2=0x559047cde660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047cde660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 -- 172.21.15.19:0/3052832102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 msgr2=0x559047cde120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208c835640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047cde120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047cde120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 -- 172.21.15.19:0/3052832102 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559047cd8730 con 0x559047ce6370 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f2087fff640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047cde660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f208d036640 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2070097430 tx=0x7f207000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45786a61b33ff28a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f207e7fc640 1 -- 172.21.15.19:0/3052832102 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20700b9070 con 0x559047ce6370 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f207e7fc640 1 -- 172.21.15.19:0/3052832102 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207009be50 con 0x559047ce6370 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.311+0000 7f207e7fc640 1 -- 172.21.15.19:0/3052832102 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207009ae90 con 0x559047ce6370 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- 172.21.15.19:0/3052832102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 msgr2=0x559047cfae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f2070097430 tx=0x7f207000c8c0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- 172.21.15.19:0/3052832102 shutdown_connections 2024-09-12T03:27:30.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047ce6370 0x559047cfae40 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdcb40 0x559047cde660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 --2- 172.21.15.19:0/3052832102 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047cdb5d0 0x559047cde120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- 172.21.15.19:0/3052832102 >> 172.21.15.19:0/3052832102 conn(0x559047ca4310 msgr2=0x559047ce5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- 172.21.15.19:0/3052832102 shutdown_connections 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- 172.21.15.19:0/3052832102 wait complete. 2024-09-12T03:27:30.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.312+0000 7f2090e72740 1 -- start start 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 Processor -- start 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cf8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047d5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20881b0ae0 con 0x559047cdcb40 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20881b0410 con 0x559047ce6370 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20881ad5c0 con 0x559047cdb5d0 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2090e72740 10 client.0 _ll_register_callbacks cb 0x7f207009b5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:30.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:50464/0 (socket says 172.21.15.19:50464) 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 -- 172.21.15.19:0/3184712501 learned_addr learned my addr 172.21.15.19:0/3184712501 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f208d036640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047d5e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f208c835640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cf8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 -- 172.21.15.19:0/3184712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 msgr2=0x559047cf8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cf8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 -- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 msgr2=0x559047d5e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.313+0000 7f2087fff640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047d5e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2087fff640 1 -- 172.21.15.19:0/3184712501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559047d5ec80 con 0x559047cdcb40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f208c835640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cf8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2087fff640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x559047ce2860 tx=0x7f2078009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4c1ad36bba2b209 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2067fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2078004250 con 0x559047cdcb40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f20780043f0 con 0x559047cdcb40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f207009be50 con 0x559047cdcb40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f20780045f0 con 0x559047cdcb40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2090e72740 10 client.16986 fetch_fsmap learned FSMap version 40 2024-09-12T03:27:30.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x559047d5fe00 con 0x559047cdcb40 2024-09-12T03:27:30.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.314+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207009be50 con 0x559047cdcb40 2024-09-12T03:27:30.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 40) ==== 8406+0+0 (secure 0 0 0) 0x7f2078029020 con 0x559047cdcb40 2024-09-12T03:27:30.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 10 client.16986 fetch_fsmap finished waiting for FSMap version 40 2024-09-12T03:27:30.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 10 client.16986 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 20 client.16986 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047d605e0 0x559047d62a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 4 client.16986 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559047c48d70 con 0x559047d605e0 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f208c835640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047d605e0 0x559047d62a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2067fff640 10 client.16986 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:27:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f208c835640 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047d605e0 0x559047d62a00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e16faef4d0d3698d server_cookie=8d1b361db690961 in_seq=0 out_seq=0 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2067fff640 1 -- 172.21.15.19:0/3184712501 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559047c48d70 con 0x559047d605e0 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2067fff640 10 client.16986 handle_command_reply: tid=0 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.315+0000 7f2090e72740 1 client.16986 shutdown 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 msgr2=0x559047cf8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x559047ce2860 tx=0x7f2078009aa0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 shutdown_connections 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559047d605e0 0x559047d62a00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559047ce6370 0x559047d5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559047cdcb40 0x559047cf8540 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 --2- 172.21.15.19:0/3184712501 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559047cdb5d0 0x559047cf8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 >> 172.21.15.19:0/3184712501 conn(0x559047ca4310 msgr2=0x559047cd64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2067fff640 0 client.16986 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 shutdown_connections 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 1 -- 172.21.15.19:0/3184712501 wait complete. 2024-09-12T03:27:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f2090e72740 20 client.16986 trim_cache size 0 max 16384 2024-09-12T03:27:30.344 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 msgr2=0x7f20740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 0x7f20740a50c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2078005fd0 tx=0x7f2078009b90 comp rx=0 tx=0).stop 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 msgr2=0x7f208819a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.316+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f207000c970 tx=0x7f207000ce30 comp rx=0 tx=0).stop 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 shutdown_connections 2024-09-12T03:27:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20740a2c00 0x7f20740a50c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20881082a0 0x7f208819a5f0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20881036e0 0x7f20881a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 --2- 172.21.15.19:0/1772189976 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088102d30 0x7f20881a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 >> 172.21.15.19:0/1772189976 conn(0x7f20880fc7d0 msgr2=0x7f20880fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 shutdown_connections 2024-09-12T03:27:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:30.317+0000 7f208ea99640 1 -- 172.21.15.19:0/1772189976 wait complete. 2024-09-12T03:27:31.122 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:27:31.381 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:31.527 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:31 smithi019 ceph-mon[30953]: pgmap v591: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T03:27:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:31 smithi135 ceph-mon[35474]: pgmap v591: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T03:27:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:31 smithi121 ceph-mon[34963]: pgmap v591: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-12T03:27:31.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 -- 172.21.15.19:0/3949647126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 msgr2=0x7f58c41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/3949647126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41021c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009900 tx=0x7f58b402a410 comp rx=0 tx=0).stop 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 -- 172.21.15.19:0/3949647126 shutdown_connections 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/3949647126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4106f10 0x7f58c410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/3949647126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4102790 0x7f58c41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/3949647126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41021c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 -- 172.21.15.19:0/3949647126 >> 172.21.15.19:0/3949647126 conn(0x7f58c40fd8c0 msgr2=0x7f58c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 -- 172.21.15.19:0/3949647126 shutdown_connections 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.862+0000 7f58ca6a0640 1 -- 172.21.15.19:0/3949647126 wait complete. 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 Processor -- start 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 -- start start 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4102790 0x7f58c4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4106f10 0x7f58c419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4112010 con 0x7f58c4101de0 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4111e90 con 0x7f58c4106f10 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.863+0000 7f58ca6a0640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58c4112190 con 0x7f58c4102790 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59898/0 (socket says 172.21.15.19:59898) 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 -- 172.21.15.19:0/1855076186 learned_addr learned my addr 172.21.15.19:0/1855076186 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4102790 0x7f58c4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4106f10 0x7f58c419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 -- 172.21.15.19:0/1855076186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4102790 msgr2=0x7f58c4198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4102790 0x7f58c4198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 -- 172.21.15.19:0/1855076186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4106f10 msgr2=0x7f58c419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4106f10 0x7f58c419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 -- 172.21.15.19:0/1855076186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c419d560 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f58b4005ab0 tx=0x7f58b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c17fa640 1 -- 172.21.15.19:0/1855076186 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b403f030 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c17fa640 1 -- 172.21.15.19:0/1855076186 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b4003ce0 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c419d7f0 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58c17fa640 1 -- 172.21.15.19:0/1855076186 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4035070 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.864+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c41a5130 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58c17fa640 1 -- 172.21.15.19:0/1855076186 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f58b4042030 con 0x7f58c4101de0 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 Processor -- start 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 -- start start 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a02f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a03a370 0x555f8a03c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f89f952f0 con 0x555f8a02f5d0 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555f89f3ccf0 con 0x555f8a030b40 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58cca79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f89f912b0 con 0x555f8a03a370 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.867+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c17fa640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 0x7f58ac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59914/0 (socket says 172.21.15.19:59914) 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 -- 172.21.15.19:0/1693249548 learned_addr learned my addr 172.21.15.19:0/1693249548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a03a370 0x555f8a03c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a02f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 0x7f58ac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c17fa640 1 -- 172.21.15.19:0/1855076186 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58b40ef900 con 0x7f58c4101de0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 -- 172.21.15.19:0/1693249548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a03a370 msgr2=0x555f8a03c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a03a370 0x555f8a03c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 -- 172.21.15.19:0/1693249548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 msgr2=0x555f8a02f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a02f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 -- 172.21.15.19:0/1693249548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f89f213a0 con 0x555f8a02f5d0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f58b40c48b0 tx=0x7f58b402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9983f9ece9430a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58aa7fc640 1 -- 172.21.15.19:0/1693249548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4049070 con 0x555f8a02f5d0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 0x7f58ac0a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc009650 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58aa7fc640 1 -- 172.21.15.19:0/1693249548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b40bc870 con 0x555f8a02f5d0 2024-09-12T03:27:31.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.868+0000 7f58aa7fc640 1 -- 172.21.15.19:0/1693249548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b403b460 con 0x555f8a02f5d0 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- 172.21.15.19:0/1693249548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 msgr2=0x555f8a02fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f58b40c48b0 tx=0x7f58b402a410 comp rx=0 tx=0).stop 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- 172.21.15.19:0/1693249548 shutdown_connections 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a03a370 0x555f8a03c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a02fa10 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 --2- 172.21.15.19:0/1693249548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a02f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- 172.21.15.19:0/1693249548 >> 172.21.15.19:0/1693249548 conn(0x555f89ff8310 msgr2=0x555f8a02bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- 172.21.15.19:0/1693249548 shutdown_connections 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- 172.21.15.19:0/1693249548 wait complete. 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.869+0000 7f58cca79740 1 -- start start 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 Processor -- start 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 0x555f8a0370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a037860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555f8a03ce30 con 0x555f8a02f5d0 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555f89f50060 con 0x555f8a03a370 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x555f8a03cfb0 con 0x555f8a030b40 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58cca79740 10 client.0 _ll_register_callbacks cb 0x555f89ca38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59924/0 (socket says 172.21.15.19:59924) 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 -- 172.21.15.19:0/1872240129 learned_addr learned my addr 172.21.15.19:0/1872240129 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a037860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 0x555f8a0370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 -- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 msgr2=0x555f8a0370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 0x555f8a0370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 -- 172.21.15.19:0/1872240129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 msgr2=0x555f8a037860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a037860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c3fff640 1 -- 172.21.15.19:0/1872240129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f89d063c0 con 0x555f8a02f5d0 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.870+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 0x555f8a0370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58c3fff640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b8b40 tx=0x7f58b40bbde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0551a4de52bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b4049030 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58b40b87e0 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b40bc870 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f58b40c0520 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 10 client.16998 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f8a03ce30 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b40bc870 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f58b40b6ce0 con 0x555f8a02f5d0 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 10 client.16998 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 20 client.16998 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a049a60 0x555f8a04beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.871+0000 7f58cca79740 4 client.16998 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.872+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f89fa2860 con 0x555f8a049a60 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.872+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a049a60 0x555f8a04beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.872+0000 7f58c37fe640 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a049a60 0x555f8a04beb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92dabf466349d50 server_cookie=3756cfe96b7d520f in_seq=0 out_seq=0 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.872+0000 7f58a37fe640 10 client.16998 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58a37fe640 1 -- 172.21.15.19:0/1872240129 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f89fa2860 con 0x555f8a049a60 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58a37fe640 10 client.16998 handle_command_reply: tid=0 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 client.16998 shutdown 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 msgr2=0x555f8a039e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f58b40b8b40 tx=0x7f58b40bbde0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 shutdown_connections 2024-09-12T03:27:31.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a049a60 0x555f8a04beb0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58a37fe640 0 client.16998 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a037860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a030b40 0x555f8a0370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 --2- 172.21.15.19:0/1872240129 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a02f5d0 0x555f8a039e20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 >> 172.21.15.19:0/1872240129 conn(0x555f89ff8310 msgr2=0x555f8a031990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 shutdown_connections 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 1 -- 172.21.15.19:0/1872240129 wait complete. 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.873+0000 7f58cca79740 20 client.16998 trim_cache size 0 max 16384 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 Processor -- start 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 -- start start 2024-09-12T03:27:31.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a02be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a042550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41ad690 con 0x555f8a030b40 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41af2c0 con 0x555f8a03a370 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.886+0000 7f58cca79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41add60 con 0x555f8a02f5d0 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59940/0 (socket says 172.21.15.19:59940) 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 -- 172.21.15.19:0/3653367078 learned_addr learned my addr 172.21.15.19:0/3653367078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c3fff640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a02be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 -- 172.21.15.19:0/3653367078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 msgr2=0x555f8a02be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a02be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 -- 172.21.15.19:0/3653367078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 msgr2=0x555f8a042550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a042550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a042550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 -- 172.21.15.19:0/3653367078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8a04c530 con 0x555f8a030b40 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c3fff640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a02be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a042550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58c37fe640 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009bc0 tx=0x7f58b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245ce6c40cf1a941 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58a9ffb640 1 -- 172.21.15.19:0/3653367078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b801d310 con 0x555f8a030b40 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58a9ffb640 1 -- 172.21.15.19:0/3653367078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b801d990 con 0x555f8a030b40 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.887+0000 7f58a9ffb640 1 -- 172.21.15.19:0/3653367078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80159c0 con 0x555f8a030b40 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- 172.21.15.19:0/3653367078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 msgr2=0x555f8a036990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009bc0 tx=0x7f58b8008040 comp rx=0 tx=0).stop 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- 172.21.15.19:0/3653367078 shutdown_connections 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a03a370 0x555f8a042550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a030b40 0x555f8a036990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 --2- 172.21.15.19:0/3653367078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a02be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- 172.21.15.19:0/3653367078 >> 172.21.15.19:0/3653367078 conn(0x555f89ff8310 msgr2=0x555f8a02b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- 172.21.15.19:0/3653367078 shutdown_connections 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- 172.21.15.19:0/3653367078 wait complete. 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.888+0000 7f58cca79740 1 -- start start 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 Processor -- start 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a0ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a0acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41aeb30 con 0x555f8a03a370 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41ae460 con 0x555f8a030b40 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58c41ab640 con 0x555f8a02f5d0 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58cca79740 10 client.0 _ll_register_callbacks cb 0x7f58bc00b410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c8c3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c8c3d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59946/0 (socket says 172.21.15.19:59946) 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c8c3d640 1 -- 172.21.15.19:0/2722064381 learned_addr learned my addr 172.21.15.19:0/2722064381 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a0acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c3fff640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a0ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 -- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 msgr2=0x555f8a0ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a0ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 -- 172.21.15.19:0/2722064381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 msgr2=0x555f8a0ad1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c37fe640 1 -- 172.21.15.19:0/2722064381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8a046000 con 0x555f8a030b40 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.889+0000 7f58c8c3d640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58c37fe640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a0acc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x555f8a0301f0 tx=0x7f58b801d0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d32608b97f8a09b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58b80155c0 con 0x555f8a030b40 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58b801d990 con 0x555f8a030b40 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58b801d310 con 0x555f8a030b40 2024-09-12T03:27:31.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f58b801d510 con 0x555f8a030b40 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58cca79740 10 client.26423 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555f8a0b8970 con 0x555f8a030b40 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58b801d990 con 0x555f8a030b40 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.890+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f58b8026720 con 0x555f8a030b40 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 10 client.26423 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 10 client.26423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 20 client.26423 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a0b90f0 0x555f8a0bb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 4 client.26423 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555f89f9cd70 con 0x555f8a0b90f0 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58c3fff640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a0b90f0 0x555f8a0bb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58a37fe640 10 client.26423 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:31.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58c3fff640 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a0b90f0 0x555f8a0bb510 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b03760c546a4b8a server_cookie=36b9d08ce76f2b31 in_seq=0 out_seq=0 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58a37fe640 1 -- 172.21.15.19:0/2722064381 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555f89f9cd70 con 0x555f8a0b90f0 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58a37fe640 10 client.26423 handle_command_reply: tid=0 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.891+0000 7f58cca79740 1 client.26423 shutdown 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 msgr2=0x555f8a0acc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a0acc90 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x555f8a0301f0 tx=0x7f58b801d0d0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 shutdown_connections 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555f8a03a370 0x555f8a0ad1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x555f8a0b90f0 0x555f8a0bb510 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555f8a030b40 0x555f8a0acc90 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 --2- 172.21.15.19:0/2722064381 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555f8a02f5d0 0x555f8a0ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58a37fe640 0 client.26423 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 >> 172.21.15.19:0/2722064381 conn(0x555f89ff8310 msgr2=0x555f8a02a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 shutdown_connections 2024-09-12T03:27:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 1 -- 172.21.15.19:0/2722064381 wait complete. 2024-09-12T03:27:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58cca79740 20 client.26423 trim_cache size 0 max 16384 2024-09-12T03:27:31.912 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 msgr2=0x7f58ac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 0x7f58ac0a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f58bc009900 tx=0x7f58bc009650 comp rx=0 tx=0).stop 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 msgr2=0x7f58c41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.892+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f58b4005ab0 tx=0x7f58b40042f0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 shutdown_connections 2024-09-12T03:27:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f58c4106f10 0x7f58c419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f58c4102790 0x7f58c4198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58ac0a2b80 0x7f58ac0a5040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 --2- 172.21.15.19:0/1855076186 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f58c4101de0 0x7f58c41985b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 >> 172.21.15.19:0/1855076186 conn(0x7f58c40fd8c0 msgr2=0x7f58c40fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 shutdown_connections 2024-09-12T03:27:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:31.893+0000 7f58ca6a0640 1 -- 172.21.15.19:0/1855076186 wait complete. 2024-09-12T03:27:32.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:32 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:32.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:32 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:32.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:32 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:33 smithi135 ceph-mon[35474]: pgmap v592: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:27:33.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:33 smithi121 ceph-mon[34963]: pgmap v592: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:27:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:33 smithi019 ceph-mon[30953]: pgmap v592: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:27:33.749 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:27:33.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:27:34.023 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:34.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 -- 172.21.15.19:0/1763998930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c102d30 msgr2=0x7f9f3c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 --2- 172.21.15.19:0/1763998930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c102d30 0x7f9f3c103110 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009930 tx=0x7f9f2c02a420 comp rx=0 tx=0).stop 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 -- 172.21.15.19:0/1763998930 shutdown_connections 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 --2- 172.21.15.19:0/1763998930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 --2- 172.21.15.19:0/1763998930 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 --2- 172.21.15.19:0/1763998930 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c102d30 0x7f9f3c103110 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 -- 172.21.15.19:0/1763998930 >> 172.21.15.19:0/1763998930 conn(0x7f9f3c0fc7d0 msgr2=0x7f9f3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 -- 172.21.15.19:0/1763998930 shutdown_connections 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.534+0000 7f9f40f27640 1 -- 172.21.15.19:0/1763998930 wait complete. 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 Processor -- start 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 -- start start 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 0x7f9f3c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1120a0 con 0x7f9f3c1082a0 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c111f20 con 0x7f9f3c1036e0 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f40f27640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c112220 con 0x7f9f3c102d30 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59954/0 (socket says 172.21.15.19:59954) 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 -- 172.21.15.19:0/3409190207 learned_addr learned my addr 172.21.15.19:0/3409190207 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 0x7f9f3c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 -- 172.21.15.19:0/3409190207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 msgr2=0x7f9f3c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 0x7f9f3c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 -- 172.21.15.19:0/3409190207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 msgr2=0x7f9f3c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3ad9d640 1 -- 172.21.15.19:0/3409190207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f3c19d5c0 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 0x7f9f3c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.535+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f3ad9d640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400cad0 tx=0x7f9f3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f337fe640 1 -- 172.21.15.19:0/3409190207 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400ae20 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f3c19d8b0 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f337fe640 1 -- 172.21.15.19:0/3409190207 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f34005020 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f337fe640 1 -- 172.21.15.19:0/3409190207 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400f6f0 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.536+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f3c1a5190 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.538+0000 7f9f337fe640 1 -- 172.21.15.19:0/3409190207 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f340051d0 con 0x7f9f3c1082a0 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.538+0000 7f9f43300740 1 Processor -- start 2024-09-12T03:27:34.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 -- start start 2024-09-12T03:27:34.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0515e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0527540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635c047b280 con 0x5635c0516360 2024-09-12T03:27:34.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5635c0422b10 con 0x5635c0515a40 2024-09-12T03:27:34.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f43300740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635c0477240 con 0x5635c0525150 2024-09-12T03:27:34.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f337fe640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 0x7f9f280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59960/0 (socket says 172.21.15.19:59960) 2024-09-12T03:27:34.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 -- 172.21.15.19:0/2210751892 learned_addr learned my addr 172.21.15.19:0/2210751892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:34.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0515e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0527540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f337fe640 1 -- 172.21.15.19:0/3409190207 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f340cf110 con 0x7f9f3c1082a0 2024-09-12T03:27:34.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3ad9d640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 0x7f9f280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 -- 172.21.15.19:0/2210751892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 msgr2=0x5635c0527540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0527540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 -- 172.21.15.19:0/2210751892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 msgr2=0x5635c0515e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0515e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.539+0000 7f9f3a59c640 1 -- 172.21.15.19:0/2210751892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c042f1f0 con 0x5635c0516360 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.540+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9b39521d374d90 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.540+0000 7f9f3ad9d640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 0x7f9f280a5040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f340ccc20 tx=0x7f9f34005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.540+0000 7f9f30ff9640 1 -- 172.21.15.19:0/2210751892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c03f070 con 0x5635c0516360 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.540+0000 7f9f30ff9640 1 -- 172.21.15.19:0/2210751892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c0317b0 con 0x5635c0516360 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.540+0000 7f9f30ff9640 1 -- 172.21.15.19:0/2210751892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c035070 con 0x5635c0516360 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- 172.21.15.19:0/2210751892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 msgr2=0x5635c0524c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c006010 tx=0x7f9f2c009680 comp rx=0 tx=0).stop 2024-09-12T03:27:34.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- 172.21.15.19:0/2210751892 shutdown_connections 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0527540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0524c10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- 172.21.15.19:0/2210751892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0515e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- 172.21.15.19:0/2210751892 >> 172.21.15.19:0/2210751892 conn(0x5635c05120f0 msgr2=0x5635c05124d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- 172.21.15.19:0/2210751892 shutdown_connections 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- 172.21.15.19:0/2210751892 wait complete. 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- start start 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 Processor -- start 2024-09-12T03:27:34.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0522040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0522ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5635c0527d20 con 0x5635c0516360 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5635c0435f10 con 0x5635c0515a40 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5635c0527ea0 con 0x5635c0525150 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.541+0000 7f9f43300740 10 client.0 _ll_register_callbacks cb 0x5635c03fea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59976/0 (socket says 172.21.15.19:59976) 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 -- 172.21.15.19:0/770306781 learned_addr learned my addr 172.21.15.19:0/770306781 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0522ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0522040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 -- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 msgr2=0x5635c0522ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0522ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 -- 172.21.15.19:0/770306781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 msgr2=0x5635c0522040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0522040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 -- 172.21.15.19:0/770306781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c01ec6a0 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2cfd4a5ea0104b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c035040 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f2c0046c0 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f2c0317b0 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9f2c03f040 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f43300740 10 client.17016 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.542+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5635c0533970 con 0x5635c0516360 2024-09-12T03:27:34.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f2c0317b0 con 0x5635c0516360 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f9f2c03b8d0 con 0x5635c0516360 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 10 client.17016 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 10 client.17016 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 20 client.17016 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c0534cc0 0x5635c05370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 4 client.17016 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635c04887f0 con 0x5635c0534cc0 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.543+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c0534cc0 0x5635c05370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c0534cc0 0x5635c05370b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b436883bff9dae7 server_cookie=ba7d1451e990a22f in_seq=0 out_seq=0 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f21ffb640 10 client.17016 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f21ffb640 1 -- 172.21.15.19:0/770306781 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5635c04887f0 con 0x5635c0534cc0 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f21ffb640 10 client.17016 handle_command_reply: tid=0 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 client.17016 shutdown 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 msgr2=0x5635c0522580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c009260 comp rx=0 tx=0).stop 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 shutdown_connections 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0525150 0x5635c0522ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0516360 0x5635c0522580 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c0534cc0 0x5635c05370b0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 --2- 172.21.15.19:0/770306781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c0522040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f21ffb640 0 client.17016 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.544+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 >> 172.21.15.19:0/770306781 conn(0x5635c05120f0 msgr2=0x5635c0517e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.545+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 shutdown_connections 2024-09-12T03:27:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.545+0000 7f9f43300740 1 -- 172.21.15.19:0/770306781 wait complete. 2024-09-12T03:27:34.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.545+0000 7f9f43300740 20 client.17016 trim_cache size 0 max 16384 2024-09-12T03:27:34.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 Processor -- start 2024-09-12T03:27:34.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 -- start start 2024-09-12T03:27:34.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0515a40 0x5635c053c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 0x5635c053cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ab3f0 con 0x5635c0525150 2024-09-12T03:27:34.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ad020 con 0x5635c0516360 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f43300740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1abac0 con 0x5635c0515a40 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59992/0 (socket says 172.21.15.19:59992) 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.562+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1960420584 learned_addr learned my addr 172.21.15.19:0/1960420584 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 0x5635c053cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1960420584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0515a40 msgr2=0x5635c053c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0515a40 0x5635c053c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1960420584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 msgr2=0x5635c053cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 0x5635c053cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1960420584 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635c0516190 con 0x5635c0525150 2024-09-12T03:27:34.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 0x5635c053cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:27:34.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0fee80 tx=0x7f9f240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4954d3849ac3439e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f23fff640 1 -- 172.21.15.19:0/1960420584 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24026030 con 0x5635c0525150 2024-09-12T03:27:34.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f23fff640 1 -- 172.21.15.19:0/1960420584 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24014e10 con 0x5635c0525150 2024-09-12T03:27:34.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.563+0000 7f9f23fff640 1 -- 172.21.15.19:0/1960420584 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24012e30 con 0x5635c0525150 2024-09-12T03:27:34.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- 172.21.15.19:0/1960420584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 msgr2=0x5635c051f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0fee80 tx=0x7f9f240046c0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- 172.21.15.19:0/1960420584 shutdown_connections 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c051f2a0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0516360 0x5635c053cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- 172.21.15.19:0/1960420584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0515a40 0x5635c053c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- 172.21.15.19:0/1960420584 >> 172.21.15.19:0/1960420584 conn(0x5635c05120f0 msgr2=0x5635c0523fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- 172.21.15.19:0/1960420584 shutdown_connections 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- 172.21.15.19:0/1960420584 wait complete. 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 -- start start 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 Processor -- start 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c05290d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0516360 0x5635c0529610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.564+0000 7f9f43300740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f43300740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ac890 con 0x5635c0525150 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f43300740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1ac1c0 con 0x5635c0515a40 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f43300740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3c1a93a0 con 0x5635c0516360 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f43300740 10 client.0 _ll_register_callbacks cb 0x7f9f2400b960 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:59994/0 (socket says 172.21.15.19:59994) 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1453879447 learned_addr learned my addr 172.21.15.19:0/1453879447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c05290d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0516360 0x5635c0529610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0516360 msgr2=0x5635c0529610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0516360 0x5635c0529610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1453879447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 msgr2=0x5635c05290d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c05290d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 -- 172.21.15.19:0/1453879447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635c059e930 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f39d9b640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400bba0 tx=0x7f9f24012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee068d3aa82d88b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f21ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f24037060 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f24005230 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.565+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f24014e10 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f9f24026070 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 10 client.17022 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5635c059fbc0 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f24014e10 con 0x5635c0525150 2024-09-12T03:27:34.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f9f2400f040 con 0x5635c0525150 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 10 client.17022 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 10 client.17022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 20 client.17022 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c05a0290 0x5635c05a26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 4 client.17022 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5635c0482d00 con 0x5635c05a0290 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.566+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c05a0290 0x5635c05a26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.567+0000 7f9f3a59c640 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c05a0290 0x5635c05a26b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239d4957c7ffa19f server_cookie=b4136ec97f51e8a6 in_seq=0 out_seq=0 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.567+0000 7f9f21ffb640 10 client.17022 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f21ffb640 1 -- 172.21.15.19:0/1453879447 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5635c0482d00 con 0x5635c05a0290 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f21ffb640 10 client.17022 handle_command_reply: tid=0 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 client.17022 shutdown 2024-09-12T03:27:34.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 msgr2=0x5635c0529b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400bba0 tx=0x7f9f24012c10 comp rx=0 tx=0).stop 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 shutdown_connections 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5635c0525150 0x5635c0529b50 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5635c0516360 0x5635c0529610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5635c05a0290 0x5635c05a26b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 --2- 172.21.15.19:0/1453879447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5635c0515a40 0x5635c05290d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 >> 172.21.15.19:0/1453879447 conn(0x5635c05120f0 msgr2=0x5635c0511270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f21ffb640 0 client.17022 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 shutdown_connections 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 1 -- 172.21.15.19:0/1453879447 wait complete. 2024-09-12T03:27:34.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.568+0000 7f9f43300740 20 client.17022 trim_cache size 0 max 16384 2024-09-12T03:27:34.576 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:34.576 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:27:34.576 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3f8073c3-a55e-4ef4-9399-8da6a71f7bc3", 2024-09-12T03:27:34.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:27:34.576 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 msgr2=0x7f9f280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 0x7f9f280a5040 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f340ccc20 tx=0x7f9f34005b40 comp rx=0 tx=0).stop 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 msgr2=0x7f9f3c19cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400cad0 tx=0x7f9f3400cf90 comp rx=0 tx=0).stop 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 shutdown_connections 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f280a2b80 0x7f9f280a5040 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f3c1082a0 0x7f9f3c19cee0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f3c1036e0 0x7f9f3c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 --2- 172.21.15.19:0/3409190207 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f3c102d30 0x7f9f3c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:34.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 >> 172.21.15.19:0/3409190207 conn(0x7f9f3c0fc7d0 msgr2=0x7f9f3c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:34.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 shutdown_connections 2024-09-12T03:27:34.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:34.569+0000 7f9f40f27640 1 -- 172.21.15.19:0/3409190207 wait complete. 2024-09-12T03:27:35.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:27:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: pgmap v593: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:27:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:35 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: pgmap v593: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:35 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:35.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: pgmap v593: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.9 KiB/s wr, 12 op/s 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:35 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:35.863 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:36 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:27:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:36 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:36.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:36 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:27:36.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:36 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:27:36.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:36 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:36.642 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:36 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:27:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:36 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:27:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:36 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:36 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:27:36.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 -- 172.21.15.19:0/4119603497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49941049a0 msgr2=0x7f49941089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 --2- 172.21.15.19:0/4119603497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49941049a0 0x7f49941089f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4980009860 tx=0x7f498002a340 comp rx=0 tx=0).stop 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 -- 172.21.15.19:0/4119603497 shutdown_connections 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 --2- 172.21.15.19:0/4119603497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994109120 0x7f4994110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 --2- 172.21.15.19:0/4119603497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f49941049a0 0x7f49941089f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 --2- 172.21.15.19:0/4119603497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4994103ff0 0x7f49941043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 -- 172.21.15.19:0/4119603497 >> 172.21.15.19:0/4119603497 conn(0x7f49940ffa70 msgr2=0x7f4994101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 -- 172.21.15.19:0/4119603497 shutdown_connections 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.640+0000 7f499b1b7640 1 -- 172.21.15.19:0/4119603497 wait complete. 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.641+0000 7f499b1b7640 1 Processor -- start 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 -- start start 2024-09-12T03:27:36.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49941049a0 0x7f499419abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4994109120 0x7f499419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4994114170 con 0x7f4994103ff0 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4994113ff0 con 0x7f4994109120 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f499b1b7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49941142f0 con 0x7f49941049a0 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f4998f53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f4998f53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60012/0 (socket says 172.21.15.19:60012) 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f4998f53640 1 -- 172.21.15.19:0/858251793 learned_addr learned my addr 172.21.15.19:0/858251793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f4999754640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4994109120 0x7f499419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.642+0000 7f4993fff640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49941049a0 0x7f499419abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 -- 172.21.15.19:0/858251793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49941049a0 msgr2=0x7f499419abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49941049a0 0x7f499419abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 -- 172.21.15.19:0/858251793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4994109120 msgr2=0x7f499419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4994109120 0x7f499419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 -- 172.21.15.19:0/858251793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499419f660 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.643+0000 7f4998f53640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f498800e9c0 tx=0x7f498800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.644+0000 7f4991ffb640 1 -- 172.21.15.19:0/858251793 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498800ccc0 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.644+0000 7f4991ffb640 1 -- 172.21.15.19:0/858251793 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4988005500 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.644+0000 7f4991ffb640 1 -- 172.21.15.19:0/858251793 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4988010430 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.644+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499419f950 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.644+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49941a7230 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.648+0000 7f4991ffb640 1 -- 172.21.15.19:0/858251793 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f49880046c0 con 0x7f4994103ff0 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.649+0000 7f499d590740 1 Processor -- start 2024-09-12T03:27:36.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.649+0000 7f499d590740 1 -- start start 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f9ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f611ef52e0 con 0x55f611f8f4a0 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f611e9cce0 con 0x55f611f9a680 2024-09-12T03:27:36.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f499d590740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f611ef12a0 con 0x55f611f8fdc0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f4991ffb640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 0x7f49840a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f4999754640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f4999754640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60018/0 (socket says 172.21.15.19:60018) 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f4999754640 1 -- 172.21.15.19:0/3983809485 learned_addr learned my addr 172.21.15.19:0/3983809485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.650+0000 7f4991ffb640 1 -- 172.21.15.19:0/858251793 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f49880cf310 con 0x7f4994103ff0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 0x7f49840a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4998f53640 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4993fff640 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f9ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 -- 172.21.15.19:0/3983809485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 msgr2=0x55f611f9a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f9a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 -- 172.21.15.19:0/3983809485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 msgr2=0x55f611f9ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f9ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.651+0000 7f4999754640 1 -- 172.21.15.19:0/3983809485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f611e81440 con 0x55f611f8f4a0 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.652+0000 7f4999754640 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f498c012d30 tx=0x7f498c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469a28a64b5e0100 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.652+0000 7f4999754640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 0x7f49840a5250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f498c005c40 tx=0x7f498c01e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.652+0000 7f497effd640 1 -- 172.21.15.19:0/3983809485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c02a070 con 0x55f611f8f4a0 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.652+0000 7f497effd640 1 -- 172.21.15.19:0/3983809485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c003e50 con 0x55f611f8f4a0 2024-09-12T03:27:36.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.652+0000 7f497effd640 1 -- 172.21.15.19:0/3983809485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0043f0 con 0x55f611f8f4a0 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 -- 172.21.15.19:0/3983809485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 msgr2=0x55f611f8f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f498c012d30 tx=0x7f498c0107a0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 -- 172.21.15.19:0/3983809485 shutdown_connections 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f9ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f9a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 --2- 172.21.15.19:0/3983809485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f8f880 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 -- 172.21.15.19:0/3983809485 >> 172.21.15.19:0/3983809485 conn(0x55f611f8ba30 msgr2=0x55f611f8be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.653+0000 7f499d590740 1 -- 172.21.15.19:0/3983809485 shutdown_connections 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.654+0000 7f499d590740 1 -- 172.21.15.19:0/3983809485 wait complete. 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.654+0000 7f499d590740 1 -- start start 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.655+0000 7f499d590740 1 Processor -- start 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.655+0000 7f499d590740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.655+0000 7f499d590740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.655+0000 7f499d590740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f97840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.655+0000 7f4998f53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60034/0 (socket says 172.21.15.19:60034) 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 -- 172.21.15.19:0/976317156 learned_addr learned my addr 172.21.15.19:0/976317156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f611fa5a60 con 0x55f611f8f4a0 2024-09-12T03:27:36.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f611eb0050 con 0x55f611f9a680 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f611fa5be0 con 0x55f611f8fdc0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f499d590740 10 client.0 _ll_register_callbacks cb 0x55f611ef0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4993fff640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4999754640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f97840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 -- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 msgr2=0x55f611f972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 -- 172.21.15.19:0/976317156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 msgr2=0x55f611f97840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f97840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.656+0000 7f4998f53640 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f611c663c0 con 0x55f611f8f4a0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f4998f53640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f498800c6a0 tx=0x7f498809b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c80ef9b37f52e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f4973fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49880a4500 con 0x55f611f8f4a0 2024-09-12T03:27:36.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f49880ba270 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c003e50 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f49880a4790 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.657+0000 7f499d590740 10 client.17034 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f611fb1420 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49880b9070 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f4988097870 con 0x55f611f8f4a0 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 10 client.17034 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 20 client.17034 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f611fb28e0 0x55f611fb4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 4 client.17034 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:36.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.658+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f498c003e50 con 0x55f611fb28e0 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.659+0000 7f4993fff640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f611fb28e0 0x55f611fb4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.659+0000 7f4973fff640 10 client.17034 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.660+0000 7f4993fff640 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f611fb28e0 0x55f611fb4d20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428c35557bd37613 server_cookie=c158ef4de62d2bb7 in_seq=0 out_seq=0 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f4973fff640 1 -- 172.21.15.19:0/976317156 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f498c003e50 con 0x55f611fb28e0 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f4973fff640 10 client.17034 handle_command_reply: tid=0 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 client.17034 shutdown 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 msgr2=0x55f611f9a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f498800c6a0 tx=0x7f498809b540 comp rx=0 tx=0).stop 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 shutdown_connections 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f611fb28e0 0x55f611fb4d20 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f4973fff640 0 client.17034 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:36.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f9a680 0x55f611f97840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f8fdc0 0x55f611f972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 --2- 172.21.15.19:0/976317156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f8f4a0 0x55f611f9a010 secure :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f498800c6a0 tx=0x7f498809b540 comp rx=0 tx=0).stop 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.662+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 >> 172.21.15.19:0/976317156 conn(0x55f611f8ba30 msgr2=0x55f611f91710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.663+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 shutdown_connections 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.663+0000 7f499d590740 1 -- 172.21.15.19:0/976317156 wait complete. 2024-09-12T03:27:36.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.663+0000 7f499d590740 20 client.17034 trim_cache size 0 max 16384 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.690+0000 7f499d590740 1 Processor -- start 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.690+0000 7f499d590740 1 -- start start 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fbab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611f949a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ab400 con 0x55f611f94520 2024-09-12T03:27:36.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ad030 con 0x55f611f8fdc0 2024-09-12T03:27:36.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f499d590740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49941abad0 con 0x55f611f9a680 2024-09-12T03:27:36.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f4993fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fbab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f4993fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fbab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:39006/0 (socket says 172.21.15.19:39006) 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f4993fff640 1 -- 172.21.15.19:0/957545588 learned_addr learned my addr 172.21.15.19:0/957545588 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f4999754640 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611f949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.691+0000 7f4998f53640 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 -- 172.21.15.19:0/957545588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 msgr2=0x55f611fbab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fbab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 -- 172.21.15.19:0/957545588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 msgr2=0x55f611fba5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fba5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 -- 172.21.15.19:0/957545588 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f611f8f220 con 0x55f611f94520 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f4999754640 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611f949a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f498c0094a0 tx=0x7f498c012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92745406dca240cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f497e7fc640 1 -- 172.21.15.19:0/957545588 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c033070 con 0x55f611f94520 2024-09-12T03:27:36.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f497e7fc640 1 -- 172.21.15.19:0/957545588 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f498c0124d0 con 0x55f611f94520 2024-09-12T03:27:36.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.692+0000 7f497e7fc640 1 -- 172.21.15.19:0/957545588 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c02a070 con 0x55f611f94520 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 -- 172.21.15.19:0/957545588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 msgr2=0x55f611f949a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611f949a0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f498c0094a0 tx=0x7f498c012920 comp rx=0 tx=0).stop 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 -- 172.21.15.19:0/957545588 shutdown_connections 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611f949a0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fbab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 --2- 172.21.15.19:0/957545588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fba5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.693+0000 7f499d590740 1 -- 172.21.15.19:0/957545588 >> 172.21.15.19:0/957545588 conn(0x55f611f8ba30 msgr2=0x55f611f8ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.694+0000 7f499d590740 1 -- 172.21.15.19:0/957545588 shutdown_connections 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.694+0000 7f499d590740 1 -- 172.21.15.19:0/957545588 wait complete. 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.694+0000 7f499d590740 1 -- start start 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.694+0000 7f499d590740 1 Processor -- start 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611fa9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fa9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ac8a0 con 0x55f611f94520 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f49941ac1d0 con 0x55f611f8fdc0 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f49941a93b0 con 0x55f611f9a680 2024-09-12T03:27:36.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f499d590740 10 client.0 _ll_register_callbacks cb 0x7f498809b980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4993fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611fa9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59884/0 (socket says 172.21.15.19:59884) 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 -- 172.21.15.19:0/616969482 learned_addr learned my addr 172.21.15.19:0/616969482 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4999754640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fa9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 -- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 msgr2=0x55f611fa9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fa9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 -- 172.21.15.19:0/616969482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 msgr2=0x55f611fa9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611fa9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 -- 172.21.15.19:0/616969482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f611faa0c0 con 0x55f611f8fdc0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4993fff640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611fa9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.695+0000 7f4998f53640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4988097d10 tx=0x7f498809b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71f52f6ae8253311 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:36.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f4973fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:27:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498809af70 con 0x55f611f8fdc0 2024-09-12T03:27:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f498809b110 con 0x55f611f8fdc0 2024-09-12T03:27:36.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f498c0124d0 con 0x55f611f8fdc0 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4988086410 con 0x55f611f8fdc0 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.696+0000 7f499d590740 10 client.26450 fetch_fsmap learned FSMap version 41 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f61201e320 con 0x55f611f8fdc0 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49880b9040 con 0x55f611f8fdc0 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 41) ==== 8402+0+0 (secure 0 0 0) 0x7f49880a1060 con 0x55f611f8fdc0 2024-09-12T03:27:36.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 10 client.26450 fetch_fsmap finished waiting for FSMap version 41 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 10 client.26450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 20 client.26450 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f61201e9f0 0x55f612020de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 4 client.26450 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f498c0124d0 con 0x55f61201e9f0 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.697+0000 7f4993fff640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f61201e9f0 0x55f612020de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.698+0000 7f4993fff640 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f61201e9f0 0x55f612020de0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83956b53ff1f54cc server_cookie=a20a59c1a2eea9d6 in_seq=0 out_seq=0 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.698+0000 7f4973fff640 10 client.26450 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.699+0000 7f4973fff640 1 -- 172.21.15.19:0/616969482 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x7f498c0124d0 con 0x55f61201e9f0 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.699+0000 7f4973fff640 10 client.26450 handle_command_reply: tid=0 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.699+0000 7f499d590740 1 client.26450 shutdown 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.699+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 msgr2=0x55f611fab290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.699+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4988097d10 tx=0x7f498809b5b0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 shutdown_connections 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f611f9a680 0x55f611fa9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f611f94520 0x55f611fa9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f61201e9f0 0x55f612020de0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f4973fff640 0 client.26450 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 --2- 172.21.15.19:0/616969482 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f611f8fdc0 0x55f611fab290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 >> 172.21.15.19:0/616969482 conn(0x55f611f8ba30 msgr2=0x55f611f8ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 shutdown_connections 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.700+0000 7f499d590740 1 -- 172.21.15.19:0/616969482 wait complete. 2024-09-12T03:27:36.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.701+0000 7f499d590740 20 client.26450 trim_cache size 0 max 16384 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (4857 inodes in the stack)", 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "3f8073c3-a55e-4ef4-9399-8da6a71f7bc3": { 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "3f8073c3-a55e-4ef4-9399-8da6a71f7bc3", 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:36.705 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 msgr2=0x7f49840a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 0x7f49840a5250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f498c005c40 tx=0x7f498c01e000 comp rx=0 tx=0).stop 2024-09-12T03:27:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 msgr2=0x7f499419a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:36.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f498800e9c0 tx=0x7f498800ee80 comp rx=0 tx=0).stop 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 shutdown_connections 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4994109120 0x7f499419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f49941049a0 0x7f499419abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f49840a2d90 0x7f49840a5250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 --2- 172.21.15.19:0/858251793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4994103ff0 0x7f499419a6b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 >> 172.21.15.19:0/858251793 conn(0x7f49940ffa70 msgr2=0x7f4994100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 shutdown_connections 2024-09-12T03:27:36.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:36.702+0000 7f499b1b7640 1 -- 172.21.15.19:0/858251793 wait complete. 2024-09-12T03:27:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:37 smithi121 ceph-mon[34963]: pgmap v594: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-12T03:27:37.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:37 smithi019 ceph-mon[30953]: pgmap v594: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-12T03:27:37.756 INFO:tasks.cephfs.filesystem:scrub status for tag:3f8073c3-a55e-4ef4-9399-8da6a71f7bc3 - {'path': '/', 'tag': '3f8073c3-a55e-4ef4-9399-8da6a71f7bc3', 'options': 'recursive,force'} 2024-09-12T03:27:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:37 smithi135 ceph-mon[35474]: pgmap v594: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2024-09-12T03:27:38.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:38 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:38 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:38 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:38 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:38 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:38 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:38 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:38 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:38 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:39 smithi121 ceph-mon[34963]: pgmap v595: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-09-12T03:27:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:39 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:39.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:39 smithi019 ceph-mon[30953]: pgmap v595: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-09-12T03:27:39.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:39 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:39 smithi135 ceph-mon[35474]: pgmap v595: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 8.5 KiB/s wr, 26 op/s 2024-09-12T03:27:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:27:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:39 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:27:40.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:40.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:40.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:27:40.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:40.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:40.670 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:27:40.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:27:40.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:27:40.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:27:41.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:41 smithi121 ceph-mon[34963]: pgmap v596: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 KiB/s wr, 36 op/s 2024-09-12T03:27:41.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:41 smithi019 ceph-mon[30953]: pgmap v596: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 KiB/s wr, 36 op/s 2024-09-12T03:27:41.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:41 smithi135 ceph-mon[35474]: pgmap v596: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 9.2 KiB/s wr, 36 op/s 2024-09-12T03:27:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:43 smithi121 ceph-mon[34963]: pgmap v597: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.9 KiB/s wr, 49 op/s 2024-09-12T03:27:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:43 smithi019 ceph-mon[30953]: pgmap v597: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.9 KiB/s wr, 49 op/s 2024-09-12T03:27:43.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:43 smithi135 ceph-mon[35474]: pgmap v597: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 9.9 KiB/s wr, 49 op/s 2024-09-12T03:27:44.453 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-09-12T03:27:44.454 INFO:tasks.workunit.client.0.smithi019.stdout:Final score for writes: 607 2024-09-12T03:27:44.454 INFO:tasks.workunit.client.0.smithi019.stdout:Final score for reads : 657497 2024-09-12T03:27:44.454 INFO:tasks.workunit.client.0.smithi019.stdout: 2024-09-12T03:27:44.454 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:27:44.455 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:27:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:45 smithi121 ceph-mon[34963]: pgmap v598: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 KiB/s wr, 49 op/s 2024-09-12T03:27:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:45 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:45 smithi019 ceph-mon[30953]: pgmap v598: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 KiB/s wr, 49 op/s 2024-09-12T03:27:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:45 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:45 smithi135 ceph-mon[35474]: pgmap v598: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 12 KiB/s wr, 49 op/s 2024-09-12T03:27:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:45 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:47 smithi121 ceph-mon[34963]: pgmap v599: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 KiB/s wr, 53 op/s 2024-09-12T03:27:47.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:47 smithi019 ceph-mon[30953]: pgmap v599: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 KiB/s wr, 53 op/s 2024-09-12T03:27:47.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:47 smithi135 ceph-mon[35474]: pgmap v599: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 20 KiB/s wr, 53 op/s 2024-09-12T03:27:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:49 smithi135 ceph-mon[35474]: pgmap v600: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 208 KiB/s wr, 74 op/s 2024-09-12T03:27:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:49 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:49 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:49.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:49 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:49 smithi121 ceph-mon[34963]: pgmap v600: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 208 KiB/s wr, 74 op/s 2024-09-12T03:27:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:49 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:49 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:49 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:49 smithi019 ceph-mon[30953]: pgmap v600: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 208 KiB/s wr, 74 op/s 2024-09-12T03:27:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:49 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:49 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:49 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:50.620 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:27:50.620 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T03:27:50.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:27:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:50 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:50 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:27:50.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:50 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:50 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:50 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:27:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:50 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:50 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:50 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:27:50.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:50 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:50.911 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 -- 172.21.15.19:0/69595778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 msgr2=0x7f1980103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 --2- 172.21.15.19:0/69595778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f1980103130 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1970009900 tx=0x7f197002a480 comp rx=0 tx=0).stop 2024-09-12T03:27:51.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 -- 172.21.15.19:0/69595778 shutdown_connections 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 --2- 172.21.15.19:0/69595778 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 0x7f198010e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 --2- 172.21.15.19:0/69595778 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 0x7f198010bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 --2- 172.21.15.19:0/69595778 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f1980103130 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 -- 172.21.15.19:0/69595778 >> 172.21.15.19:0/69595778 conn(0x7f19800fc7d0 msgr2=0x7f19800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 -- 172.21.15.19:0/69595778 shutdown_connections 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.526+0000 7f198497d640 1 -- 172.21.15.19:0/69595778 wait complete. 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 Processor -- start 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 -- start start 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 0x7f1980100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 0x7f1980101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:51.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1980111f80 con 0x7f1980102d50 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1980111e00 con 0x7f198010c180 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f198497d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1980112100 con 0x7f1980103670 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36672/0 (socket says 172.21.15.19:36672) 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197e59c640 1 -- 172.21.15.19:0/676620887 learned_addr learned my addr 172.21.15.19:0/676620887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197ed9d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 0x7f1980101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197dd9b640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 0x7f1980100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.527+0000 7f197e59c640 1 -- 172.21.15.19:0/676620887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 msgr2=0x7f1980100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f197e59c640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 0x7f1980100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f197e59c640 1 -- 172.21.15.19:0/676620887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 msgr2=0x7f1980101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f197e59c640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 0x7f1980101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f197e59c640 1 -- 172.21.15.19:0/676620887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19801a6940 con 0x7f1980102d50 2024-09-12T03:27:51.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f197e59c640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1970006010 tx=0x7f1970004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197003f030 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19801a6bd0 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1970004440 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970035070 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.528+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19801a7100 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.529+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1980103130 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.531+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1970042030 con 0x7f1980102d50 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.531+0000 7f196f7fe640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 0x7f19680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.531+0000 7f197dd9b640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 0x7f19680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:51.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.531+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f19700ef650 con 0x7f1980102d50 2024-09-12T03:27:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.532+0000 7f197dd9b640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 0x7f19680a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1980102440 tx=0x7f1974005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:51.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.535+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f19700bc5c0 con 0x7f1980102d50 2024-09-12T03:27:51.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:51 smithi121 ceph-mon[34963]: pgmap v601: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-12T03:27:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:51 smithi019 ceph-mon[30953]: pgmap v601: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-12T03:27:51.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.754+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1980107220 con 0x7f1980102d50 2024-09-12T03:27:51.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.754+0000 7f196f7fe640 1 -- 172.21.15.19:0/676620887 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11989 (secure 0 0 0) 0x7f19700bbec0 con 0x7f1980102d50 2024-09-12T03:27:51.755 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:27:51.755 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":44,"btime":"2024-09-12T03:27:51:330488+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:27:50.644842+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:27:51.756 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 44 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 msgr2=0x7f19680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 0x7f19680a5040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1980102440 tx=0x7f1974005f90 comp rx=0 tx=0).stop 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 msgr2=0x7f19801009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f1970006010 tx=0x7f1970004040 comp rx=0 tx=0).stop 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 shutdown_connections 2024-09-12T03:27:51.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f198010c180 0x7f1980101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1980103670 0x7f1980100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f19680a2b80 0x7f19680a5040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 --2- 172.21.15.19:0/676620887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1980102d50 0x7f19801009b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 >> 172.21.15.19:0/676620887 conn(0x7f19800fc7d0 msgr2=0x7f19800fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 shutdown_connections 2024-09-12T03:27:51.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:51.757+0000 7f198497d640 1 -- 172.21.15.19:0/676620887 wait complete. 2024-09-12T03:27:51.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:51 smithi135 ceph-mon[35474]: pgmap v601: 97 pgs: 97 active+clean; 2.4 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 557 KiB/s wr, 96 op/s 2024-09-12T03:27:52.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:52 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:52.563 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:52 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/676620887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:52.592 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T03:27:52.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T03:27:52.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:52 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:52.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:52 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/676620887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:52.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:52 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:52.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:52 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/676620887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:27:52.924 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:27:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 -- 172.21.15.19:0/915860204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 msgr2=0x7f867c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 --2- 172.21.15.19:0/915860204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c10da40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8674009ca0 tx=0x7f867402a950 comp rx=0 tx=0).stop 2024-09-12T03:27:53.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 -- 172.21.15.19:0/915860204 shutdown_connections 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 --2- 172.21.15.19:0/915860204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c10da40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 --2- 172.21.15.19:0/915860204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c102570 0x7f867c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 --2- 172.21.15.19:0/915860204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c100e20 0x7f867c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.454+0000 7f86814fe640 1 -- 172.21.15.19:0/915860204 >> 172.21.15.19:0/915860204 conn(0x7f867c0fc7d0 msgr2=0x7f867c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 -- 172.21.15.19:0/915860204 shutdown_connections 2024-09-12T03:27:53.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 -- 172.21.15.19:0/915860204 wait complete. 2024-09-12T03:27:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 Processor -- start 2024-09-12T03:27:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 -- start start 2024-09-12T03:27:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 0x7f867c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.455+0000 7f86814fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c102570 0x7f867c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f86814fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c19cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f86814fe640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f867c110fd0 con 0x7f867c102ef0 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f86814fe640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f867c110e50 con 0x7f867c102570 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f86814fe640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f867c111150 con 0x7f867c100e20 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 0x7f867c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f8673fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c102570 0x7f867c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c19cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 0x7f867c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:48660/0 (socket says 172.21.15.19:48660) 2024-09-12T03:27:53.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867affd640 1 -- 172.21.15.19:0/887240096 learned_addr learned my addr 172.21.15.19:0/887240096 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 -- 172.21.15.19:0/887240096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 msgr2=0x7f867c198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 0x7f867c198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 -- 172.21.15.19:0/887240096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c102570 msgr2=0x7f867c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c102570 0x7f867c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 -- 172.21.15.19:0/887240096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867c19d410 con 0x7f867c102ef0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f867b7fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c19cd30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8674005e30 tx=0x7f8674003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f8678ff9640 1 -- 172.21.15.19:0/887240096 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674034040 con 0x7f867c102ef0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f8678ff9640 1 -- 172.21.15.19:0/887240096 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f867402f930 con 0x7f867c102ef0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867c19d6a0 con 0x7f867c102ef0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.456+0000 7f8678ff9640 1 -- 172.21.15.19:0/887240096 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86740045d0 con 0x7f867c102ef0 2024-09-12T03:27:53.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.457+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867c1a4fe0 con 0x7f867c102ef0 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.459+0000 7f8678ff9640 1 -- 172.21.15.19:0/887240096 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8674037020 con 0x7f867c102ef0 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.459+0000 7f86838d7740 1 Processor -- start 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.459+0000 7f86838d7740 1 -- start start 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281ce0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281ce0940 0x55b281ce2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b281c3b2a0 con 0x55b281cd5760 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b281be2b10 con 0x55b281ce0940 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f86838d7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b281c37260 con 0x55b281cd6080 2024-09-12T03:27:53.473 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36698/0 (socket says 172.21.15.19:36698) 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 -- 172.21.15.19:0/3662721668 learned_addr learned my addr 172.21.15.19:0/3662721668 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f867affd640 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281ce0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8678ff9640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 0x7f86680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281ce0940 0x55b281ce2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f867b7fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 0x7f86680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 -- 172.21.15.19:0/3662721668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 msgr2=0x55b281ce0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281ce0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 -- 172.21.15.19:0/3662721668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281ce0940 msgr2=0x55b281ce2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281ce0940 0x55b281ce2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 -- 172.21.15.19:0/3662721668 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2819acb90 con 0x55b281cd5760 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8678ff9640 1 -- 172.21.15.19:0/887240096 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86740efd00 con 0x7f867c102ef0 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.460+0000 7f8673fff640 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55b281cd3070 tx=0x7f8664009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaee76c46fb79ed3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f867b7fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 0x7f86680a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8674006010 tx=0x7f867408b490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f86717fa640 1 -- 172.21.15.19:0/3662721668 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664013130 con 0x55b281cd5760 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f86717fa640 1 -- 172.21.15.19:0/3662721668 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664013e50 con 0x55b281cd5760 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f86717fa640 1 -- 172.21.15.19:0/3662721668 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664004c60 con 0x55b281cd5760 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f86838d7740 1 -- 172.21.15.19:0/3662721668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 msgr2=0x55b281cd5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.461+0000 7f86838d7740 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x55b281cd3070 tx=0x7f8664009850 comp rx=0 tx=0).stop 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- 172.21.15.19:0/3662721668 shutdown_connections 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281ce0940 0x55b281ce2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281ce0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- 172.21.15.19:0/3662721668 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281cd5760 0x55b281cd5b40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- 172.21.15.19:0/3662721668 >> 172.21.15.19:0/3662721668 conn(0x55b281cd1cf0 msgr2=0x55b281cd20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- 172.21.15.19:0/3662721668 shutdown_connections 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- 172.21.15.19:0/3662721668 wait complete. 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- start start 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 Processor -- start 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b281ce3510 con 0x55b281ce0940 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b281bc36e0 con 0x55b281cd5760 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b281ce3690 con 0x55b281cd6080 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f86838d7740 10 client.0 _ll_register_callbacks cb 0x55b281be7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.462+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36712/0 (socket says 172.21.15.19:36712) 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f8673fff640 1 -- 172.21.15.19:0/3277570274 learned_addr learned my addr 172.21.15.19:0/3277570274 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f8673fff640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 -- 172.21.15.19:0/3277570274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 msgr2=0x55b281cdd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 -- 172.21.15.19:0/3277570274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 msgr2=0x55b281cde2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 -- 172.21.15.19:0/3277570274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b281ceca00 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f8673fff640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f867affd640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cddd70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f866c012760 tx=0x7f866c012c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73252e454f6eaef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f86627fc640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c004050 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f866c0041f0 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.463+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8664013e50 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f866c0043f0 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 10 client.26440 fetch_fsmap learned FSMap version 44 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b281cef440 con 0x55b281cd6080 2024-09-12T03:27:53.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664013e50 con 0x55b281cd6080 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f866c003c70 con 0x55b281cd6080 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 10 client.26440 fetch_fsmap finished waiting for FSMap version 44 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 10 client.26440 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 20 client.26440 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281cf09a0 0x55b281cf2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 4 client.26440 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b281c489d0 con 0x55b281cf09a0 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.464+0000 7f8673fff640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281cf09a0 0x55b281cf2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.465+0000 7f86627fc640 10 client.26440 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.465+0000 7f8673fff640 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281cf09a0 0x55b281cf2d90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a65d9acffdb59d server_cookie=85b5935710e470d8 in_seq=0 out_seq=0 2024-09-12T03:27:53.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86627fc640 1 -- 172.21.15.19:0/3277570274 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b281c489d0 con 0x55b281cf09a0 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86627fc640 10 client.26440 handle_command_reply: tid=0 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 client.26440 shutdown 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 msgr2=0x55b281cddd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cddd70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f866c012760 tx=0x7f866c012c20 comp rx=0 tx=0).stop 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 shutdown_connections 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cde2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cddd70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281cf09a0 0x55b281cf2d90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 --2- 172.21.15.19:0/3277570274 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 >> 172.21.15.19:0/3277570274 conn(0x55b281cd1cf0 msgr2=0x55b281cd79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86627fc640 0 client.26440 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 shutdown_connections 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 1 -- 172.21.15.19:0/3277570274 wait complete. 2024-09-12T03:27:53.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.466+0000 7f86838d7740 20 client.26440 trim_cache size 0 max 16384 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 Processor -- start 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 -- start start 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cde100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1ad3b0 con 0x55b281ce0940 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1aefe0 con 0x55b281cd5760 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f86838d7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1ada80 con 0x55b281cd6080 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36722/0 (socket says 172.21.15.19:36722) 2024-09-12T03:27:53.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 -- 172.21.15.19:0/3989575289 learned_addr learned my addr 172.21.15.19:0/3989575289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f867affd640 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 -- 172.21.15.19:0/3989575289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 msgr2=0x55b281cde100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cde100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 -- 172.21.15.19:0/3989575289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 msgr2=0x55b281cdf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.484+0000 7f8673fff640 1 -- 172.21.15.19:0/3989575289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b281cd5590 con 0x55b281ce0940 2024-09-12T03:27:53.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.485+0000 7f8673fff640 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8664004810 tx=0x7f866401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda1fe50b5faccb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.485+0000 7f8670ff9640 1 -- 172.21.15.19:0/3989575289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866400fb20 con 0x55b281ce0940 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.485+0000 7f8670ff9640 1 -- 172.21.15.19:0/3989575289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664013e50 con 0x55b281ce0940 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.485+0000 7f8670ff9640 1 -- 172.21.15.19:0/3989575289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664021480 con 0x55b281ce0940 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- 172.21.15.19:0/3989575289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 msgr2=0x55b281cf8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8664004810 tx=0x7f866401c040 comp rx=0 tx=0).stop 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- 172.21.15.19:0/3989575289 shutdown_connections 2024-09-12T03:27:53.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cf8660 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cde100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- 172.21.15.19:0/3989575289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- 172.21.15.19:0/3989575289 >> 172.21.15.19:0/3989575289 conn(0x55b281cd1cf0 msgr2=0x55b281cd0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- 172.21.15.19:0/3989575289 shutdown_connections 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- 172.21.15.19:0/3989575289 wait complete. 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- start start 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 Processor -- start 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cdaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1ae850 con 0x55b281ce0940 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1ae150 con 0x55b281cd5760 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f867c1ab390 con 0x55b281cd6080 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.486+0000 7f86838d7740 10 client.0 _ll_register_callbacks cb 0x7f866c009f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f867affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cdaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36732/0 (socket says 172.21.15.19:36732) 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 -- 172.21.15.19:0/3385513661 learned_addr learned my addr 172.21.15.19:0/3385513661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 -- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 msgr2=0x55b281cdaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cdaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 -- 172.21.15.19:0/3385513661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 msgr2=0x55b281cdaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 -- 172.21.15.19:0/3385513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b281ccd260 con 0x55b281ce0940 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f867affd640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cdaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f8673fff640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f86640047e0 tx=0x7f8664005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467da833ad66484a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f86627fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866400fd20 con 0x55b281ce0940 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8664021620 con 0x55b281ce0940 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8664013e50 con 0x55b281ce0940 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.487+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f8664021820 con 0x55b281ce0940 2024-09-12T03:27:53.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 10 client.17064 fetch_fsmap learned FSMap version 44 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55b281d5b8c0 con 0x55b281ce0940 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8664013e50 con 0x55b281ce0940 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 44) ==== 8398+0+0 (secure 0 0 0) 0x7f86640109d0 con 0x55b281ce0940 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 10 client.17064 fetch_fsmap finished waiting for FSMap version 44 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 10 client.17064 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 20 client.17064 populate_metadata read hostname 'smithi019' 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281d5bdb0 0x55b281d5e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 4 client.17064 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x55b281c42d20 con 0x55b281d5bdb0 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.488+0000 7f867affd640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281d5bdb0 0x55b281d5e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:27:53.494 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.489+0000 7f86627fc640 10 client.17064 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:53.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:53.489+0000 7f867affd640 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281d5bdb0 0x55b281d5e1d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ca130376280447 server_cookie=c0a4425c3d7398af in_seq=0 out_seq=0 2024-09-12T03:27:53.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:53 smithi121 ceph-mon[34963]: pgmap v602: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 588 KiB/s wr, 166 op/s 2024-09-12T03:27:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:53 smithi019 ceph-mon[30953]: pgmap v602: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 588 KiB/s wr, 166 op/s 2024-09-12T03:27:53.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:53 smithi135 ceph-mon[35474]: pgmap v602: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 588 KiB/s wr, 166 op/s 2024-09-12T03:27:55.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:55 smithi121 ceph-mon[34963]: pgmap v603: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 587 KiB/s wr, 198 op/s 2024-09-12T03:27:55.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:55 smithi019 ceph-mon[30953]: pgmap v603: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 587 KiB/s wr, 198 op/s 2024-09-12T03:27:55.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:55 smithi135 ceph-mon[35474]: pgmap v603: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 587 KiB/s wr, 198 op/s 2024-09-12T03:27:56.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:56 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:56.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:56 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:56.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:56 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:27:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:57 smithi121 ceph-mon[34963]: pgmap v604: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-12T03:27:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:57 smithi019 ceph-mon[30953]: pgmap v604: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-12T03:27:57.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:57 smithi135 ceph-mon[35474]: pgmap v604: 97 pgs: 97 active+clean; 2.4 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 263 op/s 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86627fc640 1 -- 172.21.15.19:0/3385513661 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b281c42d20 con 0x55b281d5bdb0 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86627fc640 10 client.17064 handle_command_reply: tid=0 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 client.17064 shutdown 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 msgr2=0x55b281cdb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f86640047e0 tx=0x7f8664005da0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 shutdown_connections 2024-09-12T03:27:59.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b281ce0940 0x55b281cdb480 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b281cd6080 0x55b281cdaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b281d5bdb0 0x55b281d5e1d0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 --2- 172.21.15.19:0/3385513661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b281cd5760 0x55b281cdaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86627fc640 0 client.17064 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 >> 172.21.15.19:0/3385513661 conn(0x55b281cd1cf0 msgr2=0x55b281cd71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 shutdown_connections 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.170+0000 7f86838d7740 1 -- 172.21.15.19:0/3385513661 wait complete. 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.171+0000 7f86838d7740 20 client.17064 trim_cache size 0 max 16384 2024-09-12T03:27:59.172 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 45, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 30, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "9e9af05e": { 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 30, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:27:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 139.4, 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:27:59.174 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.171+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 msgr2=0x7f86680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.171+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 0x7f86680a5040 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8674006010 tx=0x7f867408b490 comp rx=0 tx=0).stop 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.171+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 msgr2=0x7f867c19cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.171+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c19cd30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8674005e30 tx=0x7f8674003b80 comp rx=0 tx=0).stop 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 shutdown_connections 2024-09-12T03:27:59.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86680a2b80 0x7f86680a5040 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f867c102ef0 0x7f867c19cd30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f867c102570 0x7f867c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 --2- 172.21.15.19:0/887240096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f867c100e20 0x7f867c198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 >> 172.21.15.19:0/887240096 conn(0x7f867c0fc7d0 msgr2=0x7f867c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 shutdown_connections 2024-09-12T03:27:59.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:27:59.172+0000 7f86814fe640 1 -- 172.21.15.19:0/887240096 wait complete. 2024-09-12T03:27:59.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:27:59 smithi019 ceph-mon[30953]: pgmap v605: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-09-12T03:27:59.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:27:59 smithi121 ceph-mon[34963]: pgmap v605: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-09-12T03:27:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:27:59 smithi135 ceph-mon[35474]: pgmap v605: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 304 op/s 2024-09-12T03:27:59.869 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9e9af05e', quiesce took 9.2 seconds. Will release after: 10.2 2024-09-12T03:28:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:01 smithi121 ceph-mon[34963]: pgmap v606: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 946 KiB/s wr, 289 op/s 2024-09-12T03:28:01.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:01 smithi019 ceph-mon[30953]: pgmap v606: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 946 KiB/s wr, 289 op/s 2024-09-12T03:28:01.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:01 smithi135 ceph-mon[35474]: pgmap v606: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 946 KiB/s wr, 289 op/s 2024-09-12T03:28:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:03 smithi121 ceph-mon[34963]: pgmap v607: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 602 KiB/s wr, 274 op/s 2024-09-12T03:28:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:03 smithi019 ceph-mon[30953]: pgmap v607: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 602 KiB/s wr, 274 op/s 2024-09-12T03:28:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:03 smithi135 ceph-mon[35474]: pgmap v607: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 602 KiB/s wr, 274 op/s 2024-09-12T03:28:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:05 smithi121 ceph-mon[34963]: pgmap v608: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 569 KiB/s wr, 200 op/s 2024-09-12T03:28:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:05 smithi019 ceph-mon[30953]: pgmap v608: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 569 KiB/s wr, 200 op/s 2024-09-12T03:28:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:05 smithi135 ceph-mon[35474]: pgmap v608: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 569 KiB/s wr, 200 op/s 2024-09-12T03:28:07.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:07 smithi121 ceph-mon[34963]: pgmap v609: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 571 KiB/s wr, 157 op/s 2024-09-12T03:28:07.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:07 smithi019 ceph-mon[30953]: pgmap v609: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 571 KiB/s wr, 157 op/s 2024-09-12T03:28:07.757 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:28:07.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:07 smithi135 ceph-mon[35474]: pgmap v609: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 571 KiB/s wr, 157 op/s 2024-09-12T03:28:08.031 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 -- 172.21.15.19:0/2823275713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 msgr2=0x7fdd54107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/2823275713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54107b80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009930 tx=0x7fdd4402a420 comp rx=0 tx=0).stop 2024-09-12T03:28:08.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 -- 172.21.15.19:0/2823275713 shutdown_connections 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/2823275713 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54108290 0x7fdd5410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/2823275713 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54107b80 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/2823275713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54102d90 0x7fdd54103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 -- 172.21.15.19:0/2823275713 >> 172.21.15.19:0/2823275713 conn(0x7fdd540fc7d0 msgr2=0x7fdd540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 -- 172.21.15.19:0/2823275713 shutdown_connections 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.611+0000 7fdd5bd14640 1 -- 172.21.15.19:0/2823275713 wait complete. 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 Processor -- start 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 -- start start 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54102d90 0x7fdd541985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 0x7fdd5419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54112050 con 0x7fdd54103740 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd54111ed0 con 0x7fdd54108290 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd5bd14640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541121d0 con 0x7fdd54102d90 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.612+0000 7fdd592af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43100/0 (socket says 172.21.15.19:43100) 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 -- 172.21.15.19:0/222258077 learned_addr learned my addr 172.21.15.19:0/222258077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54102d90 0x7fdd541985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 0x7fdd5419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 -- 172.21.15.19:0/222258077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54102d90 msgr2=0x7fdd541985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54102d90 0x7fdd541985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 -- 172.21.15.19:0/222258077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 msgr2=0x7fdd5419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.621 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 0x7fdd5419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 -- 172.21.15.19:0/222258077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5419d5a0 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 0x7fdd5419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd592af640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005940 tx=0x7fdd440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd4affd640 1 -- 172.21.15.19:0/222258077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4403f030 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd4affd640 1 -- 172.21.15.19:0/222258077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd44003ce0 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5419d830 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd4affd640 1 -- 172.21.15.19:0/222258077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44035070 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.613+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541a5170 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.616+0000 7fdd5e0ed740 1 Processor -- start 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.616+0000 7fdd4affd640 1 -- 172.21.15.19:0/222258077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdd44042030 con 0x7fdd54103740 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.616+0000 7fdd5e0ed740 1 -- start start 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.616+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.616+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200285dc0 0x556200290140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200290680 0x556200292a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5562001eb2e0 con 0x5562002854a0 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x556200192ce0 con 0x556200285dc0 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5562001e72a0 con 0x556200290680 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43104/0 (socket says 172.21.15.19:43104) 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/974076691 learned_addr learned my addr 172.21.15.19:0/974076691 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200285dc0 0x556200290140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd592af640 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200290680 0x556200292a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/974076691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200290680 msgr2=0x556200292a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200290680 0x556200292a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/974076691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200285dc0 msgr2=0x556200290140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200285dc0 0x556200290140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd4affd640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 0x7fdd400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/974076691 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556200177440 con 0x5562002854a0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd592af640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 0x7fdd400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd4affd640 1 -- 172.21.15.19:0/222258077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdd440ef1d0 con 0x7fdd54103740 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.617+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ae80 tx=0x7fdd4c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f8312e3cee584 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.618+0000 7fdd3ffff640 1 -- 172.21.15.19:0/974076691 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003b70 con 0x5562002854a0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.618+0000 7fdd3ffff640 1 -- 172.21.15.19:0/974076691 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c003d10 con 0x5562002854a0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.618+0000 7fdd3ffff640 1 -- 172.21.15.19:0/974076691 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c010b00 con 0x5562002854a0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/974076691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 msgr2=0x556200285880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd592af640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 0x7fdd400a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdd440ed050 tx=0x7fdd4402a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ae80 tx=0x7fdd4c00e720 comp rx=0 tx=0).stop 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/974076691 shutdown_connections 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200290680 0x556200292a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200285dc0 0x556200290140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/974076691 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200285880 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/974076691 >> 172.21.15.19:0/974076691 conn(0x556200281a30 msgr2=0x556200281e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/974076691 shutdown_connections 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/974076691 wait complete. 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 -- start start 2024-09-12T03:28:08.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.619+0000 7fdd5e0ed740 1 Processor -- start 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620028d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620028d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x556200297660 con 0x5562002854a0 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5562001a6050 con 0x556200290680 2024-09-12T03:28:08.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5562002977e0 con 0x556200285dc0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5e0ed740 10 client.0 _ll_register_callbacks cb 0x5562001e6160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43108/0 (socket says 172.21.15.19:43108) 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/2555766463 learned_addr learned my addr 172.21.15.19:0/2555766463 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620028d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 msgr2=0x55620028d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620028d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/2555766463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 msgr2=0x55620028d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620028d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.620+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/2555766463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561fff5c3c0 con 0x5562002854a0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c004920 tx=0x7fdd4c010ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e3f5810e8861a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:08.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c025030 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdd4c004a40 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd4c003d10 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd4c033050 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd5e0ed740 10 client.17076 fetch_fsmap learned FSMap version 45 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5562002a3050 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd4c003d10 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.621+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 45) ==== 8402+0+0 (secure 0 0 0) 0x7fdd4c01a8b0 con 0x5562002854a0 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 10 client.17076 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 20 client.17076 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5562002a4460 0x5562002a6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 4 client.17076 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:08.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562001f8850 con 0x5562002a4460 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5562002a4460 0x5562002a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5562002a4460 0x5562002a6850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e597a99479a1c011 server_cookie=5e102293725ce038 in_seq=0 out_seq=0 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.622+0000 7fdd3d7fa640 10 client.17076 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/2555766463 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562001f8850 con 0x5562002a4460 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd3d7fa640 10 client.17076 handle_command_reply: tid=0 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 client.17076 shutdown 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 msgr2=0x556200290010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c004920 tx=0x7fdd4c010ab0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 shutdown_connections 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5562002a4460 0x5562002a6850 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620028d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620028d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/2555766463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x556200290010 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd3d7fa640 0 client.17076 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.623+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 >> 172.21.15.19:0/2555766463 conn(0x556200281a30 msgr2=0x556200287760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.624+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 shutdown_connections 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.624+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/2555766463 wait complete. 2024-09-12T03:28:08.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.624+0000 7fdd5e0ed740 20 client.17076 trim_cache size 0 max 16384 2024-09-12T03:28:08.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 Processor -- start 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 -- start start 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562002854a0 0x55620028cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x5562002abf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ab470 con 0x556200285dc0 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ad0d0 con 0x556200290680 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541abb40 con 0x5562002854a0 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43118/0 (socket says 172.21.15.19:43118) 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 -- 172.21.15.19:0/3142970104 learned_addr learned my addr 172.21.15.19:0/3142970104 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 -- 172.21.15.19:0/3142970104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562002854a0 msgr2=0x55620028cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562002854a0 0x55620028cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x5562002abf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 -- 172.21.15.19:0/3142970104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 msgr2=0x5562002abf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x5562002abf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.641+0000 7fdd59ab0640 1 -- 172.21.15.19:0/3142970104 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620028fb30 con 0x556200285dc0 2024-09-12T03:28:08.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.642+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd50011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cb1cee7a86c04d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.642+0000 7fdd3f7fe640 1 -- 172.21.15.19:0/3142970104 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004020 con 0x556200285dc0 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.642+0000 7fdd3f7fe640 1 -- 172.21.15.19:0/3142970104 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd50005500 con 0x556200285dc0 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.642+0000 7fdd3f7fe640 1 -- 172.21.15.19:0/3142970104 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004d00 con 0x556200285dc0 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3142970104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 msgr2=0x556200280a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fdd50000c00 tx=0x7fdd50011810 comp rx=0 tx=0).stop 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3142970104 shutdown_connections 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x5562002abf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x556200285dc0 0x556200280a80 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3142970104 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5562002854a0 0x55620028cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3142970104 >> 172.21.15.19:0/3142970104 conn(0x556200281a30 msgr2=0x556200280370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3142970104 shutdown_connections 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3142970104 wait complete. 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 -- start start 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.643+0000 7fdd5e0ed740 1 Processor -- start 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620029a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620029a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ac940 con 0x5562002854a0 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541ac210 con 0x556200290680 2024-09-12T03:28:08.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdd541a9470 con 0x556200285dc0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 10 client.0 _ll_register_callbacks cb 0x7fdd4c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43130/0 (socket says 172.21.15.19:43130) 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/3277540807 learned_addr learned my addr 172.21.15.19:0/3277540807 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620029a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620029a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 msgr2=0x55620029a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620029a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/3277540807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 msgr2=0x55620029a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620029a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 -- 172.21.15.19:0/3277540807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620029b1c0 con 0x5562002854a0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5a2b1640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00f010 tx=0x7fdd4c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc17a46bb8124b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:08.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c025040 con 0x5562002854a0 2024-09-12T03:28:08.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdd4c004a00 con 0x5562002854a0 2024-09-12T03:28:08.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.644+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd50005500 con 0x5562002854a0 2024-09-12T03:28:08.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdd4c00acd0 con 0x5562002854a0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 10 client.17082 fetch_fsmap learned FSMap version 45 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55620030ee90 con 0x5562002854a0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd50005500 con 0x5562002854a0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 45) ==== 8402+0+0 (secure 0 0 0) 0x7fdd4c042020 con 0x5562002854a0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 10 client.17082 fetch_fsmap finished waiting for FSMap version 45 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 10 client.17082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 20 client.17082 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55620030f5e0 0x556200311a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 4 client.17082 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.645+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562001f2d60 con 0x55620030f5e0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55620030f5e0 0x556200311a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd59ab0640 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55620030f5e0 0x556200311a00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f8c1edf128324e server_cookie=22db56bfd0cbb8e8 in_seq=0 out_seq=0 2024-09-12T03:28:08.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd3d7fa640 10 client.17082 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:08.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd3d7fa640 1 -- 172.21.15.19:0/3277540807 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562001f2d60 con 0x55620030f5e0 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd3d7fa640 10 client.17082 handle_command_reply: tid=0 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd5e0ed740 1 client.17082 shutdown 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 msgr2=0x55620029d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.646+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00f010 tx=0x7fdd4c00e720 comp rx=0 tx=0).stop 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 shutdown_connections 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55620030f5e0 0x556200311a00 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x556200290680 0x55620029a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x556200285dc0 0x55620029a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 --2- 172.21.15.19:0/3277540807 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5562002854a0 0x55620029d100 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd3d7fa640 0 client.17082 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 >> 172.21.15.19:0/3277540807 conn(0x556200281a30 msgr2=0x55620028eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 shutdown_connections 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 1 -- 172.21.15.19:0/3277540807 wait complete. 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5e0ed740 20 client.17082 trim_cache size 0 max 16384 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:28:08.656 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 msgr2=0x7fdd400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 0x7fdd400a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdd440ed050 tx=0x7fdd4402a420 comp rx=0 tx=0).stop 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 msgr2=0x7fdd54198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005940 tx=0x7fdd440042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 shutdown_connections 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdd54108290 0x7fdd5419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdd400a2c00 0x7fdd400a50c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdd54103740 0x7fdd54198b30 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 --2- 172.21.15.19:0/222258077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdd54102d90 0x7fdd541985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.647+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 >> 172.21.15.19:0/222258077 conn(0x7fdd540fc7d0 msgr2=0x7fdd54105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.648+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 shutdown_connections 2024-09-12T03:28:08.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:08.648+0000 7fdd5bd14640 1 -- 172.21.15.19:0/222258077 wait complete. 2024-09-12T03:28:09.353 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:28:09.353 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:09.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:09.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:09 smithi121 ceph-mon[34963]: pgmap v610: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 89 op/s 2024-09-12T03:28:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:09 smithi019 ceph-mon[30953]: pgmap v610: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 89 op/s 2024-09-12T03:28:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:09 smithi135 ceph-mon[35474]: pgmap v610: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 89 op/s 2024-09-12T03:28:10.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.212+0000 7f20d97ff640 1 -- 172.21.15.19:0/1981116126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 msgr2=0x7f20d410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.212+0000 7f20d97ff640 1 --2- 172.21.15.19:0/1981116126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d410e610 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009ca0 tx=0x7f20cc02a950 comp rx=0 tx=0).stop 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.212+0000 7f20d97ff640 1 -- 172.21.15.19:0/1981116126 shutdown_connections 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 --2- 172.21.15.19:0/1981116126 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d410e610 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 --2- 172.21.15.19:0/1981116126 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 0x7f20d410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 --2- 172.21.15.19:0/1981116126 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 -- 172.21.15.19:0/1981116126 >> 172.21.15.19:0/1981116126 conn(0x7f20d40fc7a0 msgr2=0x7f20d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:10.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 -- 172.21.15.19:0/1981116126 shutdown_connections 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 -- 172.21.15.19:0/1981116126 wait complete. 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.213+0000 7f20d97ff640 1 Processor -- start 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 -- start start 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 0x7f20d4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4111fc0 con 0x7f20d410c170 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4111e40 con 0x7f20d4102d40 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d97ff640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20d4112140 con 0x7f20d4103660 2024-09-12T03:28:10.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:43150/0 (socket says 172.21.15.19:43150) 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 -- 172.21.15.19:0/2964273696 learned_addr learned my addr 172.21.15.19:0/2964273696 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d27fc640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 0x7f20d4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d2ffd640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 -- 172.21.15.19:0/2964273696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 msgr2=0x7f20d4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 0x7f20d4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 -- 172.21.15.19:0/2964273696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 msgr2=0x7f20d41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d37fe640 1 -- 172.21.15.19:0/2964273696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d419d390 con 0x7f20d410c170 2024-09-12T03:28:10.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.214+0000 7f20d2ffd640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20d37fe640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009dc0 tx=0x7f20cc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc034040 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc02fe20 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d419d620 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc0047f0 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.215+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d41a4f60 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.216+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d406b070 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.217+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20cc037020 con 0x7f20d410c170 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.218+0000 7f20bbfff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 0x7f20bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:10.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.218+0000 7f20d2ffd640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 0x7f20bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:10.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.218+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f20cc0effa0 con 0x7f20d410c170 2024-09-12T03:28:10.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.219+0000 7f20d2ffd640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 0x7f20bc0a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20c40058c0 tx=0x7f20c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:10.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.222+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20cc0bceb0 con 0x7f20d410c170 2024-09-12T03:28:10.434 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:10.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.465+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20d41071e0 con 0x7f20d410c170 2024-09-12T03:28:10.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.466+0000 7f20bbfff640 1 -- 172.21.15.19:0/2964273696 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11989 (secure 0 0 0) 0x7f20cc0bc7b0 con 0x7f20d410c170 2024-09-12T03:28:10.466 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:28:10.467 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":46,"btime":"2024-09-12T03:28:09:392224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:28:09.254791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:28:10.467 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 46 2024-09-12T03:28:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 msgr2=0x7f20bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 0x7f20bc0a5040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20c40058c0 tx=0x7f20c4005850 comp rx=0 tx=0).stop 2024-09-12T03:28:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 msgr2=0x7f20d419ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009dc0 tx=0x7f20cc003f10 comp rx=0 tx=0).stop 2024-09-12T03:28:10.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 shutdown_connections 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20bc0a2b80 0x7f20bc0a5040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f20d410c170 0x7f20d419ccb0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f20d4103660 0x7f20d4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 --2- 172.21.15.19:0/2964273696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f20d4102d40 0x7f20d41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 >> 172.21.15.19:0/2964273696 conn(0x7f20d40fc7a0 msgr2=0x7f20d40fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 shutdown_connections 2024-09-12T03:28:10.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:10.469+0000 7f20d97ff640 1 -- 172.21.15.19:0/2964273696 wait complete. 2024-09-12T03:28:10.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:10 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:10.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:10 smithi019 ceph-mon[30953]: pgmap v611: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 45 op/s 2024-09-12T03:28:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:10 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:10.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:10 smithi135 ceph-mon[35474]: pgmap v611: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 45 op/s 2024-09-12T03:28:10.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:10 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:10.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:10 smithi121 ceph-mon[34963]: pgmap v611: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 49 KiB/s wr, 45 op/s 2024-09-12T03:28:11.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:28:11.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 -- 172.21.15.19:0/3316891137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102570 msgr2=0x7fd84c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:11.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 --2- 172.21.15.19:0/3316891137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102570 0x7fd84c1029b0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd83c009900 tx=0x7fd83c02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:11.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 -- 172.21.15.19:0/3316891137 shutdown_connections 2024-09-12T03:28:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 --2- 172.21.15.19:0/3316891137 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102ef0 0x7fd84c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 --2- 172.21.15.19:0/3316891137 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102570 0x7fd84c1029b0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 --2- 172.21.15.19:0/3316891137 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 0x7fd84c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.529+0000 7fd852eea640 1 -- 172.21.15.19:0/3316891137 >> 172.21.15.19:0/3316891137 conn(0x7fd84c0fc7d0 msgr2=0x7fd84c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.530+0000 7fd852eea640 1 -- 172.21.15.19:0/3316891137 shutdown_connections 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.530+0000 7fd852eea640 1 -- 172.21.15.19:0/3316891137 wait complete. 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.530+0000 7fd852eea640 1 Processor -- start 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.530+0000 7fd852eea640 1 -- start start 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 0x7fd84c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 0x7fd84c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c110fd0 con 0x7fd84c102ef0 2024-09-12T03:28:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c110e50 con 0x7fd84c100e20 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd852eea640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fd84c111150 con 0x7fd84c102570 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42608/0 (socket says 172.21.15.19:42608) 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 -- 172.21.15.19:0/2510515646 learned_addr learned my addr 172.21.15.19:0/2510515646 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd850c86640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 0x7fd84c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:11.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd843fff640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 0x7fd84c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 -- 172.21.15.19:0/2510515646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 msgr2=0x7fd84c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 0x7fd84c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 -- 172.21.15.19:0/2510515646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 msgr2=0x7fd84c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 0x7fd84c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 -- 172.21.15.19:0/2510515646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84c1a6a00 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd843fff640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 0x7fd84c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.531+0000 7fd851487640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd84400eb10 tx=0x7fd84400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.532+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844004b20 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.532+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84c1a6cf0 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.532+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd844004cc0 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.532+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd844010760 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.533+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84c1a7250 con 0x7fd84c102ef0 2024-09-12T03:28:11.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.534+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84c101200 con 0x7fd84c102ef0 2024-09-12T03:28:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.534+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd844005000 con 0x7fd84c102ef0 2024-09-12T03:28:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.535+0000 7fd841ffb640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 0x7fd8340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:11.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.535+0000 7fd850c86640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 0x7fd8340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.535+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fd8440cf5a0 con 0x7fd84c102ef0 2024-09-12T03:28:11.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.536+0000 7fd850c86640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 0x7fd8340a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd84800bbe0 tx=0x7fd84800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:11.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.540+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd84409c510 con 0x7fd84c102ef0 2024-09-12T03:28:11.614 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:11.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:11 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2964273696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:11.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.763+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd84c10a0a0 con 0x7fd84c102ef0 2024-09-12T03:28:11.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.764+0000 7fd841ffb640 1 -- 172.21.15.19:0/2510515646 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+11989 (secure 0 0 0) 0x7fd84409be10 con 0x7fd84c102ef0 2024-09-12T03:28:11.764 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:28:11.765 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":46,"btime":"2024-09-12T03:28:09:392224+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":46,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:28:09.254791+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:28:11.765 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 46 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 msgr2=0x7fd8340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 0x7fd8340a5040 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd84800bbe0 tx=0x7fd84800b390 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 msgr2=0x7fd84c0fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd84400eb10 tx=0x7fd84400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 shutdown_connections 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fd8340a2b80 0x7fd8340a5040 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fd84c102ef0 0x7fd84c0fd700 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fd84c102570 0x7fd84c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 --2- 172.21.15.19:0/2510515646 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd84c100e20 0x7fd84c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 >> 172.21.15.19:0/2510515646 conn(0x7fd84c0fc7d0 msgr2=0x7fd84c06a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:11.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 shutdown_connections 2024-09-12T03:28:11.768 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:11.766+0000 7fd852eea640 1 -- 172.21.15.19:0/2510515646 wait complete. 2024-09-12T03:28:11.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:11 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2964273696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:11.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:11 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2964273696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:12.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:12 smithi019 ceph-mon[30953]: pgmap v612: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 35 op/s 2024-09-12T03:28:12.638 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:12 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2510515646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:12.669 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 9e9af05e --release --await' 2024-09-12T03:28:12.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 9e9af05e --release --await 2024-09-12T03:28:12.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:12 smithi135 ceph-mon[35474]: pgmap v612: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 35 op/s 2024-09-12T03:28:12.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:12 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2510515646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:12.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 -- 172.21.15.19:0/2114708181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa208103670 msgr2=0x7fa20810bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2114708181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa208103670 0x7fa20810bb60 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009900 tx=0x7fa1fc02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 -- 172.21.15.19:0/2114708181 shutdown_connections 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2114708181 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa20810c180 0x7fa20810e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2114708181 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa208103670 0x7fa20810bb60 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2114708181 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 0x7fa208103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.873+0000 7fa20cf97640 1 -- 172.21.15.19:0/2114708181 >> 172.21.15.19:0/2114708181 conn(0x7fa2080fc7d0 msgr2=0x7fa2080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.874+0000 7fa20cf97640 1 -- 172.21.15.19:0/2114708181 shutdown_connections 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.874+0000 7fa20cf97640 1 -- 172.21.15.19:0/2114708181 wait complete. 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.874+0000 7fa20cf97640 1 Processor -- start 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.874+0000 7fa20cf97640 1 -- start start 2024-09-12T03:28:12.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 0x7fa2081009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa208103670 0x7fa208100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa208111f80 con 0x7fa20810c180 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa208111e00 con 0x7fa208103670 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20cf97640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa208112100 con 0x7fa208102d50 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42614/0 (socket says 172.21.15.19:42614) 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa208103670 0x7fa208100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 -- 172.21.15.19:0/2022429232 learned_addr learned my addr 172.21.15.19:0/2022429232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa20659c640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 0x7fa2081009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 -- 172.21.15.19:0/2022429232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 msgr2=0x7fa2081009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 0x7fa2081009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 -- 172.21.15.19:0/2022429232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa208103670 msgr2=0x7fa208100ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa208103670 0x7fa208100ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 -- 172.21.15.19:0/2022429232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2081a6900 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.875+0000 7fa206d9d640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa20000cad0 tx=0x7fa20000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.876+0000 7fa1f77fe640 1 -- 172.21.15.19:0/2022429232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20000ae20 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.876+0000 7fa1f77fe640 1 -- 172.21.15.19:0/2022429232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa200005020 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.876+0000 7fa1f77fe640 1 -- 172.21.15.19:0/2022429232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20000f6f0 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.876+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2081a6bf0 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.876+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2081a70b0 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.878+0000 7fa1f77fe640 1 -- 172.21.15.19:0/2022429232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa2000051d0 con 0x7fa20810c180 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.878+0000 7fa20f370740 1 Processor -- start 2024-09-12T03:28:12.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.878+0000 7fa20f370740 1 -- start start 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51397ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b513988760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b5138e12f0 con 0x55b51397cb40 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b513888cf0 con 0x55b51397b5d0 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20f370740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b5138dd2b0 con 0x55b513986370 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42628/0 (socket says 172.21.15.19:42628) 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 -- 172.21.15.19:0/4048784008 learned_addr learned my addr 172.21.15.19:0/4048784008 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa20659c640 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51397ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.893 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa1f77fe640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 -- 172.21.15.19:0/4048784008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 msgr2=0x55b513988760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b513988760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 -- 172.21.15.19:0/4048784008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 msgr2=0x55b51397ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51397ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 -- 172.21.15.19:0/4048784008 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b51386d3a0 con 0x55b51397cb40 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa1f77fe640 1 -- 172.21.15.19:0/2022429232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa2000cf020 con 0x7fa20810c180 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa206d9d640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.879+0000 7fa205d9b640 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc02a9a0 tx=0x7fa1fc0315b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd6edc211cd0a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa1f4ff9640 1 -- 172.21.15.19:0/4048784008 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc040070 con 0x55b51397cb40 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa1f4ff9640 1 -- 172.21.15.19:0/4048784008 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc005020 con 0x55b51397cb40 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa1f4ff9640 1 -- 172.21.15.19:0/4048784008 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc035040 con 0x55b51397cb40 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 -- 172.21.15.19:0/4048784008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 msgr2=0x55b51397b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.894 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc02a9a0 tx=0x7fa1fc0315b0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa206d9d640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 0x7fa1f00a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa20000a6e0 tx=0x7fa200005b40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 -- 172.21.15.19:0/4048784008 shutdown_connections 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b513988760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51397ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 --2- 172.21.15.19:0/4048784008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51397b090 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.880+0000 7fa20f370740 1 -- 172.21.15.19:0/4048784008 >> 172.21.15.19:0/4048784008 conn(0x55b513944310 msgr2=0x55b513977c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- 172.21.15.19:0/4048784008 shutdown_connections 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- 172.21.15.19:0/4048784008 wait complete. 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- start start 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 Processor -- start 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397b5d0 0x55b513985d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 0x55b513982fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.895 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b513989010 con 0x55b513986370 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b51389c060 con 0x55b51397cb40 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b513989190 con 0x55b51397b5d0 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.881+0000 7fa20f370740 10 client.0 _ll_register_callbacks cb 0x55b5135ef8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42640/0 (socket says 172.21.15.19:42640) 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 -- 172.21.15.19:0/3379496590 learned_addr learned my addr 172.21.15.19:0/3379496590 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397b5d0 0x55b513985d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa20659c640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 0x55b513982fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 -- 172.21.15.19:0/3379496590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397b5d0 msgr2=0x55b513985d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.896 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397b5d0 0x55b513985d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 -- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 msgr2=0x55b513982fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 0x55b513982fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 -- 172.21.15.19:0/3379496590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5136523c0 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa20659c640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 0x55b513982fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc031b60 tx=0x7fa1fc02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b3fb6aa9b0d40f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa1edffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc035070 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc005020 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1fc004750 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc040040 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa20f370740 10 client.17106 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.882+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b513994a90 con 0x55b513986370 2024-09-12T03:28:12.897 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc005020 con 0x55b513986370 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7fa1fc056980 con 0x55b513986370 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 10 client.17106 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 10 client.17106 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 20 client.17106 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513996140 0x55b513998560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 4 client.17106 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5138ee860 con 0x55b513996140 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.883+0000 7fa20659c640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513996140 0x55b513998560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa1edffb640 10 client.17106 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20659c640 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513996140 0x55b513998560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2bdea1041ac04774 server_cookie=942b064958a9d146 in_seq=0 out_seq=0 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa1edffb640 1 -- 172.21.15.19:0/3379496590 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b5138ee860 con 0x55b513996140 2024-09-12T03:28:12.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa1edffb640 10 client.17106 handle_command_reply: tid=0 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 client.17106 shutdown 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 msgr2=0x55b513983560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc031b60 tx=0x7fa1fc02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 shutdown_connections 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b513986370 0x55b513983560 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397cb40 0x55b513982fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.884+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513996140 0x55b513998560 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa20f370740 1 --2- 172.21.15.19:0/3379496590 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397b5d0 0x55b513985d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa1edffb640 0 client.17106 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 >> 172.21.15.19:0/3379496590 conn(0x55b513944310 msgr2=0x55b51397d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 shutdown_connections 2024-09-12T03:28:12.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa20f370740 1 -- 172.21.15.19:0/3379496590 wait complete. 2024-09-12T03:28:12.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.885+0000 7fa20f370740 20 client.17106 trim_cache size 0 max 16384 2024-09-12T03:28:12.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:12 smithi121 ceph-mon[34963]: pgmap v612: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 10 KiB/s wr, 35 op/s 2024-09-12T03:28:12.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:12 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2510515646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.900+0000 7fa20f370740 1 Processor -- start 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.900+0000 7fa20f370740 1 -- start start 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397cb40 0x55b51397e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b513986370 0x55b51397e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081af6d0 con 0x55b51397b5d0 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081b1330 con 0x55b513986370 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20f370740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081afda0 con 0x55b51397cb40 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42652/0 (socket says 172.21.15.19:42652) 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 -- 172.21.15.19:0/1073408100 learned_addr learned my addr 172.21.15.19:0/1073408100 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa20659c640 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397cb40 0x55b51397e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b513986370 0x55b51397e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 -- 172.21.15.19:0/1073408100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397cb40 msgr2=0x55b51397e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397cb40 0x55b51397e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 -- 172.21.15.19:0/1073408100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b513986370 msgr2=0x55b51397e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b513986370 0x55b51397e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 -- 172.21.15.19:0/1073408100 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b513998bc0 con 0x55b51397b5d0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa205d9b640 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc047040 tx=0x7fa1fc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99170d530c7e809 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.901+0000 7fa1effff640 1 -- 172.21.15.19:0/1073408100 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc035040 con 0x55b51397b5d0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa1effff640 1 -- 172.21.15.19:0/1073408100 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc0315b0 con 0x55b51397b5d0 2024-09-12T03:28:12.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa1effff640 1 -- 172.21.15.19:0/1073408100 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0566c0 con 0x55b51397b5d0 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 -- 172.21.15.19:0/1073408100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 msgr2=0x55b5139922e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc047040 tx=0x7fa1fc031400 comp rx=0 tx=0).stop 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 -- 172.21.15.19:0/1073408100 shutdown_connections 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b513986370 0x55b51397e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b51397cb40 0x55b51397e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 --2- 172.21.15.19:0/1073408100 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397b5d0 0x55b5139922e0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 -- 172.21.15.19:0/1073408100 >> 172.21.15.19:0/1073408100 conn(0x55b513944310 msgr2=0x55b513982f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 -- 172.21.15.19:0/1073408100 shutdown_connections 2024-09-12T03:28:12.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.902+0000 7fa20f370740 1 -- 172.21.15.19:0/1073408100 wait complete. 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 -- start start 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 Processor -- start 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51398e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b51397f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081b0ba0 con 0x55b51397cb40 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081b0470 con 0x55b51397b5d0 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa2081ad6b0 con 0x55b513986370 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20f370740 10 client.0 _ll_register_callbacks cb 0x7fa1f800b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42656/0 (socket says 172.21.15.19:42656) 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa20659c640 1 -- 172.21.15.19:0/3704781045 learned_addr learned my addr 172.21.15.19:0/3704781045 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:12.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.903+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b51397f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51398e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 -- 172.21.15.19:0/3704781045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 msgr2=0x55b51397f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b51397f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 -- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 msgr2=0x55b51398e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51398e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 -- 172.21.15.19:0/3704781045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b51397fdc0 con 0x55b51397cb40 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51398e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20659c640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8000c00 tx=0x7fa1f801d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5ffc4f25df748b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa1edffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8004220 con 0x55b51397cb40 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa1f80043c0 con 0x55b51397cb40 2024-09-12T03:28:12.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc0315b0 con 0x55b51397cb40 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fa1f8011830 con 0x55b51397cb40 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20f370740 10 client.17112 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.904+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55b513a00740 con 0x55b51397cb40 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc0315b0 con 0x55b51397cb40 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7fa1f8025400 con 0x55b51397cb40 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 10 client.17112 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 10 client.17112 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 20 client.17112 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513a00f80 0x55b513a033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 4 client.17112 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b5138e8d70 con 0x55b513a00f80 2024-09-12T03:28:12.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.905+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513a00f80 0x55b513a033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa205d9b640 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513a00f80 0x55b513a033a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4cf49a0169b9307 server_cookie=91b3f59334b1967d in_seq=0 out_seq=0 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa1edffb640 10 client.17112 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa1edffb640 1 -- 172.21.15.19:0/3704781045 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b5138e8d70 con 0x55b513a00f80 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa1edffb640 10 client.17112 handle_command_reply: tid=0 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 client.17112 shutdown 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 msgr2=0x55b51398e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8000c00 tx=0x7fa1f801d810 comp rx=0 tx=0).stop 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 shutdown_connections 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55b513a00f80 0x55b513a033a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b513986370 0x55b51397f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b51397cb40 0x55b51398e990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 --2- 172.21.15.19:0/3704781045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b51397b5d0 0x55b51398e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 >> 172.21.15.19:0/3704781045 conn(0x55b513944310 msgr2=0x55b5139764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa1edffb640 0 client.17112 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 shutdown_connections 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 1 -- 172.21.15.19:0/3704781045 wait complete. 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.906+0000 7fa20f370740 20 client.17112 trim_cache size 0 max 16384 2024-09-12T03:28:12.913 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:28:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 msgr2=0x7fa1f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 0x7fa1f00a5040 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa20000a6e0 tx=0x7fa200005b40 comp rx=0 tx=0).stop 2024-09-12T03:28:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 msgr2=0x7fa208101430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fa20000cad0 tx=0x7fa20000cf90 comp rx=0 tx=0).stop 2024-09-12T03:28:12.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 shutdown_connections 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa1f00a2b80 0x7fa1f00a5040 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa20810c180 0x7fa208101430 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa208103670 0x7fa208100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 --2- 172.21.15.19:0/2022429232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa208102d50 0x7fa2081009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 >> 172.21.15.19:0/2022429232 conn(0x7fa2080fc7d0 msgr2=0x7fa2080fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 shutdown_connections 2024-09-12T03:28:12.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:12.907+0000 7fa20cf97640 1 -- 172.21.15.19:0/2022429232 wait complete. 2024-09-12T03:28:13.642 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:28:13.683 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:14.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- 172.21.15.19:0/1412945008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138101de0 msgr2=0x7f81381021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 --2- 172.21.15.19:0/1412945008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138101de0 0x7f81381021c0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8128009900 tx=0x7f812802a410 comp rx=0 tx=0).stop 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- 172.21.15.19:0/1412945008 shutdown_connections 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 --2- 172.21.15.19:0/1412945008 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 --2- 172.21.15.19:0/1412945008 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138102790 0x7f81381067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 --2- 172.21.15.19:0/1412945008 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138101de0 0x7f81381021c0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- 172.21.15.19:0/1412945008 >> 172.21.15.19:0/1412945008 conn(0x7f81380fd8c0 msgr2=0x7f81380ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- 172.21.15.19:0/1412945008 shutdown_connections 2024-09-12T03:28:14.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- 172.21.15.19:0/1412945008 wait complete. 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 Processor -- start 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.204+0000 7f813e5cd640 1 -- start start 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138101de0 0x7f8138198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8138112010 con 0x7f8138102790 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8138111e90 con 0x7f8138106f10 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813e5cd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8138112190 con 0x7f8138101de0 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42678/0 (socket says 172.21.15.19:42678) 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 -- 172.21.15.19:0/982797140 learned_addr learned my addr 172.21.15.19:0/982797140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f8137fff640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138101de0 0x7f8138198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813cb6a640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 -- 172.21.15.19:0/982797140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138101de0 msgr2=0x7f8138198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138101de0 0x7f8138198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 -- 172.21.15.19:0/982797140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 msgr2=0x7f813819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 -- 172.21.15.19:0/982797140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813819d630 con 0x7f8138102790 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f813cb6a640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:14.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.205+0000 7f81377fe640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b4a0 tx=0x7f812c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.206+0000 7f81357fa640 1 -- 172.21.15.19:0/982797140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c014070 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.206+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813819d920 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.206+0000 7f81357fa640 1 -- 172.21.15.19:0/982797140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c010af0 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.206+0000 7f81357fa640 1 -- 172.21.15.19:0/982797140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c017d10 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.206+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81381a5200 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81357fa640 1 -- 172.21.15.19:0/982797140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f812c017070 con 0x7f8138102790 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 Processor -- start 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 -- start start 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf139320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578bf09e620 con 0x5578bf139860 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5578bf045e50 con 0x5578bf148650 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f81409a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578bf09a5e0 con 0x5578bf138f40 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f8137fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f8137fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42680/0 (socket says 172.21.15.19:42680) 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f8137fff640 1 -- 172.21.15.19:0/2625809134 learned_addr learned my addr 172.21.15.19:0/2625809134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.209+0000 7f813cb6a640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf139320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f81357fa640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 0x7f81200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f81377fe640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 -- 172.21.15.19:0/2625809134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 msgr2=0x5578bf139320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf139320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 -- 172.21.15.19:0/2625809134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 msgr2=0x5578bf14aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf14aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 -- 172.21.15.19:0/2625809134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578bf036d40 con 0x5578bf139860 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f81377fe640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f813cb6a640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 0x7f81200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f81357fa640 1 -- 172.21.15.19:0/982797140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f812c0ce4b0 con 0x7f8138102790 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f8137fff640 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8128005ab0 tx=0x7f8128031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edec6216bc8cb80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f811e7fc640 1 -- 172.21.15.19:0/2625809134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812803f070 con 0x5578bf139860 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f811e7fc640 1 -- 172.21.15.19:0/2625809134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8128031a20 con 0x5578bf139860 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.210+0000 7f811e7fc640 1 -- 172.21.15.19:0/2625809134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128035070 con 0x5578bf139860 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- 172.21.15.19:0/2625809134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 msgr2=0x5578bf148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f813cb6a640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 0x7f81200a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8130007fb0 tx=0x7f8130004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f8128005ab0 tx=0x7f8128031630 comp rx=0 tx=0).stop 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- 172.21.15.19:0/2625809134 shutdown_connections 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf14aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf148110 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 --2- 172.21.15.19:0/2625809134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf139320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- 172.21.15.19:0/2625809134 >> 172.21.15.19:0/2625809134 conn(0x5578bf1355f0 msgr2=0x5578bf1359d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.250 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- 172.21.15.19:0/2625809134 shutdown_connections 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- 172.21.15.19:0/2625809134 wait complete. 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.211+0000 7f81409a6740 1 -- start start 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 Processor -- start 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf145510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf145a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578bf14b220 con 0x5578bf148650 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5578bf026f10 con 0x5578bf139860 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578bf14b3a0 con 0x5578bf138f40 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81409a6740 10 client.0 _ll_register_callbacks cb 0x5578bf04adb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:14.251 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42694/0 (socket says 172.21.15.19:42694) 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 -- 172.21.15.19:0/1350866059 learned_addr learned my addr 172.21.15.19:0/1350866059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f81377fe640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f8137fff640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 -- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 msgr2=0x5578bf145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 -- 172.21.15.19:0/1350866059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 msgr2=0x5578bf145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.212+0000 7f813cb6a640 1 -- 172.21.15.19:0/1350866059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578bedad580 con 0x5578bf148650 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f813cb6a640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8130011b50 tx=0x7f81300075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850c53c3c46f9507 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130022020 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f813001e650 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8128031a20 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f813001e850 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 10 client.17124 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5578bee77960 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8128031a20 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7f813001daf0 con 0x5578bf148650 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 10 client.17124 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 10 client.17124 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:14.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 20 client.17124 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf158240 0x5578bf15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 4 client.17124 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.213+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578bf0abb90 con 0x5578bf158240 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.214+0000 7f8137fff640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf158240 0x5578bf15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.214+0000 7f8137fff640 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf158240 0x5578bf15a6e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51962b97edb8bc39 server_cookie=7827e4e125a9512e in_seq=0 out_seq=0 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.214+0000 7f81177fe640 10 client.17124 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81177fe640 1 -- 172.21.15.19:0/1350866059 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578bf0abb90 con 0x5578bf158240 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81177fe640 10 client.17124 handle_command_reply: tid=0 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 client.17124 shutdown 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 msgr2=0x5578bf145f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f8130011b50 tx=0x7f81300075f0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 shutdown_connections 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf145f90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf158240 0x5578bf15a6e0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 --2- 172.21.15.19:0/1350866059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 >> 172.21.15.19:0/1350866059 conn(0x5578bf1355f0 msgr2=0x5578bf13b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.215+0000 7f81177fe640 0 client.17124 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.216+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 shutdown_connections 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.216+0000 7f81409a6740 1 -- 172.21.15.19:0/1350866059 wait complete. 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.216+0000 7f81409a6740 20 client.17124 trim_cache size 0 max 16384 2024-09-12T03:28:14.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 Processor -- start 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 -- start start 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf144470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81381ab410 con 0x5578bf139860 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81381ad070 con 0x5578bf148650 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81409a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81381abae0 con 0x5578bf138f40 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42698/0 (socket says 172.21.15.19:42698) 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 -- 172.21.15.19:0/4144988289 learned_addr learned my addr 172.21.15.19:0/4144988289 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 -- 172.21.15.19:0/4144988289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 msgr2=0x5578bf144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f8137fff640 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f813cb6a640 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 -- 172.21.15.19:0/4144988289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 msgr2=0x5578bf160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.233+0000 7f81377fe640 1 -- 172.21.15.19:0/4144988289 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578bf1402e0 con 0x5578bf139860 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81377fe640 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f812c009420 tx=0x7f812c017730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753ebe4aff852330 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f811dffb640 1 -- 172.21.15.19:0/4144988289 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c01e040 con 0x5578bf139860 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f811dffb640 1 -- 172.21.15.19:0/4144988289 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c099020 con 0x5578bf139860 2024-09-12T03:28:14.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f811dffb640 1 -- 172.21.15.19:0/4144988289 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812c097520 con 0x5578bf139860 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 -- 172.21.15.19:0/4144988289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 msgr2=0x5578bf15fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f812c009420 tx=0x7f812c017730 comp rx=0 tx=0).stop 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 -- 172.21.15.19:0/4144988289 shutdown_connections 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf148650 0x5578bf160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf139860 0x5578bf15fff0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 --2- 172.21.15.19:0/4144988289 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf144470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 -- 172.21.15.19:0/4144988289 >> 172.21.15.19:0/4144988289 conn(0x5578bf1355f0 msgr2=0x5578bf147590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.234+0000 7f81409a6740 1 -- 172.21.15.19:0/4144988289 shutdown_connections 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 -- 172.21.15.19:0/4144988289 wait complete. 2024-09-12T03:28:14.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 -- start start 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 Processor -- start 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf150630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf150b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f81381ac8e0 con 0x5578bf148650 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f81381ac1b0 con 0x5578bf139860 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.235+0000 7f81409a6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f81381a9410 con 0x5578bf138f40 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81409a6740 10 client.0 _ll_register_callbacks cb 0x7f81300117d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42708/0 (socket says 172.21.15.19:42708) 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 -- 172.21.15.19:0/3961711184 learned_addr learned my addr 172.21.15.19:0/3961711184 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f8137fff640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81377fe640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 -- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 msgr2=0x5578bf150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 -- 172.21.15.19:0/3961711184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 msgr2=0x5578bf150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 -- 172.21.15.19:0/3961711184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578bf14db40 con 0x5578bf148650 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81377fe640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:14.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f813cb6a640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8130011570 tx=0x7f813001e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4bbc834f5ea75 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81177fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8130022070 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8130026ac0 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.236+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f812c099020 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f8130010e90 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 10 client.17130 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x5578bf1c3770 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f812c099020 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7f813003c030 con 0x5578bf148650 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 10 client.17130 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 10 client.17130 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:14.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 20 client.17130 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf1c3f40 0x5578bf1c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 4 client.17130 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9e9af05e", "release": true, "await": true}] 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9e9af05e", "release": true, "await": true}) -- 0x5578bf0a60a0 con 0x5578bf1c3f40 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.237+0000 7f8137fff640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf1c3f40 0x5578bf1c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.238+0000 7f8137fff640 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf1c3f40 0x5578bf1c6360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aeac8f1acb1eef2b server_cookie=61608c370daa5907 in_seq=0 out_seq=0 2024-09-12T03:28:14.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.238+0000 7f81177fe640 10 client.17130 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:14.280 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:14.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 -- 172.21.15.19:0/3428309873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14106a80 msgr2=0x7f3e14106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 --2- 172.21.15.19:0/3428309873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14106a80 0x7f3e14106e60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009930 tx=0x7f3e0402a420 comp rx=0 tx=0).stop 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 -- 172.21.15.19:0/3428309873 shutdown_connections 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 --2- 172.21.15.19:0/3428309873 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 --2- 172.21.15.19:0/3428309873 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14100a80 0x7f3e14100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 --2- 172.21.15.19:0/3428309873 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14106a80 0x7f3e14106e60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 -- 172.21.15.19:0/3428309873 >> 172.21.15.19:0/3428309873 conn(0x7f3e140fc7a0 msgr2=0x7f3e140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 -- 172.21.15.19:0/3428309873 shutdown_connections 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.736+0000 7f3e19747640 1 -- 172.21.15.19:0/3428309873 wait complete. 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 Processor -- start 2024-09-12T03:28:14.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 -- start start 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e141989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14106a80 0x7f3e1419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14111f60 con 0x7f3e14100a80 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e14111de0 con 0x7f3e14101420 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e19747640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141120e0 con 0x7f3e14106a80 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.737+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42716/0 (socket says 172.21.15.19:42716) 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1848811422 learned_addr learned my addr 172.21.15.19:0/1848811422 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e141989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e137fe640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14106a80 0x7f3e1419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1848811422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14106a80 msgr2=0x7f3e1419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14106a80 0x7f3e1419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1848811422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 msgr2=0x7f3e141989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e141989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1848811422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1419d450 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e141989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009230 tx=0x7f3e04004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e10ff9640 1 -- 172.21.15.19:0/1848811422 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0403f030 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1419d6e0 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e10ff9640 1 -- 172.21.15.19:0/1848811422 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e04003ad0 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e10ff9640 1 -- 172.21.15.19:0/1848811422 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04035070 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.738+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e141a5020 con 0x7f3e14100a80 2024-09-12T03:28:14.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.741+0000 7f3e10ff9640 1 -- 172.21.15.19:0/1848811422 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e04042030 con 0x7f3e14100a80 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.741+0000 7f3e1bb20740 1 Processor -- start 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.741+0000 7f3e1bb20740 1 -- start start 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.741+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.741+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e93760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c069dec2f0 con 0x55c069e87b40 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c069d93cf0 con 0x55c069e865d0 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c069de82b0 con 0x55c069e91370 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42722/0 (socket says 172.21.15.19:42722) 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 -- 172.21.15.19:0/2760542671 learned_addr learned my addr 172.21.15.19:0/2760542671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e93760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e10ff9640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 0x7f3e000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 -- 172.21.15.19:0/2760542671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 msgr2=0x55c069e93760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e93760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 -- 172.21.15.19:0/2760542671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 msgr2=0x55c069e86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 -- 172.21.15.19:0/2760542671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c069d783a0 con 0x55c069e87b40 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 0x7f3e000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e10ff9640 1 -- 172.21.15.19:0/1848811422 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3e040ef5a0 con 0x7f3e14100a80 2024-09-12T03:28:14.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.742+0000 7f3e137fe640 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c011590 tx=0x7f3e0c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53398b248104228 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e097fa640 1 -- 172.21.15.19:0/2760542671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c017050 con 0x55c069e87b40 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e137fe640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 0x7f3e000a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c022000 tx=0x7f3e0c023000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e097fa640 1 -- 172.21.15.19:0/2760542671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c01bb90 con 0x55c069e87b40 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e097fa640 1 -- 172.21.15.19:0/2760542671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00e3b0 con 0x55c069e87b40 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e1bb20740 1 -- 172.21.15.19:0/2760542671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 msgr2=0x55c069e86090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c011590 tx=0x7f3e0c011a50 comp rx=0 tx=0).stop 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e1bb20740 1 -- 172.21.15.19:0/2760542671 shutdown_connections 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e93760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.743+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/2760542671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e86090 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- 172.21.15.19:0/2760542671 >> 172.21.15.19:0/2760542671 conn(0x55c069e4f310 msgr2=0x55c069e82c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- 172.21.15.19:0/2760542671 shutdown_connections 2024-09-12T03:28:14.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- 172.21.15.19:0/2760542671 wait complete. 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- start start 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 Processor -- start 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e87b40 0x55c069e8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c069e94070 con 0x55c069e865d0 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c069da7060 con 0x55c069e87b40 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c069e941f0 con 0x55c069e91370 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e1bb20740 10 client.0 _ll_register_callbacks cb 0x55c069afa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42734/0 (socket says 172.21.15.19:42734) 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1761925892 learned_addr learned my addr 172.21.15.19:0/1761925892 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:14.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e87b40 0x55c069e8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.744+0000 7f3e137fe640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 msgr2=0x55c069e8e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1761925892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e87b40 msgr2=0x55c069e8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e87b40 0x55c069e8e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1761925892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c069b5d3c0 con 0x55c069e865d0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3e040ed0b0 tx=0x7f3e04043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab28a3c2458e58d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04049070 con 0x55c069e865d0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3e040b8530 con 0x55c069e865d0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e0c01bb90 con 0x55c069e865d0 2024-09-12T03:28:14.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3e040bba00 con 0x55c069e865d0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e1bb20740 10 client.17142 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c069e9fb40 con 0x55c069e865d0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e0c01bb90 con 0x55c069e865d0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.745+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7f3e04031bd0 con 0x55c069e865d0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 10 client.17142 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 10 client.17142 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 20 client.17142 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069ea0fa0 0x55c069ea33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 4 client.17142 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c069df9860 con 0x55c069ea0fa0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069ea0fa0 0x55c069ea33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069ea0fa0 0x55c069ea33b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=162fc5725bd90302 server_cookie=7a054702f8ded546 in_seq=0 out_seq=0 2024-09-12T03:28:14.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:14.746+0000 7f3dfa7fc640 10 client.17142 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:28:14.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:14 smithi121 ceph-mon[34963]: pgmap v613: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-09-12T03:28:14.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:14 smithi019 ceph-mon[30953]: pgmap v613: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-09-12T03:28:15.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:14 smithi135 ceph-mon[35474]: pgmap v613: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.1 KiB/s wr, 16 op/s 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1761925892 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c069df9860 con 0x55c069ea0fa0 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3dfa7fc640 10 client.17142 handle_command_reply: tid=0 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 client.17142 shutdown 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 msgr2=0x55c069e90d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3e040ed0b0 tx=0x7f3e04043000 comp rx=0 tx=0).stop 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 shutdown_connections 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069ea0fa0 0x55c069ea33b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e87b40 0x55c069e8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1761925892 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e865d0 0x55c069e90d90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3dfa7fc640 0 client.17142 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:28:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 >> 172.21.15.19:0/1761925892 conn(0x55c069e4f310 msgr2=0x55c069e88930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 shutdown_connections 2024-09-12T03:28:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1761925892 wait complete. 2024-09-12T03:28:15.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.181+0000 7f3e1bb20740 20 client.17142 trim_cache size 0 max 16384 2024-09-12T03:28:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 Processor -- start 2024-09-12T03:28:15.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 -- start start 2024-09-12T03:28:15.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e89660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ad570 con 0x55c069e87b40 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141af1d0 con 0x55c069e865d0 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.195+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141adc40 con 0x55c069e91370 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42748/0 (socket says 172.21.15.19:42748) 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 -- 172.21.15.19:0/3763203318 learned_addr learned my addr 172.21.15.19:0/3763203318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56800/0 (socket says 172.21.15.19:56800) 2024-09-12T03:28:15.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e137fe640 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e89660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 -- 172.21.15.19:0/3763203318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 msgr2=0x55c069e89660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e89660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 -- 172.21.15.19:0/3763203318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 msgr2=0x55c069e81bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 -- 172.21.15.19:0/3763203318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c069e831a0 con 0x55c069e87b40 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc011630 tx=0x7f3dfc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245bef5d8e11772 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e08ff9640 1 -- 172.21.15.19:0/3763203318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc010c60 con 0x55c069e87b40 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e08ff9640 1 -- 172.21.15.19:0/3763203318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dfc010e00 con 0x55c069e87b40 2024-09-12T03:28:15.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.196+0000 7f3e08ff9640 1 -- 172.21.15.19:0/3763203318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dfc004a90 con 0x55c069e87b40 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 -- 172.21.15.19:0/3763203318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 msgr2=0x55c069e89120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc011630 tx=0x7f3dfc00b410 comp rx=0 tx=0).stop 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 -- 172.21.15.19:0/3763203318 shutdown_connections 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e89660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e89120 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/3763203318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e81bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 -- 172.21.15.19:0/3763203318 >> 172.21.15.19:0/3763203318 conn(0x55c069e4f310 msgr2=0x55c069e814e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 -- 172.21.15.19:0/3763203318 shutdown_connections 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.197+0000 7f3e1bb20740 1 -- 172.21.15.19:0/3763203318 wait complete. 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 -- start start 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 Processor -- start 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e8b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141aea40 con 0x55c069e87b40 2024-09-12T03:28:15.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ae310 con 0x55c069e865d0 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3e141ab550 con 0x55c069e91370 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e1bb20740 10 client.0 _ll_register_callbacks cb 0x7f3e04005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e0bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42756/0 (socket says 172.21.15.19:42756) 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e12ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e8b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e0bfff640 1 -- 172.21.15.19:0/1058973430 learned_addr learned my addr 172.21.15.19:0/1058973430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.198+0000 7f3e137fe640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 msgr2=0x55c069e8bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1058973430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 msgr2=0x55c069e8b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:15.204 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 -- 172.21.15.19:0/1058973430 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c069f0adc0 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e12ffd640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e8b1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e040ba170 tx=0x7f3e04003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc06934237456d60 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3dfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04049070 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3e040b8530 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3dfc010e00 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f3e040bb820 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e1bb20740 10 client.26540 fetch_fsmap learned FSMap version 46 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.199+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55c069f0c040 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dfc010e00 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 46) ==== 8398+0+0 (secure 0 0 0) 0x7f3e040bb310 con 0x55c069e865d0 2024-09-12T03:28:15.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 10 client.26540 fetch_fsmap finished waiting for FSMap version 46 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 10 client.26540 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 20 client.26540 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069f0c580 0x55c069f0e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 4 client.26540 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c069df3d70 con 0x55c069f0c580 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.200+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069f0c580 0x55c069f0e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3dfa7fc640 10 client.26540 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e0bfff640 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069f0c580 0x55c069f0e970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd5e0d167de5af26 server_cookie=6b52733cb6cefb51 in_seq=0 out_seq=0 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3dfa7fc640 1 -- 172.21.15.19:0/1058973430 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c069df3d70 con 0x55c069f0c580 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3dfa7fc640 10 client.26540 handle_command_reply: tid=0 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 client.26540 shutdown 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 msgr2=0x55c069e8b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e8b1a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3e040ba170 tx=0x7f3e04003eb0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 shutdown_connections 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c069f0c580 0x55c069f0e970 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c069e91370 0x55c069e8bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c069e87b40 0x55c069e8b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 --2- 172.21.15.19:0/1058973430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c069e865d0 0x55c069e8b1a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 >> 172.21.15.19:0/1058973430 conn(0x55c069e4f310 msgr2=0x55c069e81d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3dfa7fc640 0 client.26540 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 shutdown_connections 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 1 -- 172.21.15.19:0/1058973430 wait complete. 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.201+0000 7f3e1bb20740 20 client.26540 trim_cache size 0 max 16384 2024-09-12T03:28:15.207 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 msgr2=0x7f3e000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 0x7f3e000a5040 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c022000 tx=0x7f3e0c023000 comp rx=0 tx=0).stop 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 msgr2=0x7f3e141984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009230 tx=0x7f3e04004270 comp rx=0 tx=0).stop 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 shutdown_connections 2024-09-12T03:28:15.208 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3e14106a80 0x7f3e1419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3e14101420 0x7f3e141989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3e000a2b80 0x7f3e000a5040 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 --2- 172.21.15.19:0/1848811422 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3e14100a80 0x7f3e141984a0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 >> 172.21.15.19:0/1848811422 conn(0x7f3e140fc7a0 msgr2=0x7f3e140fd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 shutdown_connections 2024-09-12T03:28:15.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.202+0000 7f3e19747640 1 -- 172.21.15.19:0/1848811422 wait complete. 2024-09-12T03:28:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.207+0000 7f81177fe640 1 -- 172.21.15.19:0/3961711184 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5578bf0a60a0 con 0x5578bf1c3f40 2024-09-12T03:28:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.207+0000 7f81177fe640 10 client.17130 handle_command_reply: tid=0 2024-09-12T03:28:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.207+0000 7f81409a6740 1 client.17130 shutdown 2024-09-12T03:28:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 msgr2=0x5578bf14d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8130011570 tx=0x7f813001e700 comp rx=0 tx=0).stop 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 shutdown_connections 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578bf148650 0x5578bf14d600 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5578bf1c3f40 0x5578bf1c6360 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578bf139860 0x5578bf150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81177fe640 0 client.17130 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 --2- 172.21.15.19:0/3961711184 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578bf138f40 0x5578bf150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 >> 172.21.15.19:0/3961711184 conn(0x5578bf1355f0 msgr2=0x5578bf146d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 shutdown_connections 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 1 -- 172.21.15.19:0/3961711184 wait complete. 2024-09-12T03:28:15.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.208+0000 7f81409a6740 20 client.17130 trim_cache size 0 max 16384 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 46, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 32, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "9e9af05e": { 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 32, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:28:15.212 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 139.4, 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:15.213 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 msgr2=0x7f81200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 0x7f81200a5040 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8130007fb0 tx=0x7f8130004b50 comp rx=0 tx=0).stop 2024-09-12T03:28:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 msgr2=0x7f8138198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:15.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f812c00b4a0 tx=0x7f812c00b960 comp rx=0 tx=0).stop 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 shutdown_connections 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8138106f10 0x7f813819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f81200a2b80 0x7f81200a5040 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8138102790 0x7f8138198bc0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 --2- 172.21.15.19:0/982797140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8138101de0 0x7f8138198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 >> 172.21.15.19:0/982797140 conn(0x7f81380fd8c0 msgr2=0x7f81380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 shutdown_connections 2024-09-12T03:28:15.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:15.209+0000 7f813e5cd640 1 -- 172.21.15.19:0/982797140 wait complete. 2024-09-12T03:28:15.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:28:16.608 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9e9af05e', total seconds elapsed: 26.0 2024-09-12T03:28:16.609 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-09-12T03:28:16.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:16.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:16.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:16 smithi121 ceph-mon[34963]: pgmap v614: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:28:16.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:16 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:16 smithi019 ceph-mon[30953]: pgmap v614: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:28:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:16 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:16 smithi135 ceph-mon[35474]: pgmap v614: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-12T03:28:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:16 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 -- 172.21.15.19:0/1546588432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8108290 msgr2=0x7fb4d810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/1546588432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8108290 0x7fb4d810ec30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009cd0 tx=0x7fb4d002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 -- 172.21.15.19:0/1546588432 shutdown_connections 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/1546588432 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8108290 0x7fb4d810ec30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/1546588432 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8103740 0x7fb4d8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/1546588432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8102d90 0x7fb4d8103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 -- 172.21.15.19:0/1546588432 >> 172.21.15.19:0/1546588432 conn(0x7fb4d80fc7d0 msgr2=0x7fb4d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.187+0000 7fb4ddb65640 1 -- 172.21.15.19:0/1546588432 shutdown_connections 2024-09-12T03:28:17.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.188+0000 7fb4ddb65640 1 -- 172.21.15.19:0/1546588432 wait complete. 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.188+0000 7fb4ddb65640 1 Processor -- start 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 -- start start 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 0x7fb4d8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8108290 0x7fb4d819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8112090 con 0x7fb4d8103740 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8111f10 con 0x7fb4d8108290 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4ddb65640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8112210 con 0x7fb4d8102d90 2024-09-12T03:28:17.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42776/0 (socket says 172.21.15.19:42776) 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/2047111544 learned_addr learned my addr 172.21.15.19:0/2047111544 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 0x7fb4d8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8108290 0x7fb4d819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/2047111544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 msgr2=0x7fb4d8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 0x7fb4d8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/2047111544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8108290 msgr2=0x7fb4d819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8108290 0x7fb4d819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/2047111544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d819d510 con 0x7fb4d8103740 2024-09-12T03:28:17.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 0x7fb4d8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.189+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9c0 tx=0x7fb4cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.190+0000 7fb4d4ff9640 1 -- 172.21.15.19:0/2047111544 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00cd80 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.190+0000 7fb4d4ff9640 1 -- 172.21.15.19:0/2047111544 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc005500 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.190+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d819d800 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.190+0000 7fb4d4ff9640 1 -- 172.21.15.19:0/2047111544 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc010690 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.190+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81a50e0 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.192+0000 7fb4d4ff9640 1 -- 172.21.15.19:0/2047111544 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb4cc0046c0 con 0x7fb4d8103740 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.192+0000 7fb4dff3e740 1 Processor -- start 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 -- start start 2024-09-12T03:28:17.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589cd80a2f0 con 0x5589cd8b3780 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589cd7b1cf0 con 0x5589cd8a5b40 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5589cd8062b0 con 0x5589cd8a45d0 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56838/0 (socket says 172.21.15.19:56838) 2024-09-12T03:28:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42788/0 (socket says 172.21.15.19:42788) 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 -- 172.21.15.19:0/1196404607 learned_addr learned my addr 172.21.15.19:0/1196404607 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8a4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57670/0 (socket says 172.21.15.19:57670) 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d4ff9640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 -- 172.21.15.19:0/1196404607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 msgr2=0x5589cd8a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 -- 172.21.15.19:0/1196404607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 msgr2=0x5589cd8b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 -- 172.21.15.19:0/1196404607 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cd7963a0 con 0x5589cd8a5b40 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.193+0000 7fb4d4ff9640 1 -- 172.21.15.19:0/2047111544 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4cc0cf350 con 0x7fb4d8103740 2024-09-12T03:28:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.194+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0004ab0 tx=0x7fb4d0031430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5e1b35832073ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.194+0000 7fb4bdffb640 1 -- 172.21.15.19:0/1196404607 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0041040 con 0x5589cd8a5b40 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.194+0000 7fb4bdffb640 1 -- 172.21.15.19:0/1196404607 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d0043070 con 0x5589cd8a5b40 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.194+0000 7fb4bdffb640 1 -- 172.21.15.19:0/1196404607 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0030b80 con 0x5589cd8a5b40 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.194+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fe0 tx=0x7fb4d004d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- 172.21.15.19:0/1196404607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 msgr2=0x5589cd8a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0004ab0 tx=0x7fb4d0031430 comp rx=0 tx=0).stop 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- 172.21.15.19:0/1196404607 shutdown_connections 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/1196404607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8a4090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- 172.21.15.19:0/1196404607 >> 172.21.15.19:0/1196404607 conn(0x5589cd86d310 msgr2=0x5589cd8a0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- 172.21.15.19:0/1196404607 shutdown_connections 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- 172.21.15.19:0/1196404607 wait complete. 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 -- start start 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.195+0000 7fb4dff3e740 1 Processor -- start 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5589cd8b6480 con 0x5589cd8a5b40 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5589cd7c5060 con 0x5589cd8b3780 2024-09-12T03:28:17.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5589cd8b6600 con 0x5589cd8a45d0 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4dff3e740 10 client.0 _ll_register_callbacks cb 0x5589cd5188c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57672/0 (socket says 172.21.15.19:57672) 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42804/0 (socket says 172.21.15.19:42804) 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d77fe640 1 -- 172.21.15.19:0/2081733277 learned_addr learned my addr 172.21.15.19:0/2081733277 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 msgr2=0x5589cd8b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2081733277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 msgr2=0x5589cd8b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2081733277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cd57b3c0 con 0x5589cd8b3780 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8b09e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009a60 tx=0x7fb4d0031620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16934ab1e5cd0643 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.196+0000 7fb4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0041030 con 0x5589cd8b3780 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4d002e040 con 0x5589cd8b3780 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d0043070 con 0x5589cd8b3780 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4d0030e70 con 0x5589cd8b3780 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 10 client.26552 fetch_fsmap learned FSMap version 47 2024-09-12T03:28:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5589cd8c1f00 con 0x5589cd8b3780 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4d0043070 con 0x5589cd8b3780 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 47) ==== 8394+0+0 (secure 0 0 0) 0x7fb4d0030070 con 0x5589cd8b3780 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 10 client.26552 fetch_fsmap finished waiting for FSMap version 47 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 10 client.26552 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 20 client.26552 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd8c3250 0x5589cd8c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 4 client.26552 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.197+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589cd817860 con 0x5589cd8c3250 2024-09-12T03:28:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.198+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd8c3250 0x5589cd8c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.198+0000 7fb4b6ffd640 10 client.26552 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.198+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd8c3250 0x5589cd8c5640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=39a80d522965cfca server_cookie=6df384bb6553106 in_seq=0 out_seq=0 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2081733277 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589cd817860 con 0x5589cd8c3250 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4b6ffd640 10 client.26552 handle_command_reply: tid=0 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 client.26552 shutdown 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 msgr2=0x5589cd8b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8b09e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0009a60 tx=0x7fb4d0031620 comp rx=0 tx=0).stop 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 shutdown_connections 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd8c3250 0x5589cd8c5640 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8b09e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a5b40 0x5589cd8b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2081733277 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4b6ffd640 0 client.26552 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 >> 172.21.15.19:0/2081733277 conn(0x5589cd86d310 msgr2=0x5589cd8a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 shutdown_connections 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2081733277 wait complete. 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.199+0000 7fb4dff3e740 20 client.26552 trim_cache size 0 max 16384 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 Processor -- start 2024-09-12T03:28:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 -- start start 2024-09-12T03:28:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 0x5589cd8af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.287 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:28:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ab3a0 con 0x5589cd8a45d0 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ad000 con 0x5589cd8b3780 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.217+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81aba70 con 0x5589cd8a5b40 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57680/0 (socket says 172.21.15.19:57680) 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/434801237 learned_addr learned my addr 172.21.15.19:0/434801237 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 0x5589cd8af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/434801237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 msgr2=0x5589cd8c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/434801237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 msgr2=0x5589cd8af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 0x5589cd8af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d6ffd640 1 -- 172.21.15.19:0/434801237 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589cd8b5ee0 con 0x5589cd8a5b40 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.218+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 0x5589cd8af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.219+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c6a0 tx=0x7fb4cc009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52c99fdd1b17a866 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.219+0000 7fb4bd7fa640 1 -- 172.21.15.19:0/434801237 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc09b930 con 0x5589cd8a5b40 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.219+0000 7fb4bd7fa640 1 -- 172.21.15.19:0/434801237 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc0b3070 con 0x5589cd8a5b40 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.219+0000 7fb4bd7fa640 1 -- 172.21.15.19:0/434801237 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc09b4c0 con 0x5589cd8a5b40 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 -- 172.21.15.19:0/434801237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 msgr2=0x5589cd8c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c6a0 tx=0x7fb4cc009700 comp rx=0 tx=0).stop 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 -- 172.21.15.19:0/434801237 shutdown_connections 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8b3780 0x5589cd8c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a5b40 0x5589cd8c8660 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/434801237 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8a45d0 0x5589cd8af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 -- 172.21.15.19:0/434801237 >> 172.21.15.19:0/434801237 conn(0x5589cd86d310 msgr2=0x5589cd8aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 -- 172.21.15.19:0/434801237 shutdown_connections 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.220+0000 7fb4dff3e740 1 -- 172.21.15.19:0/434801237 wait complete. 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.221+0000 7fb4dff3e740 1 -- start start 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.221+0000 7fb4dff3e740 1 Processor -- start 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ac870 con 0x5589cd8b3780 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ac140 con 0x5589cd8a5b40 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81a9380 con 0x5589cd8a45d0 2024-09-12T03:28:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4dff3e740 10 client.0 _ll_register_callbacks cb 0x7fb4cc0981d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42816/0 (socket says 172.21.15.19:42816) 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57682/0 (socket says 172.21.15.19:57682) 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d7fff640 1 -- 172.21.15.19:0/2042933016 learned_addr learned my addr 172.21.15.19:0/2042933016 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.222+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 -- 172.21.15.19:0/2042933016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 msgr2=0x5589cd8c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 -- 172.21.15.19:0/2042933016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 msgr2=0x5589cd8c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 -- 172.21.15.19:0/2042933016 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589cd8c6e90 con 0x5589cd8a45d0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d7fff640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4d77fe640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5589cd8b19c0 tx=0x7fb4c8004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0abc71a38a9776 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8012ca0 con 0x5589cd8a45d0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fb4c80035f0 con 0x5589cd8a45d0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.223+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc0b3070 con 0x5589cd8a45d0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.224+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4c8012e40 con 0x5589cd8a45d0 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.224+0000 7fb4dff3e740 10 client.26524 fetch_fsmap learned FSMap version 47 2024-09-12T03:28:17.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.224+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5589cd92db90 con 0x5589cd8a45d0 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4cc0b3070 con 0x5589cd8a45d0 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 47) ==== 8394+0+0 (secure 0 0 0) 0x7fb4c80279a0 con 0x5589cd8a45d0 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 10 client.26524 fetch_fsmap finished waiting for FSMap version 47 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 10 client.26524 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 20 client.26524 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd92e260 0x5589cd930680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 4 client.26524 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.225+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589cd811d70 con 0x5589cd92e260 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.226+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd92e260 0x5589cd930680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.226+0000 7fb4d6ffd640 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd92e260 0x5589cd930680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6e6219ce9b91daa9 server_cookie=908f0f0a7e0f185c in_seq=0 out_seq=0 2024-09-12T03:28:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.226+0000 7fb4b6ffd640 10 client.26524 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4b6ffd640 1 -- 172.21.15.19:0/2042933016 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589cd811d70 con 0x5589cd92e260 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4b6ffd640 10 client.26524 handle_command_reply: tid=0 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 client.26524 shutdown 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 msgr2=0x5589cd8c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x5589cd8b19c0 tx=0x7fb4c8004410 comp rx=0 tx=0).stop 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 shutdown_connections 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5589cd92e260 0x5589cd930680 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5589cd8b3780 0x5589cd8c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5589cd8a5b40 0x5589cd8c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 --2- 172.21.15.19:0/2042933016 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5589cd8a45d0 0x5589cd8c8e00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4b6ffd640 0 client.26524 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 >> 172.21.15.19:0/2042933016 conn(0x5589cd86d310 msgr2=0x5589cd8af540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 shutdown_connections 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 1 -- 172.21.15.19:0/2042933016 wait complete. 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.227+0000 7fb4dff3e740 20 client.26524 trim_cache size 0 max 16384 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 msgr2=0x7fb4c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0005fe0 tx=0x7fb4d004d000 comp rx=0 tx=0).stop 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 msgr2=0x7fb4d8198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00e9c0 tx=0x7fb4cc00ee80 comp rx=0 tx=0).stop 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 shutdown_connections 2024-09-12T03:28:17.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb4d8108290 0x7fb4d819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fb4c00a2c00 0x7fb4c00a50c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fb4d8103740 0x7fb4d8198aa0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 --2- 172.21.15.19:0/2047111544 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fb4d8102d90 0x7fb4d8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 >> 172.21.15.19:0/2047111544 conn(0x7fb4d80fc7d0 msgr2=0x7fb4d8106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 shutdown_connections 2024-09-12T03:28:17.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:17.228+0000 7fb4ddb65640 1 -- 172.21.15.19:0/2047111544 wait complete. 2024-09-12T03:28:18.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:28:18.189 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- 172.21.15.19:0/2917369744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 msgr2=0x7f33881031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 --2- 172.21.15.19:0/2917369744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f33881031c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f3384009900 tx=0x7f338402a480 comp rx=0 tx=0).stop 2024-09-12T03:28:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- 172.21.15.19:0/2917369744 shutdown_connections 2024-09-12T03:28:18.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 --2- 172.21.15.19:0/2917369744 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 0x7f338810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 --2- 172.21.15.19:0/2917369744 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 0x7f338810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 --2- 172.21.15.19:0/2917369744 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f33881031c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- 172.21.15.19:0/2917369744 >> 172.21.15.19:0/2917369744 conn(0x7f33880fc7a0 msgr2=0x7f33880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- 172.21.15.19:0/2917369744 shutdown_connections 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- 172.21.15.19:0/2917369744 wait complete. 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 Processor -- start 2024-09-12T03:28:18.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.689+0000 7f33907cc640 1 -- start start 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 0x7f3388198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 0x7f338819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3388111f70 con 0x7f3388102de0 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3388111df0 con 0x7f338810c180 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f33907cc640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33881120f0 con 0x7f3388103700 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42830/0 (socket says 172.21.15.19:42830) 2024-09-12T03:28:18.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 -- 172.21.15.19:0/3958949825 learned_addr learned my addr 172.21.15.19:0/3958949825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338dd67640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 0x7f3388198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338ed69640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 0x7f338819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 -- 172.21.15.19:0/3958949825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 msgr2=0x7f3388198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 0x7f3388198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 -- 172.21.15.19:0/3958949825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 msgr2=0x7f338819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 0x7f338819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 -- 172.21.15.19:0/3958949825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338819d400 con 0x7f3388102de0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.690+0000 7f338e568640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f3384004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.691+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338403f030 con 0x7f3388102de0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.691+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3384004440 con 0x7f3388102de0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.691+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384035070 con 0x7f3388102de0 2024-09-12T03:28:18.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.691+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338819d690 con 0x7f3388102de0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.691+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f338819db20 con 0x7f3388102de0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.692+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33881031c0 con 0x7f3388102de0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.693+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3384042030 con 0x7f3388102de0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.694+0000 7f337f7fe640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 0x7f33740a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.694+0000 7f338dd67640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 0x7f33740a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.694+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f33840ef550 con 0x7f3388102de0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.694+0000 7f338dd67640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 0x7f33740a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f338810d620 tx=0x7f3378005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:18.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.697+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f33840bbed0 con 0x7f3388102de0 2024-09-12T03:28:18.752 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:18.874 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:18 smithi019 ceph-mon[30953]: pgmap v615: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-09-12T03:28:18.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:18 smithi121 ceph-mon[34963]: pgmap v615: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-09-12T03:28:18.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.940+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33881997b0 con 0x7f3388102de0 2024-09-12T03:28:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.940+0000 7f337f7fe640 1 -- 172.21.15.19:0/3958949825 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11987 (secure 0 0 0) 0x7f33881997b0 con 0x7f3388102de0 2024-09-12T03:28:18.941 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:28:18.941 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":47,"btime":"2024-09-12T03:28:15:590229+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":47,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:28:15.208135+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:28:18.941 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 47 2024-09-12T03:28:18.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.942+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 msgr2=0x7f33740a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.942+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 0x7f33740a50c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f338810d620 tx=0x7f3378005f90 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.942+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 msgr2=0x7f3388198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.942+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f3384006010 tx=0x7f3384004040 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 shutdown_connections 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810c180 0x7f338819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388103700 0x7f3388198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33740a2c00 0x7f33740a50c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 --2- 172.21.15.19:0/3958949825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388102de0 0x7f3388198450 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 >> 172.21.15.19:0/3958949825 conn(0x7f33880fc7a0 msgr2=0x7f33880fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 shutdown_connections 2024-09-12T03:28:18.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:18.943+0000 7f33907cc640 1 -- 172.21.15.19:0/3958949825 wait complete. 2024-09-12T03:28:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:18 smithi135 ceph-mon[35474]: pgmap v615: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.0 KiB/s wr, 16 op/s 2024-09-12T03:28:19.808 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0' 2024-09-12T03:28:19.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 139.4 --await-for 45.0 2024-09-12T03:28:19.882 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:19 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3958949825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:19 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3958949825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.945 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:19 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3958949825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 -- 172.21.15.19:0/3785985793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 msgr2=0x7f007810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 --2- 172.21.15.19:0/3785985793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007810ec40 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0070009930 tx=0x7f007002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 -- 172.21.15.19:0/3785985793 shutdown_connections 2024-09-12T03:28:19.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 --2- 172.21.15.19:0/3785985793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007810ec40 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 --2- 172.21.15.19:0/3785985793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f00781036e0 0x7f0078107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 --2- 172.21.15.19:0/3785985793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0078102d30 0x7f0078103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 -- 172.21.15.19:0/3785985793 >> 172.21.15.19:0/3785985793 conn(0x7f00780fc7d0 msgr2=0x7f00780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:19.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 -- 172.21.15.19:0/3785985793 shutdown_connections 2024-09-12T03:28:19.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.990+0000 7f007ec05640 1 -- 172.21.15.19:0/3785985793 wait complete. 2024-09-12T03:28:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 Processor -- start 2024-09-12T03:28:19.992 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 -- start start 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00781036e0 0x7f00781a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00781120a0 con 0x7f00781082a0 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0078111f20 con 0x7f00781036e0 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007ec05640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0078112220 con 0x7f0078102d30 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007d1a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57722/0 (socket says 172.21.15.19:57722) 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007d1a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007819a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42848/0 (socket says 172.21.15.19:42848) 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.991+0000 7f007c9a1640 1 -- 172.21.15.19:0/1775587150 learned_addr learned my addr 172.21.15.19:0/1775587150 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:19.993 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f0077fff640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00781036e0 0x7f00781a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 -- 172.21.15.19:0/1775587150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00781036e0 msgr2=0x7f00781a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00781036e0 0x7f00781a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 -- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 msgr2=0x7f007819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 -- 172.21.15.19:0/1775587150 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007819ad00 con 0x7f0078102d30 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007c9a1640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f006800eab0 tx=0x7f006800ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f0075ffb640 1 -- 172.21.15.19:0/1775587150 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006800ce40 con 0x7f0078102d30 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f0075ffb640 1 -- 172.21.15.19:0/1775587150 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0068011950 con 0x7f0078102d30 2024-09-12T03:28:19.994 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007819aff0 con 0x7f0078102d30 2024-09-12T03:28:19.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.992+0000 7f0075ffb640 1 -- 172.21.15.19:0/1775587150 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068010ae0 con 0x7f0078102d30 2024-09-12T03:28:19.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.993+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00781a72e0 con 0x7f0078102d30 2024-09-12T03:28:19.995 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0075ffb640 1 -- 172.21.15.19:0/1775587150 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0068011b00 con 0x7f0078102d30 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0080fde740 1 Processor -- start 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0080fde740 1 -- start start 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0080fde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 0x559eb383f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0080fde740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb383fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.995+0000 7f0080fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0080fde740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559eb37a52f0 con 0x559eb383f5d0 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0080fde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559eb374ccf0 con 0x559eb3840b40 2024-09-12T03:28:19.997 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0080fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559eb37a12b0 con 0x559eb384a370 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0075ffb640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 0x7f00600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0077fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007d1a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 0x559eb383f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0077fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57738/0 (socket says 172.21.15.19:57738) 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0077fff640 1 -- 172.21.15.19:0/3923651800 learned_addr learned my addr 172.21.15.19:0/3923651800 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007d1a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 0x559eb383f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:56916/0 (socket says 172.21.15.19:56916) 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb383fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0075ffb640 1 -- 172.21.15.19:0/1775587150 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f00680cf5f0 con 0x7f0078102d30 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007d1a2640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 0x7f00600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 -- 172.21.15.19:0/3923651800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 msgr2=0x559eb384c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 -- 172.21.15.19:0/3923651800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 msgr2=0x559eb383f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:19.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 0x559eb383f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 -- 172.21.15.19:0/3923651800 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb37313a0 con 0x559eb383f5d0 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f0077fff640 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.996+0000 7f007c9a1640 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb383fa10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f006809bdf0 tx=0x7f006809ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97b65db22ce43ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.997+0000 7f007d1a2640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 0x7f00600a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0070006010 tx=0x7f0070009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.997+0000 7f005effd640 1 -- 172.21.15.19:0/3923651800 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680a4ad0 con 0x559eb383f5d0 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.997+0000 7f005effd640 1 -- 172.21.15.19:0/3923651800 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006809be60 con 0x559eb383f5d0 2024-09-12T03:28:19.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.997+0000 7f005effd640 1 -- 172.21.15.19:0/3923651800 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006809c320 con 0x559eb383f5d0 2024-09-12T03:28:20.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- 172.21.15.19:0/3923651800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 msgr2=0x559eb383fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb383fa10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f006809bdf0 tx=0x7f006809ba60 comp rx=0 tx=0).stop 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- 172.21.15.19:0/3923651800 shutdown_connections 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb384c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb383fa10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 --2- 172.21.15.19:0/3923651800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb3840b40 0x559eb383f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- 172.21.15.19:0/3923651800 >> 172.21.15.19:0/3923651800 conn(0x559eb3808310 msgr2=0x559eb383bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- 172.21.15.19:0/3923651800 shutdown_connections 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- 172.21.15.19:0/3923651800 wait complete. 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 -- start start 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.998+0000 7f0080fde740 1 Processor -- start 2024-09-12T03:28:20.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3849d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3847650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559eb384d070 con 0x559eb383f5d0 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559eb3760060 con 0x559eb384a370 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559eb384d1f0 con 0x559eb3840b40 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0080fde740 10 client.0 _ll_register_callbacks cb 0x559eb34b38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3849d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:57742/0 (socket says 172.21.15.19:57742) 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3849d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42868/0 (socket says 172.21.15.19:42868) 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 -- 172.21.15.19:0/2547006330 learned_addr learned my addr 172.21.15.19:0/2547006330 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:20.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f007d1a2640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3847650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 -- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 msgr2=0x559eb3847650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3847650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 -- 172.21.15.19:0/2547006330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 msgr2=0x559eb3849d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3849d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 -- 172.21.15.19:0/2547006330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb35163c0 con 0x559eb3840b40 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0077fff640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f006c0126e0 tx=0x7f006c012ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fe37ba90045cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:19.999+0000 7f0057fff640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006c0120d0 con 0x559eb3840b40 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f006c00f920 con 0x559eb3840b40 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006809be60 con 0x559eb3840b40 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f006c012270 con 0x559eb3840b40 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 10 client.26542 fetch_fsmap learned FSMap version 48 2024-09-12T03:28:20.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559eb3858bd0 con 0x559eb3840b40 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006809be60 con 0x559eb3840b40 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 48) ==== 8398+0+0 (secure 0 0 0) 0x7f006c01c420 con 0x559eb3840b40 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 10 client.26542 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 10 client.26542 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 20 client.26542 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb385c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 4 client.26542 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.000+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559eb37b2860 con 0x559eb385a480 2024-09-12T03:28:20.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.001+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb385c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.001+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb385c900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a18b438475d0c30c server_cookie=9ffed44da64fb835 in_seq=0 out_seq=0 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.001+0000 7f0057fff640 10 client.26542 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0057fff640 1 -- 172.21.15.19:0/2547006330 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559eb37b2860 con 0x559eb385a480 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0057fff640 10 client.26542 handle_command_reply: tid=0 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 client.26542 shutdown 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 msgr2=0x559eb3847040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f006c0126e0 tx=0x7f006c012ba0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 shutdown_connections 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb385c900 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3847650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0057fff640 0 client.26542 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3847040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 --2- 172.21.15.19:0/2547006330 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3849d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 >> 172.21.15.19:0/2547006330 conn(0x559eb3808310 msgr2=0x559eb3841990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 shutdown_connections 2024-09-12T03:28:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 1 -- 172.21.15.19:0/2547006330 wait complete. 2024-09-12T03:28:20.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.002+0000 7f0080fde740 20 client.26542 trim_cache size 0 max 16384 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.015+0000 7f0080fde740 1 Processor -- start 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.015+0000 7f0080fde740 1 -- start start 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 0x559eb383be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb3856340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00781af740 con 0x559eb3840b40 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b13d0 con 0x559eb383f5d0 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0080fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00781afe10 con 0x559eb384a370 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0077fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0077fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42876/0 (socket says 172.21.15.19:42876) 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0077fff640 1 -- 172.21.15.19:0/2745255675 learned_addr learned my addr 172.21.15.19:0/2745255675 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:20.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 0x559eb383be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007d1a2640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb3856340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 -- 172.21.15.19:0/2745255675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 msgr2=0x559eb3856340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb3856340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 -- 172.21.15.19:0/2745255675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 msgr2=0x559eb385c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 -- 172.21.15.19:0/2745255675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559eb38479c0 con 0x559eb383f5d0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f0077fff640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.016+0000 7f007c9a1640 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 0x559eb383be10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x559eb383cac0 tx=0x7f006809bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73df6a0fd1dc97ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f005e7fc640 1 -- 172.21.15.19:0/2745255675 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006809be60 con 0x559eb383f5d0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f005e7fc640 1 -- 172.21.15.19:0/2745255675 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f006809a160 con 0x559eb383f5d0 2024-09-12T03:28:20.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f005e7fc640 1 -- 172.21.15.19:0/2745255675 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f006809c320 con 0x559eb383f5d0 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 -- 172.21.15.19:0/2745255675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 msgr2=0x559eb383be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 0x559eb383be10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x559eb383cac0 tx=0x7f006809bad0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 -- 172.21.15.19:0/2745255675 shutdown_connections 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb384a370 0x559eb3856340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb3840b40 0x559eb385c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 --2- 172.21.15.19:0/2745255675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb383f5d0 0x559eb383be10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.017+0000 7f0080fde740 1 -- 172.21.15.19:0/2745255675 >> 172.21.15.19:0/2745255675 conn(0x559eb3808310 msgr2=0x559eb383b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.018+0000 7f0080fde740 1 -- 172.21.15.19:0/2745255675 shutdown_connections 2024-09-12T03:28:20.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.018+0000 7f0080fde740 1 -- 172.21.15.19:0/2745255675 wait complete. 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.018+0000 7f0080fde740 1 -- start start 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.018+0000 7f0080fde740 1 Processor -- start 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.019+0000 7f0080fde740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.019+0000 7f0080fde740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3852c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.019+0000 7f0080fde740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3843910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.019+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.019+0000 7f007c9a1640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42878/0 (socket says 172.21.15.19:42878) 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0080fde740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b0c40 con 0x559eb383f5d0 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0080fde740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f00781b0510 con 0x559eb384a370 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0080fde740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f00781ad720 con 0x559eb3840b40 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0080fde740 10 client.0 _ll_register_callbacks cb 0x7f006c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007c9a1640 1 -- 172.21.15.19:0/654784476 learned_addr learned my addr 172.21.15.19:0/654784476 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:20.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0077fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3852c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3843910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 -- 172.21.15.19:0/654784476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 msgr2=0x559eb3852c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3852c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 -- 172.21.15.19:0/654784476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 msgr2=0x559eb3852720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007d1a2640 1 -- 172.21.15.19:0/654784476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559eb384cad0 con 0x559eb384a370 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f007c9a1640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.020+0000 7f0077fff640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3852c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f007d1a2640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3843910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0070009900 tx=0x7f007002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68133d836e0bc583 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:20.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0057fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0070051070 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0070003ac0 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f006809a160 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0070035070 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0080fde740 10 client.26582 fetch_fsmap learned FSMap version 48 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x559eb38c4b80 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.021+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0070005500 con 0x559eb384a370 2024-09-12T03:28:20.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 48) ==== 8398+0+0 (secure 0 0 0) 0x7f0070004a70 con 0x559eb384a370 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 10 client.26582 fetch_fsmap finished waiting for FSMap version 48 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 10 client.26582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 20 client.26582 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb38c7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 4 client.26582 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f006809a160 con 0x559eb385a480 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f007c9a1640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb38c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f0057fff640 10 client.26582 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.022+0000 7f007c9a1640 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb38c7370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=22acb32de59cba84 server_cookie=24cff0b38c481772 in_seq=0 out_seq=0 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0057fff640 1 -- 172.21.15.19:0/654784476 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f006809a160 con 0x559eb385a480 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0057fff640 10 client.26582 handle_command_reply: tid=0 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 client.26582 shutdown 2024-09-12T03:28:20.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 msgr2=0x559eb3843910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3843910 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0070009900 tx=0x7f007002a420 comp rx=0 tx=0).stop 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 shutdown_connections 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559eb385a480 0x559eb38c7370 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559eb384a370 0x559eb3843910 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559eb3840b40 0x559eb3852c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0057fff640 0 client.26582 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 --2- 172.21.15.19:0/654784476 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559eb383f5d0 0x559eb3852720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 >> 172.21.15.19:0/654784476 conn(0x559eb3808310 msgr2=0x559eb383b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 shutdown_connections 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.023+0000 7f0080fde740 1 -- 172.21.15.19:0/654784476 wait complete. 2024-09-12T03:28:20.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.024+0000 7f0080fde740 20 client.26582 trim_cache size 0 max 16384 2024-09-12T03:28:20.029 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.024+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 msgr2=0x7f00600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.024+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 0x7f00600a50c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0070006010 tx=0x7f0070009260 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.024+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 msgr2=0x7f00781a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.024+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f006800eab0 tx=0x7f006800ef70 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 shutdown_connections 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f00600a2c00 0x7f00600a50c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f00781082a0 0x7f007819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f00781036e0 0x7f00781a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 --2- 172.21.15.19:0/1775587150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0078102d30 0x7f00781a0410 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 >> 172.21.15.19:0/1775587150 conn(0x7f00780fc7d0 msgr2=0x7f00780fe690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:20.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 shutdown_connections 2024-09-12T03:28:20.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:20.025+0000 7f007ec05640 1 -- 172.21.15.19:0/1775587150 wait complete. 2024-09-12T03:28:20.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:20 smithi019 ceph-mon[30953]: pgmap v616: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 236 KiB/s wr, 34 op/s 2024-09-12T03:28:20.858 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:20 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:20.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:28:20.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:20 smithi121 ceph-mon[34963]: pgmap v616: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 236 KiB/s wr, 34 op/s 2024-09-12T03:28:20.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:20 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:21.014 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:20 smithi135 ceph-mon[35474]: pgmap v616: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 236 KiB/s wr, 34 op/s 2024-09-12T03:28:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:20 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 -- 172.21.15.19:0/1377246360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd410c150 msgr2=0x7efcd410e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 --2- 172.21.15.19:0/1377246360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd410c150 0x7efcd410e5f0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7efccc009ca0 tx=0x7efccc02a950 comp rx=0 tx=0).stop 2024-09-12T03:28:21.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 -- 172.21.15.19:0/1377246360 shutdown_connections 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 --2- 172.21.15.19:0/1377246360 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd410c150 0x7efcd410e5f0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 --2- 172.21.15.19:0/1377246360 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4103640 0x7efcd410bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 --2- 172.21.15.19:0/1377246360 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd4102d20 0x7efcd4103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 -- 172.21.15.19:0/1377246360 >> 172.21.15.19:0/1377246360 conn(0x7efcd40fc780 msgr2=0x7efcd40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 -- 172.21.15.19:0/1377246360 shutdown_connections 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.544+0000 7efcdc58c640 1 -- 172.21.15.19:0/1377246360 wait complete. 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 Processor -- start 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 -- start start 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 0x7efcd4100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd410c150 0x7efcd4101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4111fa0 con 0x7efcd4103640 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4111e20 con 0x7efcd4102d20 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdc58c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd4112120 con 0x7efcd410c150 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46780/0 (socket says 172.21.15.19:46780) 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 -- 172.21.15.19:0/2840950693 learned_addr learned my addr 172.21.15.19:0/2840950693 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcdab29640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd410c150 0x7efcd4101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcda328640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 0x7efcd4100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 -- 172.21.15.19:0/2840950693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd410c150 msgr2=0x7efcd4101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd410c150 0x7efcd4101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 -- 172.21.15.19:0/2840950693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 msgr2=0x7efcd4100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 0x7efcd4100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcd9b27640 1 -- 172.21.15.19:0/2840950693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd41a6900 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.545+0000 7efcda328640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 0x7efcd4100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d740 tx=0x7efcc400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efccb7fe640 1 -- 172.21.15.19:0/2840950693 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4017070 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd41a6bf0 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efccb7fe640 1 -- 172.21.15.19:0/2840950693 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc400de50 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efccb7fe640 1 -- 172.21.15.19:0/2840950693 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc400f620 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.546+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41a7130 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.548+0000 7efccb7fe640 1 -- 172.21.15.19:0/2840950693 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efcc4009c30 con 0x7efcd4103640 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.548+0000 7efcde965740 1 Processor -- start 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.548+0000 7efcde965740 1 -- start start 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 0x55caca0beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0c9940 0x55caca0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55caca0242a0 con 0x55caca0bf080 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55cac9fcbb10 con 0x55caca0c9940 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcde965740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55caca020260 con 0x55caca0be760 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcda328640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 0x55caca0beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcda328640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46794/0 (socket says 172.21.15.19:46794) 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcda328640 1 -- 172.21.15.19:0/3312606809 learned_addr learned my addr 172.21.15.19:0/3312606809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcd9b27640 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0c9940 0x55caca0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 -- 172.21.15.19:0/3312606809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0c9940 msgr2=0x55caca0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0c9940 0x55caca0cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 -- 172.21.15.19:0/3312606809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 msgr2=0x55caca0c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 -- 172.21.15.19:0/3312606809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cac9d95b90 con 0x55caca0be760 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcda328640 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efccb7fe640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 0x7efcc00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.549+0000 7efcdab29640 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 0x55caca0beb40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efccc005e30 tx=0x7efccc02f450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=722f5363395bc2a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcda328640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 0x7efcc00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcc8ff9640 1 -- 172.21.15.19:0/3312606809 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc048070 con 0x55caca0be760 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcc8ff9640 1 -- 172.21.15.19:0/3312606809 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc003e50 con 0x55caca0be760 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcc8ff9640 1 -- 172.21.15.19:0/3312606809 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc034040 con 0x55caca0be760 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efccb7fe640 1 -- 172.21.15.19:0/2840950693 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efcc40cebb0 con 0x7efcd4103640 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 -- 172.21.15.19:0/3312606809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 msgr2=0x55caca0beb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 0x55caca0beb40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efccc005e30 tx=0x7efccc02f450 comp rx=0 tx=0).stop 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcda328640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 0x7efcc00a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55caca0bbb50 tx=0x7efcd0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 -- 172.21.15.19:0/3312606809 shutdown_connections 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0c9940 0x55caca0cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 --2- 172.21.15.19:0/3312606809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0be760 0x55caca0beb40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.550+0000 7efcde965740 1 -- 172.21.15.19:0/3312606809 >> 172.21.15.19:0/3312606809 conn(0x55caca0bacf0 msgr2=0x55caca0bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- 172.21.15.19:0/3312606809 shutdown_connections 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- 172.21.15.19:0/3312606809 wait complete. 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- start start 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 Processor -- start 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55caca0cc510 con 0x55caca0bf080 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55cac9fac6e0 con 0x55caca0be760 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55caca0cc690 con 0x55caca0c9940 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcde965740 10 client.0 _ll_register_callbacks cb 0x55cac9fd0a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcda328640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcdab29640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcda328640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:59996/0 (socket says 172.21.15.19:59996) 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46798/0 (socket says 172.21.15.19:46798) 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.551+0000 7efcda328640 1 -- 172.21.15.19:0/1377463420 learned_addr learned my addr 172.21.15.19:0/1377463420 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:21.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 -- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 msgr2=0x55caca0c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 -- 172.21.15.19:0/1377463420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 msgr2=0x55caca0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 -- 172.21.15.19:0/1377463420 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caca0d5a80 con 0x55caca0be760 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcdab29640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcda328640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efcd0011ae0 tx=0x7efcd0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d17e319421e21c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd001e6e0 con 0x55caca0be760 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efcd001e880 con 0x55caca0be760 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efccc003e50 con 0x55caca0be760 2024-09-12T03:28:21.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efcd001ea80 con 0x55caca0be760 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcde965740 10 client.26594 fetch_fsmap learned FSMap version 49 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.552+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55caca0d84a0 con 0x55caca0be760 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efccc003e50 con 0x55caca0be760 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7efcd0026070 con 0x55caca0be760 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 10 client.26594 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 10 client.26594 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 20 client.26594 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca0d9930 0x55caca0dbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 4 client.26594 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caca0319d0 con 0x55caca0d9930 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca0d9930 0x55caca0dbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca0d9930 0x55caca0dbdd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8f6372bd56af62b server_cookie=ffbc2368b284cfcf in_seq=0 out_seq=0 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.553+0000 7efcbdffb640 10 client.26594 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcbdffb640 1 -- 172.21.15.19:0/1377463420 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55caca0319d0 con 0x55caca0d9930 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcbdffb640 10 client.26594 handle_command_reply: tid=0 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 client.26594 shutdown 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 msgr2=0x55caca0c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efcd0011ae0 tx=0x7efcd0004730 comp rx=0 tx=0).stop 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 shutdown_connections 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0bf080 0x55caca0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca0d9930 0x55caca0dbdd0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 --2- 172.21.15.19:0/1377463420 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0c6890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.554+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 >> 172.21.15.19:0/1377463420 conn(0x55caca0bacf0 msgr2=0x55caca0b9e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.555+0000 7efcbdffb640 0 client.26594 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.555+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 shutdown_connections 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.555+0000 7efcde965740 1 -- 172.21.15.19:0/1377463420 wait complete. 2024-09-12T03:28:21.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.555+0000 7efcde965740 20 client.26594 trim_cache size 0 max 16384 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 Processor -- start 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 -- start start 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0be760 0x55caca0c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0e16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41af6b0 con 0x55caca0be760 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41b1340 con 0x55caca0bf080 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcde965740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41afd80 con 0x55caca0c9940 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcda328640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0be760 0x55caca0c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60004/0 (socket says 172.21.15.19:60004) 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.573+0000 7efcd9b27640 1 -- 172.21.15.19:0/2112575189 learned_addr learned my addr 172.21.15.19:0/2112575189 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcdab29640 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0e16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 -- 172.21.15.19:0/2112575189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 msgr2=0x55caca0e16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0e16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 -- 172.21.15.19:0/2112575189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0be760 msgr2=0x55caca0c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0be760 0x55caca0c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 -- 172.21.15.19:0/2112575189 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caca0bbda0 con 0x55caca0bf080 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcdab29640 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcd9b27640 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcc40cc6c0 tx=0x7efcc4095f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85832e10b88016c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcbffff640 1 -- 172.21.15.19:0/2112575189 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4020040 con 0x55caca0bf080 2024-09-12T03:28:21.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcbffff640 1 -- 172.21.15.19:0/2112575189 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc409bb20 con 0x55caca0bf080 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.574+0000 7efcbffff640 1 -- 172.21.15.19:0/2112575189 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4099720 con 0x55caca0bf080 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 -- 172.21.15.19:0/2112575189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 msgr2=0x55caca0c1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efcc40cc6c0 tx=0x7efcc4095f90 comp rx=0 tx=0).stop 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 -- 172.21.15.19:0/2112575189 shutdown_connections 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0c9940 0x55caca0e16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0bf080 0x55caca0c1210 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 --2- 172.21.15.19:0/2112575189 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0be760 0x55caca0c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 -- 172.21.15.19:0/2112575189 >> 172.21.15.19:0/2112575189 conn(0x55caca0bacf0 msgr2=0x55caca0c8910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 -- 172.21.15.19:0/2112575189 shutdown_connections 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.575+0000 7efcde965740 1 -- 172.21.15.19:0/2112575189 wait complete. 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 -- start start 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 Processor -- start 2024-09-12T03:28:21.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0d1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 0x55caca0d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41b0bb0 con 0x55caca0c9940 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41b0480 con 0x55caca0be760 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efcd41ad690 con 0x55caca0bf080 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcde965740 10 client.0 _ll_register_callbacks cb 0x7efcd0011970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcdab29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcd9b27640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 0x55caca0d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcda328640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0d1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcdab29640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46808/0 (socket says 172.21.15.19:46808) 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.576+0000 7efcdab29640 1 -- 172.21.15.19:0/1695492021 learned_addr learned my addr 172.21.15.19:0/1695492021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 -- 172.21.15.19:0/1695492021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 msgr2=0x55caca0d1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0d1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 -- 172.21.15.19:0/1695492021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 msgr2=0x55caca0cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 -- 172.21.15.19:0/1695492021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caca0b6260 con 0x55caca0bf080 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcda328640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcdab29640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcd9b27640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 0x55caca0d2200 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcc40058c0 tx=0x7efcc409ae70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f4d75e7afbb356b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcbdffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc40b3070 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efcc4097a90 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcc409bb20 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7efcc4020040 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcde965740 10 client.26566 fetch_fsmap learned FSMap version 49 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55caca144130 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.577+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcc409b420 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 49) ==== 8394+0+0 (secure 0 0 0) 0x7efcc409a870 con 0x55caca0bf080 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 10 client.26566 fetch_fsmap finished waiting for FSMap version 49 2024-09-12T03:28:21.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 10 client.26566 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 20 client.26566 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca144900 0x55caca146d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 4 client.26566 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}] 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.4, "await_for": 45.0}) -- 0x7efcc409bb20 con 0x55caca144900 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcda328640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca144900 0x55caca146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcbdffb640 10 client.26566 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:21.588 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:21.578+0000 7efcda328640 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca144900 0x55caca146d20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c41ee60680baea0 server_cookie=3381d98e55dd86d3 in_seq=0 out_seq=0 2024-09-12T03:28:21.593 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:21.717 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:21 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:21.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:21 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:22.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:21 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:22.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.205+0000 7fdbaf30e640 1 -- 172.21.15.19:0/2924357936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8109b10 msgr2=0x7fdba8114570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.205+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/2924357936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8109b10 0x7fdba8114570 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009900 tx=0x7fdb9802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 -- 172.21.15.19:0/2924357936 shutdown_connections 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/2924357936 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba8114ab0 0x7fdba8116ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/2924357936 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8109b10 0x7fdba8114570 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/2924357936 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba81091f0 0x7fdba81095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 -- 172.21.15.19:0/2924357936 >> 172.21.15.19:0/2924357936 conn(0x7fdba806e140 msgr2=0x7fdba806e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 -- 172.21.15.19:0/2924357936 shutdown_connections 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.206+0000 7fdbaf30e640 1 -- 172.21.15.19:0/2924357936 wait complete. 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.207+0000 7fdbaf30e640 1 Processor -- start 2024-09-12T03:28:22.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.207+0000 7fdbaf30e640 1 -- start start 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.207+0000 7fdbaf30e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.207+0000 7fdbaf30e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba8109b10 0x7fdba81a6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.207+0000 7fdbaf30e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 0x7fdba81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40720/0 (socket says 172.21.15.19:40720) 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 -- 172.21.15.19:0/802871809 learned_addr learned my addr 172.21.15.19:0/802871809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 0x7fdba81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba8109b10 0x7fdba81a6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdba811a790 con 0x7fdba8114ab0 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba811a610 con 0x7fdba8109b10 2024-09-12T03:28:22.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdba811a910 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 -- 172.21.15.19:0/802871809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba8109b10 msgr2=0x7fdba81a6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba8109b10 0x7fdba81a6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 -- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 msgr2=0x7fdba81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 0x7fdba81a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.208+0000 7fdbad0aa640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba81a11d0 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 0x7fdba81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba400e9f0 tx=0x7fdba400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdb9e7fc640 1 -- 172.21.15.19:0/802871809 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba400cd80 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba81a14c0 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdb9e7fc640 1 -- 172.21.15.19:0/802871809 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba4010910 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba81ad7b0 con 0x7fdba81091f0 2024-09-12T03:28:22.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.209+0000 7fdb9e7fc640 1 -- 172.21.15.19:0/802871809 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba40188f0 con 0x7fdba81091f0 2024-09-12T03:28:22.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 Processor -- start 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 -- start start 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc43a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc50760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557abba92f0 con 0x5557abc4e370 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5557abb50cf0 con 0x5557abc44b40 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.212+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5557abba52b0 con 0x5557abc435d0 2024-09-12T03:28:22.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdb9e7fc640 1 -- 172.21.15.19:0/802871809 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdba4010ac0 con 0x7fdba81091f0 2024-09-12T03:28:22.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60036/0 (socket says 172.21.15.19:60036) 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 -- 172.21.15.19:0/1972112361 learned_addr learned my addr 172.21.15.19:0/1972112361 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc50760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc43a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 -- 172.21.15.19:0/1972112361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 msgr2=0x5557abc43a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc43a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 -- 172.21.15.19:0/1972112361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 msgr2=0x5557abc50760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc50760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.213+0000 7fdbad8ab640 1 -- 172.21.15.19:0/1972112361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557abb353a0 con 0x5557abc44b40 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005bf0 tx=0x7fdba0011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a40fdfa38f4b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdb9e7fc640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 0x7fdb940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdb937fe640 1 -- 172.21.15.19:0/1972112361 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003b70 con 0x5557abc44b40 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdb937fe640 1 -- 172.21.15.19:0/1972112361 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba001a070 con 0x5557abc44b40 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdb937fe640 1 -- 172.21.15.19:0/1972112361 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0004650 con 0x5557abc44b40 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdb9e7fc640 1 -- 172.21.15.19:0/802871809 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdba40cf370 con 0x7fdba81091f0 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.214+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 0x7fdb940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 -- 172.21.15.19:0/1972112361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 msgr2=0x5557abc43090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdba0005bf0 tx=0x7fdba0011c20 comp rx=0 tx=0).stop 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 -- 172.21.15.19:0/1972112361 shutdown_connections 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc50760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc43a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/1972112361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc43090 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 -- 172.21.15.19:0/1972112361 >> 172.21.15.19:0/1972112361 conn(0x5557abc0c310 msgr2=0x5557abc3fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbad0aa640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 0x7fdb940a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdba40cd1f0 tx=0x7fdba40d2e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 -- 172.21.15.19:0/1972112361 shutdown_connections 2024-09-12T03:28:22.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.215+0000 7fdbb16e7740 1 -- 172.21.15.19:0/1972112361 wait complete. 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 -- start start 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 Processor -- start 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 0x5557abc4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc4e370 0x5557abc4b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557abc51070 con 0x5557abc435d0 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5557abb64060 con 0x5557abc44b40 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.216+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5557abc511f0 con 0x5557abc4e370 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbb16e7740 10 client.0 _ll_register_callbacks cb 0x5557ab8b78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60038/0 (socket says 172.21.15.19:60038) 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 -- 172.21.15.19:0/3492998412 learned_addr learned my addr 172.21.15.19:0/3492998412 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc4e370 0x5557abc4b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 0x5557abc4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 -- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc4e370 msgr2=0x5557abc4b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc4e370 0x5557abc4b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 -- 172.21.15.19:0/3492998412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 msgr2=0x5557abc4dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 0x5557abc4dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbac8a9640 1 -- 172.21.15.19:0/3492998412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ab91a3c0 con 0x5557abc44b40 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.217+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 0x5557abc4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.218+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb9802a9a0 tx=0x7fdb980317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bddb69022842c83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.218+0000 7fdb90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.218+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035040 con 0x5557abc44b40 2024-09-12T03:28:22.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.218+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdb98004240 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.218+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba001a070 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdb98004440 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 10 client.26612 fetch_fsmap learned FSMap version 50 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5557abc652e0 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba001a070 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 50) ==== 8398+0+0 (secure 0 0 0) 0x7fdb98004b80 con 0x5557abc44b40 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 10 client.26612 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 10 client.26612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 20 client.26612 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abc667f0 0x5557abc68be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 4 client.26612 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557abbb6860 con 0x5557abc667f0 2024-09-12T03:28:22.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.219+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abc667f0 0x5557abc68be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.220+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abc667f0 0x5557abc68be0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3a5f947bc8d60d0 server_cookie=5b65815edc8c66cf in_seq=0 out_seq=0 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.220+0000 7fdb90ff9640 10 client.26612 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdb90ff9640 1 -- 172.21.15.19:0/3492998412 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557abbb6860 con 0x5557abc667f0 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdb90ff9640 10 client.26612 handle_command_reply: tid=0 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 client.26612 shutdown 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 msgr2=0x5557abc4b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fdb9802a9a0 tx=0x7fdb980317e0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 shutdown_connections 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc4e370 0x5557abc4b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abc667f0 0x5557abc68be0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc4b000 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/3492998412 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc435d0 0x5557abc4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdb90ff9640 0 client.26612 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:22.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 >> 172.21.15.19:0/3492998412 conn(0x5557abc0c310 msgr2=0x5557abc45960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 shutdown_connections 2024-09-12T03:28:22.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 1 -- 172.21.15.19:0/3492998412 wait complete. 2024-09-12T03:28:22.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.221+0000 7fdbb16e7740 20 client.26612 trim_cache size 0 max 16384 2024-09-12T03:28:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.234+0000 7fdbb16e7740 1 Processor -- start 2024-09-12T03:28:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.234+0000 7fdbb16e7740 1 -- start start 2024-09-12T03:28:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc62ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc4e370 0x5557abc46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b3ba0 con 0x5557abc44b40 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b5830 con 0x5557abc4e370 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b4270 con 0x5557abc435d0 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46826/0 (socket says 172.21.15.19:46826) 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 -- 172.21.15.19:0/247529981 learned_addr learned my addr 172.21.15.19:0/247529981 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc62ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc4e370 0x5557abc46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 -- 172.21.15.19:0/247529981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 msgr2=0x5557abc62ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc62ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 -- 172.21.15.19:0/247529981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc4e370 msgr2=0x5557abc46660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc4e370 0x5557abc46660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.235+0000 7fdbac8a9640 1 -- 172.21.15.19:0/247529981 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557abc4d970 con 0x5557abc44b40 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.236+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005940 tx=0x7fdb9802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bcaa21279b1bd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.236+0000 7fdb92ffd640 1 -- 172.21.15.19:0/247529981 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98035070 con 0x5557abc44b40 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.236+0000 7fdb92ffd640 1 -- 172.21.15.19:0/247529981 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb98004fa0 con 0x5557abc44b40 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.236+0000 7fdb92ffd640 1 -- 172.21.15.19:0/247529981 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb98000e60 con 0x5557abc44b40 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- 172.21.15.19:0/247529981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 msgr2=0x5557abc46120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fdb98005940 tx=0x7fdb9802a480 comp rx=0 tx=0).stop 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- 172.21.15.19:0/247529981 shutdown_connections 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc4e370 0x5557abc46660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc44b40 0x5557abc46120 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/247529981 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc62ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- 172.21.15.19:0/247529981 >> 172.21.15.19:0/247529981 conn(0x5557abc0c310 msgr2=0x5557abc4a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- 172.21.15.19:0/247529981 shutdown_connections 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- 172.21.15.19:0/247529981 wait complete. 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 -- start start 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.237+0000 7fdbb16e7740 1 Processor -- start 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc5a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b50a0 con 0x5557abc4e370 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b4970 con 0x5557abc44b40 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fdba81b1b70 con 0x5557abc435d0 2024-09-12T03:28:22.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbb16e7740 10 client.0 _ll_register_callbacks cb 0x7fdb98009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbac8a9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60070/0 (socket says 172.21.15.19:60070) 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbac8a9640 1 -- 172.21.15.19:0/2723887249 learned_addr learned my addr 172.21.15.19:0/2723887249 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc5a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 -- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 msgr2=0x5557abc5a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc5a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 -- 172.21.15.19:0/2723887249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 msgr2=0x5557abc5a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbad8ab640 1 -- 172.21.15.19:0/2723887249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557abc5f1a0 con 0x5557abc4e370 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.238+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdbad8ab640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc5eb30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdba0011760 tx=0x7fdb98009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec715ca0db2eee35 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:22.242 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba0003b70 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb98004fa0 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fdba0003d10 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fdba0003f10 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdbb16e7740 10 client.17223 fetch_fsmap learned FSMap version 50 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5557abcd0fe0 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb98004fa0 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.239+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 50) ==== 8398+0+0 (secure 0 0 0) 0x7fdba0035430 con 0x5557abc4e370 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 10 client.17223 fetch_fsmap finished waiting for FSMap version 50 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 10 client.17223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 20 client.17223 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:22.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abcd14c0 0x5557abcd38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 4 client.17223 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:28:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557abbb0d70 con 0x5557abcd14c0 2024-09-12T03:28:22.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abcd14c0 0x5557abcd38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdb90ff9640 10 client.17223 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbac8a9640 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abcd14c0 0x5557abcd38e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7364f865864d7c73 server_cookie=4d5e1eca06f79a59 in_seq=0 out_seq=0 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdb90ff9640 1 -- 172.21.15.19:0/2723887249 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557abbb0d70 con 0x5557abcd14c0 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdb90ff9640 10 client.17223 handle_command_reply: tid=0 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.240+0000 7fdbb16e7740 1 client.17223 shutdown 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 msgr2=0x5557abc5eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.245 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc5eb30 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fdba0011760 tx=0x7fdb98009610 comp rx=0 tx=0).stop 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 shutdown_connections 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557abc4e370 0x5557abc5eb30 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5557abcd14c0 0x5557abcd38e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557abc44b40 0x5557abc5a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 --2- 172.21.15.19:0/2723887249 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557abc435d0 0x5557abc5a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 >> 172.21.15.19:0/2723887249 conn(0x5557abc0c310 msgr2=0x5557abc4c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdb90ff9640 0 client.17223 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 shutdown_connections 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 1 -- 172.21.15.19:0/2723887249 wait complete. 2024-09-12T03:28:22.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.241+0000 7fdbb16e7740 20 client.17223 trim_cache size 0 max 16384 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 msgr2=0x7fdb940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 0x7fdb940a50c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdba40cd1f0 tx=0x7fdba40d2e50 comp rx=0 tx=0).stop 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 msgr2=0x7fdba81a68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdba400e9f0 tx=0x7fdba400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 shutdown_connections 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fdb940a2c00 0x7fdb940a50c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fdba8114ab0 0x7fdba81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdba8109b10 0x7fdba81a6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 --2- 172.21.15.19:0/802871809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fdba81091f0 0x7fdba81a68e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 >> 172.21.15.19:0/802871809 conn(0x7fdba806e140 msgr2=0x7fdba8071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 shutdown_connections 2024-09-12T03:28:22.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:22.242+0000 7fdbaf30e640 1 -- 172.21.15.19:0/802871809 wait complete. 2024-09-12T03:28:22.807 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:22 smithi019 ceph-mon[30953]: pgmap v617: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 547 KiB/s wr, 126 op/s 2024-09-12T03:28:22.808 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:22 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:22.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:22 smithi121 ceph-mon[34963]: pgmap v617: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 547 KiB/s wr, 126 op/s 2024-09-12T03:28:22.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:22 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:22 smithi135 ceph-mon[35474]: pgmap v617: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 493 KiB/s rd, 547 KiB/s wr, 126 op/s 2024-09-12T03:28:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:22 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:24.313 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:28:24.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:28:24.587 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:24.884 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:24 smithi019 ceph-mon[30953]: pgmap v618: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 547 KiB/s wr, 170 op/s 2024-09-12T03:28:24.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:24 smithi121 ceph-mon[34963]: pgmap v618: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 547 KiB/s wr, 170 op/s 2024-09-12T03:28:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:24 smithi135 ceph-mon[35474]: pgmap v618: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 547 KiB/s wr, 170 op/s 2024-09-12T03:28:25.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 -- 172.21.15.19:0/2606660696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858101420 msgr2=0x7ff85810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 --2- 172.21.15.19:0/2606660696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858101420 0x7ff85810d950 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff850009cd0 tx=0x7ff85002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 -- 172.21.15.19:0/2606660696 shutdown_connections 2024-09-12T03:28:25.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 --2- 172.21.15.19:0/2606660696 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858101420 0x7ff85810d950 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 --2- 172.21.15.19:0/2606660696 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858100a80 0x7ff858100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 --2- 172.21.15.19:0/2606660696 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858106a80 0x7ff858106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 -- 172.21.15.19:0/2606660696 >> 172.21.15.19:0/2606660696 conn(0x7ff8580fc7a0 msgr2=0x7ff8580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 -- 172.21.15.19:0/2606660696 shutdown_connections 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.074+0000 7ff86023a640 1 -- 172.21.15.19:0/2606660696 wait complete. 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.075+0000 7ff86023a640 1 Processor -- start 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.075+0000 7ff86023a640 1 -- start start 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 0x7ff8581972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858101420 0x7ff858197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff858110e80 con 0x7ff858106a80 2024-09-12T03:28:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff858110d00 con 0x7ff858101420 2024-09-12T03:28:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff86023a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff858111000 con 0x7ff858100a80 2024-09-12T03:28:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46846/0 (socket says 172.21.15.19:46846) 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 -- 172.21.15.19:0/3742099825 learned_addr learned my addr 172.21.15.19:0/3742099825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858101420 0x7ff858197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 0x7ff8581972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 -- 172.21.15.19:0/3742099825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 msgr2=0x7ff8581972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 0x7ff8581972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 -- 172.21.15.19:0/3742099825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858101420 msgr2=0x7ff858197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858101420 0x7ff858197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 -- 172.21.15.19:0/3742099825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85819c290 con 0x7ff858106a80 2024-09-12T03:28:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 0x7ff8581972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff85e7d7640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff850005cc0 tx=0x7ff8500093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.095 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.076+0000 7ff84effd640 1 -- 172.21.15.19:0/3742099825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850003e00 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.077+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff85819c520 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.077+0000 7ff84effd640 1 -- 172.21.15.19:0/3742099825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff850004520 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.077+0000 7ff84effd640 1 -- 172.21.15.19:0/3742099825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850030db0 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.077+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff85806a790 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.079+0000 7ff84effd640 1 -- 172.21.15.19:0/3742099825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff8500046d0 con 0x7ff858106a80 2024-09-12T03:28:25.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.079+0000 7ff862613740 1 Processor -- start 2024-09-12T03:28:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.079+0000 7ff862613740 1 -- start start 2024-09-12T03:28:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 0x55f4dc403c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc404150 0x55f4dc406540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc35a280 con 0x55f4dc3f5360 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc301b10 con 0x55f4dc404150 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff862613740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc356240 con 0x55f4dc3f4a40 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85dfd6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 0x55f4dc403c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:40780/0 (socket says 172.21.15.19:40780) 2024-09-12T03:28:25.098 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 -- 172.21.15.19:0/4155436342 learned_addr learned my addr 172.21.15.19:0/4155436342 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff84effd640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 0x7ff8440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc404150 0x55f4dc406540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85e7d7640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 0x7ff8440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 -- 172.21.15.19:0/4155436342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc404150 msgr2=0x55f4dc406540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc404150 0x55f4dc406540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.099 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 -- 172.21.15.19:0/4155436342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 msgr2=0x55f4dc403c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 0x55f4dc403c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 -- 172.21.15.19:0/4155436342 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dc30e1f0 con 0x55f4dc3f4a40 2024-09-12T03:28:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 0x55f4dc403c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff84effd640 1 -- 172.21.15.19:0/3742099825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff85002e030 con 0x7ff858106a80 2024-09-12T03:28:25.100 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.080+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff848009fd0 tx=0x7ff848005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e642ac9aad9cb27c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff843fff640 1 -- 172.21.15.19:0/4155436342 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004050 con 0x55f4dc3f4a40 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff85e7d7640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 0x7ff8440a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff850009ca0 tx=0x7ff850090070 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff843fff640 1 -- 172.21.15.19:0/4155436342 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848005500 con 0x55f4dc3f4a40 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff843fff640 1 -- 172.21.15.19:0/4155436342 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004d00 con 0x55f4dc3f4a40 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 -- 172.21.15.19:0/4155436342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 msgr2=0x55f4dc3f4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.101 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff848009fd0 tx=0x7ff848005850 comp rx=0 tx=0).stop 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 -- 172.21.15.19:0/4155436342 shutdown_connections 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc404150 0x55f4dc406540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f5360 0x55f4dc403c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 --2- 172.21.15.19:0/4155436342 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc3f4e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 -- 172.21.15.19:0/4155436342 >> 172.21.15.19:0/4155436342 conn(0x55f4dc3f10f0 msgr2=0x55f4dc3f14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.102 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 -- 172.21.15.19:0/4155436342 shutdown_connections 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.081+0000 7ff862613740 1 -- 172.21.15.19:0/4155436342 wait complete. 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 -- start start 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 Processor -- start 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f4a40 0x55f4dc401030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 0x55f4dc401570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.103 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc406d20 con 0x55f4dc404150 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc314f10 con 0x55f4dc3f4a40 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f4dc406ea0 con 0x55f4dc3f5360 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff862613740 10 client.0 _ll_register_callbacks cb 0x55f4dc2dda40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.104 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff85dfd6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 0x55f4dc401570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46870/0 (socket says 172.21.15.19:46870) 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.082+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3374058234 learned_addr learned my addr 172.21.15.19:0/3374058234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f4a40 0x55f4dc401030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 -- 172.21.15.19:0/3374058234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f4a40 msgr2=0x55f4dc401030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f4a40 0x55f4dc401030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.105 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 -- 172.21.15.19:0/3374058234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 msgr2=0x55f4dc401ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 -- 172.21.15.19:0/3374058234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dc0cb6a0 con 0x55f4dc3f5360 2024-09-12T03:28:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 0x55f4dc401570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff854013760 tx=0x7ff854013c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec637b8a3cfd1880 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.106 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff8417fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff854011dd0 con 0x55f4dc3f5360 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff854005020 con 0x55f4dc3f5360 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff848005500 con 0x55f4dc3f5360 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff854005220 con 0x55f4dc3f5360 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff862613740 10 client.26596 fetch_fsmap learned FSMap version 51 2024-09-12T03:28:25.107 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.083+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f4dc412a10 con 0x55f4dc3f5360 2024-09-12T03:28:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848005500 con 0x55f4dc3f5360 2024-09-12T03:28:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 51) ==== 8410+0+0 (secure 0 0 0) 0x7ff85401a030 con 0x55f4dc3f5360 2024-09-12T03:28:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 10 client.26596 fetch_fsmap finished waiting for FSMap version 51 2024-09-12T03:28:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 10 client.26596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:25.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 20 client.26596 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc413f90 0x55f4dc4163b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 4 client.26596 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4dc3677f0 con 0x55f4dc413f90 2024-09-12T03:28:25.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc413f90 0x55f4dc4163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc413f90 0x55f4dc4163b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796ed171cf37a5c2 server_cookie=cfc2c708ba2a478e in_seq=0 out_seq=0 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.084+0000 7ff8417fa640 10 client.26596 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff8417fa640 1 -- 172.21.15.19:0/3374058234 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4dc3677f0 con 0x55f4dc413f90 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff8417fa640 10 client.26596 handle_command_reply: tid=0 2024-09-12T03:28:25.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 client.26596 shutdown 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 msgr2=0x55f4dc401570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 0x55f4dc401570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff854013760 tx=0x7ff854013c20 comp rx=0 tx=0).stop 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 shutdown_connections 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc401ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f5360 0x55f4dc401570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc413f90 0x55f4dc4163b0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 --2- 172.21.15.19:0/3374058234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f4a40 0x55f4dc401030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff8417fa640 0 client.26596 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 >> 172.21.15.19:0/3374058234 conn(0x55f4dc3f10f0 msgr2=0x55f4dc3f6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 shutdown_connections 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 1 -- 172.21.15.19:0/3374058234 wait complete. 2024-09-12T03:28:25.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.085+0000 7ff862613740 20 client.26596 trim_cache size 0 max 16384 2024-09-12T03:28:25.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 Processor -- start 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 -- start start 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc403250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc401900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581ab220 con 0x55f4dc404150 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581acf40 con 0x55f4dc3f5360 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff862613740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581ab8f0 con 0x55f4dc3f4a40 2024-09-12T03:28:25.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46880/0 (socket says 172.21.15.19:46880) 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3735674193 learned_addr learned my addr 172.21.15.19:0/3735674193 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc401900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc403250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3735674193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 msgr2=0x55f4dc403250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc403250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3735674193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 msgr2=0x55f4dc401900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc401900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3735674193 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4dc3f0f20 con 0x55f4dc404150 2024-09-12T03:28:25.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.104+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc401900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:25.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff848005b90 tx=0x7ff848009b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe42968e124e47a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff8437fe640 1 -- 172.21.15.19:0/3735674193 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848034030 con 0x55f4dc404150 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff8437fe640 1 -- 172.21.15.19:0/3735674193 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848004ba0 con 0x55f4dc404150 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff8437fe640 1 -- 172.21.15.19:0/3735674193 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84800fb90 con 0x55f4dc404150 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff862613740 1 -- 172.21.15.19:0/3735674193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 msgr2=0x55f4dc41bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff862613740 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7ff848005b90 tx=0x7ff848009b60 comp rx=0 tx=0).stop 2024-09-12T03:28:25.121 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.105+0000 7ff862613740 1 -- 172.21.15.19:0/3735674193 shutdown_connections 2024-09-12T03:28:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc404150 0x55f4dc41bc80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc401900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- 172.21.15.19:0/3735674193 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc3f4a40 0x55f4dc403250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- 172.21.15.19:0/3735674193 >> 172.21.15.19:0/3735674193 conn(0x55f4dc3f10f0 msgr2=0x55f4dc3f6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- 172.21.15.19:0/3735674193 shutdown_connections 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- 172.21.15.19:0/3735674193 wait complete. 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- start start 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 Processor -- start 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc404150 0x55f4dc408960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581ac7b0 con 0x55f4dc3f4a40 2024-09-12T03:28:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581ac080 con 0x55f4dc3f5360 2024-09-12T03:28:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.106+0000 7ff862613740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff8581a91d0 con 0x55f4dc404150 2024-09-12T03:28:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff862613740 10 client.0 _ll_register_callbacks cb 0x7ff854009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:25.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85dfd6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46894/0 (socket says 172.21.15.19:46894) 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3618587477 learned_addr learned my addr 172.21.15.19:0/3618587477 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85dfd6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:60112/0 (socket says 172.21.15.19:60112) 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc404150 0x55f4dc408960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc404150 msgr2=0x55f4dc408960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc404150 0x55f4dc408960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3618587477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 msgr2=0x55f4dc408420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 -- 172.21.15.19:0/3618587477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4dc47dc50 con 0x55f4dc3f4a40 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:25.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff85d7d5640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff848009b10 tx=0x7ff8480097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45c4ef7df1b887a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff8417fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848034070 con 0x55f4dc3f4a40 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff848004d50 con 0x55f4dc3f4a40 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff848004ba0 con 0x55f4dc3f4a40 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.107+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7ff84800fb10 con 0x55f4dc3f4a40 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 10 client.17241 fetch_fsmap learned FSMap version 51 2024-09-12T03:28:25.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55f4dc47ef70 con 0x55f4dc3f4a40 2024-09-12T03:28:25.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff848004ba0 con 0x55f4dc3f4a40 2024-09-12T03:28:25.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 51) ==== 8410+0+0 (secure 0 0 0) 0x7ff848045020 con 0x55f4dc3f4a40 2024-09-12T03:28:25.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 10 client.17241 fetch_fsmap finished waiting for FSMap version 51 2024-09-12T03:28:25.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 10 client.17241 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:25.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 20 client.17241 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc47f4d0 0x55f4dc4818f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 4 client.17241 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f4dc361d00 con 0x55f4dc47f4d0 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.108+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc47f4d0 0x55f4dc4818f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.109+0000 7ff85dfd6640 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc47f4d0 0x55f4dc4818f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a76e5203dc40ce server_cookie=717a82bffe53c825 in_seq=0 out_seq=0 2024-09-12T03:28:25.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.109+0000 7ff8417fa640 10 client.17241 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.109+0000 7ff8417fa640 1 -- 172.21.15.19:0/3618587477 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f4dc361d00 con 0x55f4dc47f4d0 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.109+0000 7ff8417fa640 10 client.17241 handle_command_reply: tid=0 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.109+0000 7ff862613740 1 client.17241 shutdown 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 msgr2=0x55f4dc407ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ff848009b10 tx=0x7ff8480097a0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 shutdown_connections 2024-09-12T03:28:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f4dc404150 0x55f4dc408960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f4dc47f4d0 0x55f4dc4818f0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f4dc3f5360 0x55f4dc408420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff8417fa640 0 client.17241 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:25.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 --2- 172.21.15.19:0/3618587477 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f4dc3f4a40 0x55f4dc407ee0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 >> 172.21.15.19:0/3618587477 conn(0x55f4dc3f10f0 msgr2=0x55f4dc3efa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 shutdown_connections 2024-09-12T03:28:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 1 -- 172.21.15.19:0/3618587477 wait complete. 2024-09-12T03:28:25.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.110+0000 7ff862613740 20 client.17241 trim_cache size 0 max 16384 2024-09-12T03:28:25.133 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:25.133 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:28:25.133 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "f1c6db3f-f8a4-445f-b472-d756f2b12a3e", 2024-09-12T03:28:25.133 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:28:25.134 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 msgr2=0x7ff8440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 0x7ff8440a5040 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff850009ca0 tx=0x7ff850090070 comp rx=0 tx=0).stop 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 msgr2=0x7ff85819bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7ff850005cc0 tx=0x7ff8500093d0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 shutdown_connections 2024-09-12T03:28:25.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff8440a2b80 0x7ff8440a5040 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff858106a80 0x7ff85819bbb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff858101420 0x7ff858197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 --2- 172.21.15.19:0/3742099825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff858100a80 0x7ff8581972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 >> 172.21.15.19:0/3742099825 conn(0x7ff8580fc7a0 msgr2=0x7ff8580fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 shutdown_connections 2024-09-12T03:28:25.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:25.111+0000 7ff86023a640 1 -- 172.21.15.19:0/3742099825 wait complete. 2024-09-12T03:28:25.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:25 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:25.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:25 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:28:25.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:25 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:25.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:25 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:25.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:25 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:25 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:25 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:28:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:25 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:25 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:25 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:28:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:25 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:25 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:28:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:25 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:25 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:25 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:26.346 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:26.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:26.792 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: pgmap v619: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T03:28:26.793 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:26 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:26.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:26.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: pgmap v619: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T03:28:26.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:26 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:26.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 -- 172.21.15.19:0/2502653597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40100e20 msgr2=0x7f5e40101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 --2- 172.21.15.19:0/2502653597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40100e20 0x7f5e40101200 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009900 tx=0x7f5e3002a480 comp rx=0 tx=0).stop 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 -- 172.21.15.19:0/2502653597 shutdown_connections 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 --2- 172.21.15.19:0/2502653597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 0x7f5e4010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 --2- 172.21.15.19:0/2502653597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40102570 0x7f5e401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 --2- 172.21.15.19:0/2502653597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40100e20 0x7f5e40101200 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 -- 172.21.15.19:0/2502653597 >> 172.21.15.19:0/2502653597 conn(0x7f5e400fc7d0 msgr2=0x7f5e400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 -- 172.21.15.19:0/2502653597 shutdown_connections 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.880+0000 7f5e467b5640 1 -- 172.21.15.19:0/2502653597 wait complete. 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 Processor -- start 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 -- start start 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40100e20 0x7f5e40103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 0x7f5e401041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40110fd0 con 0x7f5e40102570 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.881+0000 7f5e467b5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40110e50 con 0x7f5e40102ef0 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e467b5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e40111150 con 0x7f5e40100e20 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3ffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40100e20 0x7f5e40103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46926/0 (socket says 172.21.15.19:46926) 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/4230783109 learned_addr learned my addr 172.21.15.19:0/4230783109 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e44d52640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 0x7f5e401041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/4230783109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40100e20 msgr2=0x7f5e40103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40100e20 0x7f5e40103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/4230783109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 msgr2=0x7f5e401041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 0x7f5e401041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/4230783109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e401a6940 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e9f0 tx=0x7f5e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3d7fa640 1 -- 172.21.15.19:0/4230783109 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400cd80 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3d7fa640 1 -- 172.21.15.19:0/4230783109 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e34010910 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e3d7fa640 1 -- 172.21.15.19:0/4230783109 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34018990 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e401a6c30 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.882+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e401a7170 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.885+0000 7f5e3d7fa640 1 -- 172.21.15.19:0/4230783109 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e34010ac0 con 0x7f5e40102570 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.885+0000 7f5e48b8e740 1 Processor -- start 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.885+0000 7f5e48b8e740 1 -- start start 2024-09-12T03:28:26.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a6f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a6f680 0x55dce3a71a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dce39ca2e0 con 0x55dce3a644a0 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dce3971ce0 con 0x55dce3a64dc0 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dce39c62a0 con 0x55dce3a6f680 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e44d52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a6f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e44d52640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46930/0 (socket says 172.21.15.19:46930) 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e44d52640 1 -- 172.21.15.19:0/2045118939 learned_addr learned my addr 172.21.15.19:0/2045118939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a6f680 0x55dce3a71a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 -- 172.21.15.19:0/2045118939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a6f680 msgr2=0x55dce3a71a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a6f680 0x55dce3a71a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 -- 172.21.15.19:0/2045118939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 msgr2=0x55dce3a64880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 -- 172.21.15.19:0/2045118939 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce3956440 con 0x55dce3a64dc0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e44d52640 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3d7fa640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 0x7f5e280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a6f140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002aa70 tx=0x7f5e30005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31bd1ff82833744 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.886+0000 7f5e44d52640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 0x7f5e280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e3d7fa640 1 -- 172.21.15.19:0/4230783109 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5e340cf320 con 0x7f5e40102570 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e267fc640 1 -- 172.21.15.19:0/2045118939 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30035040 con 0x55dce3a64dc0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e267fc640 1 -- 172.21.15.19:0/2045118939 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e30004400 con 0x55dce3a64dc0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e267fc640 1 -- 172.21.15.19:0/2045118939 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e300035f0 con 0x55dce3a64dc0 2024-09-12T03:28:26.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e48b8e740 1 -- 172.21.15.19:0/2045118939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 msgr2=0x55dce3a6f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e44d52640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 0x7f5e280a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e38009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.887+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a6f140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002aa70 tx=0x7f5e30005e00 comp rx=0 tx=0).stop 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 -- 172.21.15.19:0/2045118939 shutdown_connections 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a6f680 0x55dce3a71a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a6f140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/2045118939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 -- 172.21.15.19:0/2045118939 >> 172.21.15.19:0/2045118939 conn(0x55dce3a60a30 msgr2=0x55dce3a60e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 -- 172.21.15.19:0/2045118939 shutdown_connections 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 -- 172.21.15.19:0/2045118939 wait complete. 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 -- start start 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.888+0000 7f5e48b8e740 1 Processor -- start 2024-09-12T03:28:26.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a6ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a6f680 0x55dce3a6c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dce3a72250 con 0x55dce3a64dc0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dce3985050 con 0x55dce3a6f680 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dce3a723d0 con 0x55dce3a644a0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e48b8e740 10 client.0 _ll_register_callbacks cb 0x55dce39c5160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46936/0 (socket says 172.21.15.19:46936) 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/588853518 learned_addr learned my addr 172.21.15.19:0/588853518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e44d52640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a6f680 0x55dce3a6c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a6ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.909 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 msgr2=0x55dce3a6ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a6ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/588853518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a6f680 msgr2=0x55dce3a6c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a6f680 0x55dce3a6c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/588853518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce373b3c0 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5e3409b940 tx=0x7f5e3409b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6bc42c508ad167 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.889+0000 7f5e1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3409ff40 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5e340d2c50 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e30004400 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e340d2e50 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 10 client.17253 fetch_fsmap learned FSMap version 52 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55dce3a7dba0 con 0x55dce3a64dc0 2024-09-12T03:28:26.910 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e30004400 con 0x55dce3a64dc0 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 8418+0+0 (secure 0 0 0) 0x7f5e340ce050 con 0x55dce3a64dc0 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 10 client.17253 fetch_fsmap finished waiting for FSMap version 52 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 20 client.17253 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a7f050 0x55dce3a81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 4 client.17253 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.890+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dce39d7850 con 0x55dce3a7f050 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.891+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a7f050 0x55dce3a81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.891+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a7f050 0x55dce3a81460 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53dfbb1b3b6c8515 server_cookie=bf7bbd4f2ba73916 in_seq=0 out_seq=0 2024-09-12T03:28:26.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.891+0000 7f5e1f7fe640 10 client.17253 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.906+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/588853518 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dce39d7850 con 0x55dce3a7f050 2024-09-12T03:28:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.906+0000 7f5e1f7fe640 10 client.17253 handle_command_reply: tid=0 2024-09-12T03:28:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.906+0000 7f5e48b8e740 1 client.17253 shutdown 2024-09-12T03:28:26.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 msgr2=0x55dce3a6c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5e3409b940 tx=0x7f5e3409b780 comp rx=0 tx=0).stop 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 shutdown_connections 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a7f050 0x55dce3a81460 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a6f680 0x55dce3a6c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a64dc0 0x55dce3a6c1e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/588853518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a6ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e1f7fe640 0 client.17253 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 >> 172.21.15.19:0/588853518 conn(0x55dce3a60a30 msgr2=0x55dce3a66760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 shutdown_connections 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 1 -- 172.21.15.19:0/588853518 wait complete. 2024-09-12T03:28:26.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.907+0000 7f5e48b8e740 20 client.17253 trim_cache size 0 max 16384 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 Processor -- start 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 -- start start 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a60a30 0x55dce3a6c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 0x55dce3a86d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401ab450 con 0x55dce3a644a0 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401ad1d0 con 0x55dce3a60a30 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401abb20 con 0x55dce3a64dc0 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46948/0 (socket says 172.21.15.19:46948) 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/3517611671 learned_addr learned my addr 172.21.15.19:0/3517611671 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a60a30 0x55dce3a6c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e44d52640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 0x55dce3a86d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.928+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/3517611671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 msgr2=0x55dce3a86d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 0x55dce3a86d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/3517611671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a60a30 msgr2=0x55dce3a6c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a60a30 0x55dce3a6c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e3f7fe640 1 -- 172.21.15.19:0/3517611671 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dce3a60860 con 0x55dce3a644a0 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e44d52640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 0x55dce3a86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:26.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5e340980a0 tx=0x7f5e3409bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c983a34f1ca20107 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e25ffb640 1 -- 172.21.15.19:0/3517611671 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e340b9070 con 0x55dce3a644a0 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e25ffb640 1 -- 172.21.15.19:0/3517611671 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3409a200 con 0x55dce3a644a0 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.929+0000 7f5e25ffb640 1 -- 172.21.15.19:0/3517611671 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3409ab00 con 0x55dce3a644a0 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- 172.21.15.19:0/3517611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 msgr2=0x55dce3a6e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5e340980a0 tx=0x7f5e3409bdc0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.934 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- 172.21.15.19:0/3517611671 shutdown_connections 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a64dc0 0x55dce3a86d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a644a0 0x55dce3a6e150 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/3517611671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a60a30 0x55dce3a6c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- 172.21.15.19:0/3517611671 >> 172.21.15.19:0/3517611671 conn(0x55dce3a6c880 msgr2=0x55dce3a6da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- 172.21.15.19:0/3517611671 shutdown_connections 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- 172.21.15.19:0/3517611671 wait complete. 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.930+0000 7f5e48b8e740 1 -- start start 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 Processor -- start 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a75b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a76070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401aca40 con 0x55dce3a60a30 2024-09-12T03:28:26.935 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401ac2b0 con 0x55dce3a64dc0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5e401a9460 con 0x55dce3a644a0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e48b8e740 10 client.0 _ll_register_callbacks cb 0x7f5e3400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:46950/0 (socket says 172.21.15.19:46950) 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e3ffff640 1 -- 172.21.15.19:0/4247983882 learned_addr learned my addr 172.21.15.19:0/4247983882 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a75b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.931+0000 7f5e44d52640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a76070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 -- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 msgr2=0x55dce3a75b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a75b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 -- 172.21.15.19:0/4247983882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 msgr2=0x55dce3a76070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a76070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.936 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 -- 172.21.15.19:0/4247983882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dce3a616c0 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e3ffff640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5e30000c00 tx=0x7f5e30004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfec2ac5742ad35b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e30049070 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5e30031950 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.932+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e3409a200 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f5e3005f050 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 10 client.17259 fetch_fsmap learned FSMap version 52 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55dce3aea100 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e3004a040 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 8418+0+0 (secure 0 0 0) 0x7f5e30003d00 con 0x55dce3a60a30 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 10 client.17259 fetch_fsmap finished waiting for FSMap version 52 2024-09-12T03:28:26.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 20 client.17259 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.933+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a6f680 0x55dce3aec4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.934+0000 7f5e48b8e740 4 client.17259 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.934+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5e3409a200 con 0x55dce3a6f680 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.934+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a6f680 0x55dce3aec4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:26.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.934+0000 7f5e3f7fe640 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a6f680 0x55dce3aec4c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f804b7ee1b7fc556 server_cookie=aa77a93d82342053 in_seq=0 out_seq=0 2024-09-12T03:28:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.934+0000 7f5e1f7fe640 10 client.17259 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e1f7fe640 1 -- 172.21.15.19:0/4247983882 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x7f5e3409a200 con 0x55dce3a6f680 2024-09-12T03:28:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e1f7fe640 10 client.17259 handle_command_reply: tid=0 2024-09-12T03:28:26.939 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 client.17259 shutdown 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 msgr2=0x55dce3a755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5e30000c00 tx=0x7f5e30004970 comp rx=0 tx=0).stop 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 shutdown_connections 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dce3a6f680 0x55dce3aec4c0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dce3a64dc0 0x55dce3a76070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dce3a644a0 0x55dce3a75b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 --2- 172.21.15.19:0/4247983882 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dce3a60a30 0x55dce3a755f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 >> 172.21.15.19:0/4247983882 conn(0x55dce3a6c880 msgr2=0x55dce3a6e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e1f7fe640 0 client.17259 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 shutdown_connections 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 1 -- 172.21.15.19:0/4247983882 wait complete. 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.939+0000 7f5e48b8e740 20 client.17259 trim_cache size 0 max 16384 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (262 inodes in the stack)", 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout: "f1c6db3f-f8a4-445f-b472-d756f2b12a3e": { 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:28:26.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "f1c6db3f-f8a4-445f-b472-d756f2b12a3e", 2024-09-12T03:28:26.942 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:28:26.942 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:26.942 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:26.942 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:26.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 msgr2=0x7f5e280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 0x7f5e280a5040 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009fd0 tx=0x7f5e38009500 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 msgr2=0x7f5e40103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e9f0 tx=0x7f5e3400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 shutdown_connections 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5e40102ef0 0x7f5e401041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5e280a2b80 0x7f5e280a5040 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5e40102570 0x7f5e40103c90 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 --2- 172.21.15.19:0/4230783109 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5e40100e20 0x7f5e40103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 >> 172.21.15.19:0/4230783109 conn(0x7f5e400fc7d0 msgr2=0x7f5e4006a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 shutdown_connections 2024-09-12T03:28:26.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:26.940+0000 7f5e467b5640 1 -- 172.21.15.19:0/4230783109 wait complete. 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:27.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: pgmap v619: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 220 op/s 2024-09-12T03:28:27.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:26 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:27.768 INFO:tasks.cephfs.filesystem:scrub status for tag:f1c6db3f-f8a4-445f-b472-d756f2b12a3e - {'path': '/', 'tag': 'f1c6db3f-f8a4-445f-b472-d756f2b12a3e', 'options': 'recursive,force'} 2024-09-12T03:28:27.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:27 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:27.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:27 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:27.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:27 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:27.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:27 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:27 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:27 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:28.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:28 smithi121 ceph-mon[34963]: pgmap v620: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:28.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:28 smithi019 ceph-mon[30953]: pgmap v620: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:28 smithi135 ceph-mon[35474]: pgmap v620: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcbdffb640 1 -- 172.21.15.19:0/1695492021 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7efcc409bb20 con 0x55caca144900 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcbdffb640 10 client.26566 handle_command_reply: tid=0 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 client.26566 shutdown 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 msgr2=0x55caca0d2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 0x55caca0d2200 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7efcc40058c0 tx=0x7efcc409ae70 comp rx=0 tx=0).stop 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 shutdown_connections 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55caca0c9940 0x55caca0cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55caca0bf080 0x55caca0d2200 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.560+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55caca144900 0x55caca146d20 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcde965740 1 --2- 172.21.15.19:0/1695492021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55caca0be760 0x55caca0d1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcbdffb640 0 client.26566 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 >> 172.21.15.19:0/1695492021 conn(0x55caca0bacf0 msgr2=0x55caca0b9650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 shutdown_connections 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcde965740 1 -- 172.21.15.19:0/1695492021 wait complete. 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.561+0000 7efcde965740 20 client.26566 trim_cache size 0 max 16384 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 52, 2024-09-12T03:28:29.566 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 34, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "f966fbf7": { 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 34, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 139.4, 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:28:29.567 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:29.568 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 msgr2=0x7efcc00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 0x7efcc00a5040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x55caca0bbb50 tx=0x7efcd0005f90 comp rx=0 tx=0).stop 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 msgr2=0x7efcd4100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d740 tx=0x7efcc400dc00 comp rx=0 tx=0).stop 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 shutdown_connections 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efcd410c150 0x7efcd4101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efcc00a2b80 0x7efcc00a5040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efcd4103640 0x7efcd4100ec0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 --2- 172.21.15.19:0/2840950693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efcd4102d20 0x7efcd4100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 >> 172.21.15.19:0/2840950693 conn(0x7efcd40fc780 msgr2=0x7efcd40fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 shutdown_connections 2024-09-12T03:28:29.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:29.562+0000 7efcdc58c640 1 -- 172.21.15.19:0/2840950693 wait complete. 2024-09-12T03:28:29.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:29 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:29.906 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:29 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:29 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.359 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f966fbf7', quiesce took 13.8 seconds. Will release after: 5.599999999999998 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: pgmap v621: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.901 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:28:30.902 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: pgmap v621: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:28:30.916 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: pgmap v621: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:28:31.046 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:31.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:31 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:31 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:32.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:31 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:32.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:32 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:32 smithi019 ceph-mon[30953]: pgmap v622: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 869 KiB/s wr, 235 op/s 2024-09-12T03:28:32.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:32 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:32 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:32 smithi135 ceph-mon[35474]: pgmap v622: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 869 KiB/s wr, 235 op/s 2024-09-12T03:28:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:32 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:32 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:32 smithi121 ceph-mon[34963]: pgmap v622: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 914 KiB/s rd, 869 KiB/s wr, 235 op/s 2024-09-12T03:28:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:32 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:33.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:33 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:34.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:33 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:34.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:33 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:28:34.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:34 smithi019 ceph-mon[30953]: pgmap v623: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 558 KiB/s wr, 148 op/s 2024-09-12T03:28:35.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:34 smithi135 ceph-mon[35474]: pgmap v623: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 558 KiB/s wr, 148 op/s 2024-09-12T03:28:35.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:34 smithi121 ceph-mon[34963]: pgmap v623: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 558 KiB/s wr, 148 op/s 2024-09-12T03:28:35.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:28:36.235 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:36.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:36 smithi019 ceph-mon[30953]: pgmap v624: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 559 KiB/s wr, 122 op/s 2024-09-12T03:28:37.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:36 smithi135 ceph-mon[35474]: pgmap v624: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 559 KiB/s wr, 122 op/s 2024-09-12T03:28:37.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:36 smithi121 ceph-mon[34963]: pgmap v624: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 638 KiB/s rd, 559 KiB/s wr, 122 op/s 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 -- 172.21.15.19:0/3003733386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102570 msgr2=0x7f9bc41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 --2- 172.21.15.19:0/3003733386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102570 0x7f9bc41029b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009900 tx=0x7f9bac02a480 comp rx=0 tx=0).stop 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 -- 172.21.15.19:0/3003733386 shutdown_connections 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 --2- 172.21.15.19:0/3003733386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102ef0 0x7f9bc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 --2- 172.21.15.19:0/3003733386 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102570 0x7f9bc41029b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 --2- 172.21.15.19:0/3003733386 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 -- 172.21.15.19:0/3003733386 >> 172.21.15.19:0/3003733386 conn(0x7f9bc40fc7d0 msgr2=0x7f9bc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 -- 172.21.15.19:0/3003733386 shutdown_connections 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.713+0000 7f9bca141640 1 -- 172.21.15.19:0/3003733386 wait complete. 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 Processor -- start 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 -- start start 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc41984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102570 0x7f9bc4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc41120a0 con 0x7f9bc4102ef0 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4111f20 con 0x7f9bc4102570 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bca141640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4112220 con 0x7f9bc4100e20 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bc37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc41984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.607 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bc2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102570 0x7f9bc4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.608 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40172/0 (socket says 172.21.15.19:40172) 2024-09-12T03:28:37.608 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:28:37.608 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":52,"btime":"2024-09-12T03:28:25:626044+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:28:24.885425+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.714+0000 7f9bc3fff640 1 -- 172.21.15.19:0/3446388210 learned_addr learned my addr 172.21.15.19:0/3446388210 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 -- 172.21.15.19:0/3446388210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 msgr2=0x7f9bc41984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc41984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 -- 172.21.15.19:0/3446388210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102570 msgr2=0x7f9bc4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102570 0x7f9bc4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 -- 172.21.15.19:0/3446388210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc419d4a0 con 0x7f9bc4102ef0 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc3fff640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00c970 tx=0x7f9bbc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc007bb0 con 0x7f9bc4102ef0 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc419d790 con 0x7f9bc4102ef0 2024-09-12T03:28:37.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bbc007d50 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bbc00f450 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.715+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc41a5070 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.716+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc4101200 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.717+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9bbc004f00 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.718+0000 7f9bc0ff9640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 0x7f9bb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.718+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9bbc0cef90 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.718+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 0x7f9bb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.719+0000 7f9bc37fe640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 0x7f9bb00a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005fd0 tx=0x7f9bb4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.721+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9bbc014030 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.936+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc4109090 con 0x7f9bc4102ef0 2024-09-12T03:28:37.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.937+0000 7f9bc0ff9640 1 -- 172.21.15.19:0/3446388210 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+11997 (secure 0 0 0) 0x7f9bbc09b950 con 0x7f9bc4102ef0 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 52 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 msgr2=0x7f9bb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 0x7f9bb00a5040 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005fd0 tx=0x7f9bb4009500 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 msgr2=0x7f9bc419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc00c970 tx=0x7f9bbc00ce30 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 shutdown_connections 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9bb00a2b80 0x7f9bb00a5040 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9bc4102ef0 0x7f9bc419cdc0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9bc4102570 0x7f9bc4198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 --2- 172.21.15.19:0/3446388210 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9bc4100e20 0x7f9bc41984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 >> 172.21.15.19:0/3446388210 conn(0x7f9bc40fc7d0 msgr2=0x7f9bc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:37.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 shutdown_connections 2024-09-12T03:28:37.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:36.939+0000 7f9bca141640 1 -- 172.21.15.19:0/3446388210 wait complete. 2024-09-12T03:28:37.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:37 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3446388210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:37 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3446388210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:37 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3446388210' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:28:38.658 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id f966fbf7 --release --await' 2024-09-12T03:28:38.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id f966fbf7 --release --await 2024-09-12T03:28:38.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:38 smithi019 ceph-mon[30953]: pgmap v625: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 KiB/s wr, 84 op/s 2024-09-12T03:28:38.921 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:39.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:38 smithi135 ceph-mon[35474]: pgmap v625: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 KiB/s wr, 84 op/s 2024-09-12T03:28:39.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:38 smithi121 ceph-mon[34963]: pgmap v625: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 KiB/s wr, 84 op/s 2024-09-12T03:28:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 -- 172.21.15.19:0/3219716331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 msgr2=0x7f2c40106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 --2- 172.21.15.19:0/3219716331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c40106840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f2c34009930 tx=0x7f2c3402a420 comp rx=0 tx=0).stop 2024-09-12T03:28:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 -- 172.21.15.19:0/3219716331 shutdown_connections 2024-09-12T03:28:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 --2- 172.21.15.19:0/3219716331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 0x7f2c4010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 --2- 172.21.15.19:0/3219716331 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c40106840 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 --2- 172.21.15.19:0/3219716331 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 0x7f2c401006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 -- 172.21.15.19:0/3219716331 >> 172.21.15.19:0/3219716331 conn(0x7f2c40077210 msgr2=0x7f2c400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 -- 172.21.15.19:0/3219716331 shutdown_connections 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.600+0000 7f2c45e72640 1 -- 172.21.15.19:0/3219716331 wait complete. 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.601+0000 7f2c45e72640 1 Processor -- start 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.601+0000 7f2c45e72640 1 -- start start 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 0x7f2c401984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 0x7f2c4019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40111f60 con 0x7f2c40100ca0 2024-09-12T03:28:39.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c40111de0 con 0x7f2c40106f50 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c45e72640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401120e0 con 0x7f2c401002f0 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40184/0 (socket says 172.21.15.19:40184) 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 0x7f2c4019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 -- 172.21.15.19:0/179080256 learned_addr learned my addr 172.21.15.19:0/179080256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 0x7f2c401984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 -- 172.21.15.19:0/179080256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 msgr2=0x7f2c401984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 0x7f2c401984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 -- 172.21.15.19:0/179080256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 msgr2=0x7f2c4019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 0x7f2c4019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.602+0000 7f2c3effd640 1 -- 172.21.15.19:0/179080256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c4019d450 con 0x7f2c40100ca0 2024-09-12T03:28:39.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c3effd640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005940 tx=0x7f2c340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c3cff9640 1 -- 172.21.15.19:0/179080256 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3403f030 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c4019d6e0 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c3cff9640 1 -- 172.21.15.19:0/179080256 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c34003ce0 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c3cff9640 1 -- 172.21.15.19:0/179080256 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34035070 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.603+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c401a5020 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.605+0000 7f2c3cff9640 1 -- 172.21.15.19:0/179080256 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c34042030 con 0x7f2c40100ca0 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.605+0000 7f2c4824b740 1 Processor -- start 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 -- start start 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cd7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732ce1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560732c3c620 con 0x560732ce2240 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560732be3e50 con 0x560732cd7980 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c4824b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560732c385e0 con 0x560732cd7060 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40188/0 (socket says 172.21.15.19:40188) 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 -- 172.21.15.19:0/3651894082 learned_addr learned my addr 172.21.15.19:0/3651894082 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3cff9640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 0x7f2c280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732ce1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cd7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 0x7f2c280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 -- 172.21.15.19:0/3651894082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 msgr2=0x560732cd7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cd7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 -- 172.21.15.19:0/3651894082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 msgr2=0x560732ce1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732ce1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3cff9640 1 -- 172.21.15.19:0/179080256 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2c340ef500 con 0x7f2c40100ca0 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.606+0000 7f2c3effd640 1 -- 172.21.15.19:0/3651894082 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560732bd4d40 con 0x560732ce2240 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c3effd640 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2c3408a690 tx=0x7f2c3403b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9f0d7fffe62ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 0x7f2c280a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009fd0 tx=0x7f2c38009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c25ffb640 1 -- 172.21.15.19:0/3651894082 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c34049020 con 0x560732ce2240 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c25ffb640 1 -- 172.21.15.19:0/3651894082 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c340bbdc0 con 0x560732ce2240 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c25ffb640 1 -- 172.21.15.19:0/3651894082 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3403b460 con 0x560732ce2240 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c4824b740 1 -- 172.21.15.19:0/3651894082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 msgr2=0x560732ce4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.607+0000 7f2c4824b740 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2c3408a690 tx=0x7f2c3403b720 comp rx=0 tx=0).stop 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- 172.21.15.19:0/3651894082 shutdown_connections 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732ce4630 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732ce1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- 172.21.15.19:0/3651894082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cd7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- 172.21.15.19:0/3651894082 >> 172.21.15.19:0/3651894082 conn(0x560732cd35f0 msgr2=0x560732cd39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- 172.21.15.19:0/3651894082 shutdown_connections 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- 172.21.15.19:0/3651894082 wait complete. 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- start start 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 Processor -- start 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cdf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732cdf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x560732ce4bd0 con 0x560732ce2240 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x560732bc4f10 con 0x560732cd7980 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x560732ce4d50 con 0x560732cd7060 2024-09-12T03:28:39.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.608+0000 7f2c4824b740 10 client.0 _ll_register_callbacks cb 0x560732be8db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40204/0 (socket says 172.21.15.19:40204) 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 -- 172.21.15.19:0/2973150803 learned_addr learned my addr 172.21.15.19:0/2973150803 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cdf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732cdf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 -- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 msgr2=0x560732cdf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cdf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 -- 172.21.15.19:0/2973150803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 msgr2=0x560732cdf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732cdf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 -- 172.21.15.19:0/2973150803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073294b580 con 0x560732ce2240 2024-09-12T03:28:39.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732cdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800abc0 tx=0x7f2c38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2057dcc251eaa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.609+0000 7f2c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3801edc0 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2c38004b20 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c340bbdc0 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2c38004d20 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 10 client.17277 fetch_fsmap learned FSMap version 52 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560732a15960 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c340bbdc0 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 8418+0+0 (secure 0 0 0) 0x7f2c3800f040 con 0x560732ce2240 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 10 client.17277 fetch_fsmap finished waiting for FSMap version 52 2024-09-12T03:28:39.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 10 client.17277 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 20 client.17277 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732cf1b50 0x560732cf3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 4 client.17277 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.610+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560732c49b90 con 0x560732cf1b50 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.611+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732cf1b50 0x560732cf3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.611+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732cf1b50 0x560732cf3fd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16bae7c3b6a0a3a0 server_cookie=23bda1015e34b53f in_seq=0 out_seq=0 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.611+0000 7f2c1effd640 10 client.17277 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c1effd640 1 -- 172.21.15.19:0/2973150803 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560732c49b90 con 0x560732cf1b50 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c1effd640 10 client.17277 handle_command_reply: tid=0 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 client.17277 shutdown 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 msgr2=0x560732cdfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800abc0 tx=0x7f2c38004a40 comp rx=0 tx=0).stop 2024-09-12T03:28:39.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 shutdown_connections 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cdfb70 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732cf1b50 0x560732cf3fd0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732cdf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c1effd640 0 client.17277 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2973150803 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7060 0x560732cdf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 >> 172.21.15.19:0/2973150803 conn(0x560732cd35f0 msgr2=0x560732cd9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 shutdown_connections 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.612+0000 7f2c4824b740 1 -- 172.21.15.19:0/2973150803 wait complete. 2024-09-12T03:28:39.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.613+0000 7f2c4824b740 20 client.17277 trim_cache size 0 max 16384 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 Processor -- start 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 -- start start 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7060 0x560732cd2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7980 0x560732cf98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401ad600 con 0x560732ce2240 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401af380 con 0x560732cd7060 2024-09-12T03:28:39.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c4824b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401adcd0 con 0x560732cd7980 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c3ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40212/0 (socket says 172.21.15.19:40212) 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c3ffff640 1 -- 172.21.15.19:0/265382203 learned_addr learned my addr 172.21.15.19:0/265382203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.630+0000 7f2c3effd640 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7980 0x560732cf98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7060 0x560732cd2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 -- 172.21.15.19:0/265382203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7980 msgr2=0x560732cf98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7980 0x560732cf98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 -- 172.21.15.19:0/265382203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7060 msgr2=0x560732cd2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7060 0x560732cd2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 -- 172.21.15.19:0/265382203 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560732cd1bb0 con 0x560732ce2240 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800a750 tx=0x7f2c38009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b959427a40d4b239 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c257fa640 1 -- 172.21.15.19:0/265382203 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38004ab0 con 0x560732ce2240 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c257fa640 1 -- 172.21.15.19:0/265382203 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38037070 con 0x560732ce2240 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.631+0000 7f2c257fa640 1 -- 172.21.15.19:0/265382203 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c38028400 con 0x560732ce2240 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- 172.21.15.19:0/265382203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 msgr2=0x560732cf9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2c3800a750 tx=0x7f2c38009c40 comp rx=0 tx=0).stop 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- 172.21.15.19:0/265382203 shutdown_connections 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732ce2240 0x560732cf9de0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732cd7980 0x560732cf98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 --2- 172.21.15.19:0/265382203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7060 0x560732cd2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- 172.21.15.19:0/265382203 >> 172.21.15.19:0/265382203 conn(0x560732cd35f0 msgr2=0x560732cd1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- 172.21.15.19:0/265382203 shutdown_connections 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- 172.21.15.19:0/265382203 wait complete. 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.632+0000 7f2c4824b740 1 -- start start 2024-09-12T03:28:39.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 Processor -- start 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732d56840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732ce2240 0x560732d56d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401aebf0 con 0x560732cd7060 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401ae460 con 0x560732cd7980 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2c401ab5b0 con 0x560732ce2240 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c4824b740 10 client.0 _ll_register_callbacks cb 0x7f2c3800a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40218/0 (socket says 172.21.15.19:40218) 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 -- 172.21.15.19:0/2677992411 learned_addr learned my addr 172.21.15.19:0/2677992411 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732d56840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.647 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3ffff640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732ce2240 0x560732d56d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 -- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732ce2240 msgr2=0x560732d56d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732ce2240 0x560732d56d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 -- 172.21.15.19:0/2677992411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 msgr2=0x560732d56840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.633+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732d56840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c3f7fe640 1 -- 172.21.15.19:0/2677992411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560732d572c0 con 0x560732cd7060 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732d56840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c3f7fe640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000aba0 tx=0x7f2c30010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156dfedd7b375c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c1effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c3002b060 con 0x560732cd7060 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2c3001dae0 con 0x560732cd7060 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c38037070 con 0x560732cd7060 2024-09-12T03:28:39.648 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f2c3001dce0 con 0x560732cd7060 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c4824b740 10 client.17283 fetch_fsmap learned FSMap version 52 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.634+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560732d61a40 con 0x560732cd7060 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c38037070 con 0x560732cd7060 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 52) ==== 8418+0+0 (secure 0 0 0) 0x7f2c30026520 con 0x560732cd7060 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 10 client.17283 fetch_fsmap finished waiting for FSMap version 52 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 10 client.17283 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 20 client.17283 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732d621e0 0x560732d64600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 4 client.17283 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f966fbf7", "release": true, "await": true}] 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f966fbf7", "release": true, "await": true}) -- 0x560732c440a0 con 0x560732d621e0 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732d621e0 0x560732d64600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c3effd640 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732d621e0 0x560732d64600 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ece31073ab11a102 server_cookie=675a54d01984a662 in_seq=0 out_seq=0 2024-09-12T03:28:39.649 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:39.635+0000 7f2c1effd640 10 client.17283 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:40.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.755+0000 7f2c1effd640 1 -- 172.21.15.19:0/2677992411 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560732c440a0 con 0x560732d621e0 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.755+0000 7f2c1effd640 10 client.17283 handle_command_reply: tid=0 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.755+0000 7f2c4824b740 1 client.17283 shutdown 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 msgr2=0x560732d57e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000aba0 tx=0x7f2c30010ad0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 shutdown_connections 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x560732ce2240 0x560732d56d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x560732d621e0 0x560732d64600 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c1effd640 0 client.17283 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x560732cd7980 0x560732d56840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 --2- 172.21.15.19:0/2677992411 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x560732cd7060 0x560732d57e30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 >> 172.21.15.19:0/2677992411 conn(0x560732cd35f0 msgr2=0x560732cdf8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 shutdown_connections 2024-09-12T03:28:40.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 1 -- 172.21.15.19:0/2677992411 wait complete. 2024-09-12T03:28:40.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.756+0000 7f2c4824b740 20 client.17283 trim_cache size 0 max 16384 2024-09-12T03:28:40.759 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 52, 2024-09-12T03:28:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:28:40.759 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 36, 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "f966fbf7": { 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 36, 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:40.760 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 139.4, 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:28:40.761 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:28:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:28:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:28:40.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:28:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:28:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:40.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:40.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:28:40.764 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 msgr2=0x7f2c280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 0x7f2c280a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009fd0 tx=0x7f2c38009600 comp rx=0 tx=0).stop 2024-09-12T03:28:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 msgr2=0x7f2c401989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f2c34005940 tx=0x7f2c340042f0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 shutdown_connections 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c40106f50 0x7f2c4019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2c280a2b80 0x7f2c280a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2c40100ca0 0x7f2c401989e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 --2- 172.21.15.19:0/179080256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2c401002f0 0x7f2c401984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 >> 172.21.15.19:0/179080256 conn(0x7f2c40077210 msgr2=0x7f2c401051f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:40.766 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 shutdown_connections 2024-09-12T03:28:40.767 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:40.757+0000 7f2c45e72640 1 -- 172.21.15.19:0/179080256 wait complete. 2024-09-12T03:28:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:40 smithi135 ceph-mon[35474]: pgmap v626: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 20 KiB/s wr, 60 op/s 2024-09-12T03:28:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:41.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:40 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:28:41.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:40 smithi121 ceph-mon[34963]: pgmap v626: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 20 KiB/s wr, 60 op/s 2024-09-12T03:28:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:40 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:28:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:40 smithi019 ceph-mon[30953]: pgmap v626: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 20 KiB/s wr, 60 op/s 2024-09-12T03:28:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:28:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:28:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:28:41.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:40 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:28:41.675 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f966fbf7', total seconds elapsed: 25.1 2024-09-12T03:28:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:42 smithi135 ceph-mon[35474]: pgmap v627: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 55 op/s 2024-09-12T03:28:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:42 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:43.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:42 smithi121 ceph-mon[34963]: pgmap v627: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 55 op/s 2024-09-12T03:28:43.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:42 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:42 smithi019 ceph-mon[30953]: pgmap v627: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 KiB/s wr, 55 op/s 2024-09-12T03:28:43.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:42 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: pgmap v628: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 853 B/s wr, 45 op/s 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:28:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:44 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: pgmap v628: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 853 B/s wr, 45 op/s 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:28:45.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:44 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:28:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: pgmap v628: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 853 B/s wr, 45 op/s 2024-09-12T03:28:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:28:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:28:45.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:28:45.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:28:45.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:44 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:28:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:46 smithi135 ceph-mon[35474]: pgmap v629: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 484 KiB/s wr, 94 op/s 2024-09-12T03:28:47.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:46 smithi121 ceph-mon[34963]: pgmap v629: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 484 KiB/s wr, 94 op/s 2024-09-12T03:28:47.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:46 smithi019 ceph-mon[30953]: pgmap v629: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 484 KiB/s wr, 94 op/s 2024-09-12T03:28:49.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:48 smithi135 ceph-mon[35474]: pgmap v630: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 617 KiB/s wr, 173 op/s 2024-09-12T03:28:49.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:48 smithi121 ceph-mon[34963]: pgmap v630: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 617 KiB/s wr, 173 op/s 2024-09-12T03:28:49.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:48 smithi019 ceph-mon[30953]: pgmap v630: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 617 KiB/s wr, 173 op/s 2024-09-12T03:28:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:50 smithi135 ceph-mon[35474]: pgmap v631: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 617 KiB/s wr, 186 op/s 2024-09-12T03:28:51.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:50 smithi121 ceph-mon[34963]: pgmap v631: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 617 KiB/s wr, 186 op/s 2024-09-12T03:28:51.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:50 smithi019 ceph-mon[30953]: pgmap v631: 97 pgs: 97 active+clean; 2.3 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 504 KiB/s rd, 617 KiB/s wr, 186 op/s 2024-09-12T03:28:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:52 smithi135 ceph-mon[35474]: pgmap v632: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 780 KiB/s wr, 276 op/s 2024-09-12T03:28:53.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:52 smithi121 ceph-mon[34963]: pgmap v632: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 780 KiB/s wr, 276 op/s 2024-09-12T03:28:53.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:52 smithi019 ceph-mon[30953]: pgmap v632: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 779 KiB/s rd, 780 KiB/s wr, 276 op/s 2024-09-12T03:28:55.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:54 smithi135 ceph-mon[35474]: pgmap v633: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 320 op/s 2024-09-12T03:28:55.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:54 smithi121 ceph-mon[34963]: pgmap v633: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 320 op/s 2024-09-12T03:28:55.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:54 smithi019 ceph-mon[30953]: pgmap v633: 97 pgs: 97 active+clean; 2.3 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 320 op/s 2024-09-12T03:28:57.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:56 smithi135 ceph-mon[35474]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 400 op/s 2024-09-12T03:28:57.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:56 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:57.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:56 smithi121 ceph-mon[34963]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 400 op/s 2024-09-12T03:28:57.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:56 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:57.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:56 smithi019 ceph-mon[30953]: pgmap v634: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 400 op/s 2024-09-12T03:28:57.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:56 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:57.769 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:28:58.044 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:28:58.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:57 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:58.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:57 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:58.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:57 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:28:58.795 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:28:58 smithi019 ceph-mon[30953]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 451 op/s 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 -- 172.21.15.19:0/3349252226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4102ef0 msgr2=0x7ff0a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 --2- 172.21.15.19:0/3349252226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4102ef0 0x7ff0a410da40 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7ff09c009ca0 tx=0x7ff09c02a950 comp rx=0 tx=0).stop 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 -- 172.21.15.19:0/3349252226 shutdown_connections 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 --2- 172.21.15.19:0/3349252226 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4102ef0 0x7ff0a410da40 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 --2- 172.21.15.19:0/3349252226 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 0x7ff0a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 --2- 172.21.15.19:0/3349252226 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4100e20 0x7ff0a4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 -- 172.21.15.19:0/3349252226 >> 172.21.15.19:0/3349252226 conn(0x7ff0a40fc7d0 msgr2=0x7ff0a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 -- 172.21.15.19:0/3349252226 shutdown_connections 2024-09-12T03:28:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.910+0000 7ff0aac88640 1 -- 172.21.15.19:0/3349252226 wait complete. 2024-09-12T03:28:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 Processor -- start 2024-09-12T03:28:58.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 -- start start 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 0x7ff0a40fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4102ef0 0x7ff0a40fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4110f80 con 0x7ff0a4100e20 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4110e00 con 0x7ff0a4102ef0 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0aac88640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a4111100 con 0x7ff0a4102570 2024-09-12T03:28:58.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58106/0 (socket says 172.21.15.19:58106) 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0a8a24640 1 -- 172.21.15.19:0/535747975 learned_addr learned my addr 172.21.15.19:0/535747975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff09bfff640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 0x7ff0a40fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.911+0000 7ff0a9225640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4102ef0 0x7ff0a40fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 -- 172.21.15.19:0/535747975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 msgr2=0x7ff0a40fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 0x7ff0a40fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 -- 172.21.15.19:0/535747975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4102ef0 msgr2=0x7ff0a40fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4102ef0 0x7ff0a40fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 -- 172.21.15.19:0/535747975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0a41a69d0 con 0x7ff0a4100e20 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000e9f0 tx=0x7ff0a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff099ffb640 1 -- 172.21.15.19:0/535747975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a000cd80 con 0x7ff0a4100e20 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0a41a6cc0 con 0x7ff0a4100e20 2024-09-12T03:28:58.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff099ffb640 1 -- 172.21.15.19:0/535747975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a0010910 con 0x7ff0a4100e20 2024-09-12T03:28:58.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff099ffb640 1 -- 172.21.15.19:0/535747975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00188f0 con 0x7ff0a4100e20 2024-09-12T03:28:58.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.912+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0a41a71f0 con 0x7ff0a4100e20 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff099ffb640 1 -- 172.21.15.19:0/535747975 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0a0010ac0 con 0x7ff0a4100e20 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 Processor -- start 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 -- start start 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb895c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb895ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fbb88b72e0 con 0x55fbb8951dc0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.915+0000 7ff0ad061740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fbb885ece0 con 0x55fbb895c680 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0ad061740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fbb88b32a0 con 0x55fbb89514a0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb895c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:33296/0 (socket says 172.21.15.19:33296) 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 -- 172.21.15.19:0/1919861093 learned_addr learned my addr 172.21.15.19:0/1919861093 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff099ffb640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 0x7ff08c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff09bfff640 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb895ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 0x7ff08c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff099ffb640 1 -- 172.21.15.19:0/535747975 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff0a00cf3f0 con 0x7ff0a4100e20 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 -- 172.21.15.19:0/1919861093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 msgr2=0x55fbb895ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb895ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 -- 172.21.15.19:0/1919861093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 msgr2=0x55fbb895c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb895c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.916+0000 7ff0a9225640 1 -- 172.21.15.19:0/1919861093 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb8843440 con 0x55fbb89514a0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.917+0000 7ff0a9225640 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fa0 tx=0x7ff09c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9359da2d92a0bc57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.917+0000 7ff0a9225640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 0x7ff08c0a50c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff09c047000 tx=0x7ff09c049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.917+0000 7ff08affd640 1 -- 172.21.15.19:0/1919861093 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c04e070 con 0x55fbb89514a0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.917+0000 7ff08affd640 1 -- 172.21.15.19:0/1919861093 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c02f620 con 0x55fbb89514a0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.917+0000 7ff08affd640 1 -- 172.21.15.19:0/1919861093 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c034040 con 0x55fbb89514a0 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- 172.21.15.19:0/1919861093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 msgr2=0x55fbb8951880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff09c005fa0 tx=0x7ff09c009a20 comp rx=0 tx=0).stop 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- 172.21.15.19:0/1919861093 shutdown_connections 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb895ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb895c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 --2- 172.21.15.19:0/1919861093 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb8951880 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- 172.21.15.19:0/1919861093 >> 172.21.15.19:0/1919861093 conn(0x55fbb894da30 msgr2=0x55fbb894de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- 172.21.15.19:0/1919861093 shutdown_connections 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- 172.21.15.19:0/1919861093 wait complete. 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 -- start start 2024-09-12T03:28:58.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.918+0000 7ff0ad061740 1 Processor -- start 2024-09-12T03:28:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb8959240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb8959780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fbb895f2a0 con 0x55fbb8951dc0 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fbb8872050 con 0x55fbb895c680 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fbb895f420 con 0x55fbb89514a0 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0ad061740 10 client.0 _ll_register_callbacks cb 0x55fbb88b2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb8959780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:33304/0 (socket says 172.21.15.19:33304) 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a8a24640 1 -- 172.21.15.19:0/932684595 learned_addr learned my addr 172.21.15.19:0/932684595 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 -- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 msgr2=0x55fbb895bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 -- 172.21.15.19:0/932684595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 msgr2=0x55fbb8959240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff09bfff640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb8959240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb8959240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a9225640 1 -- 172.21.15.19:0/932684595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb86283c0 con 0x55fbb895c680 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.919+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0a9225640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb8959780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006010 tx=0x7ff09c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb69bc0ec7ced6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff09bfff640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb8959240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c055020 con 0x55fbb895c680 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff09c049680 con 0x55fbb895c680 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff09c02f620 con 0x55fbb895c680 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff09c04e070 con 0x55fbb895c680 2024-09-12T03:28:58.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 10 client.26687 fetch_fsmap learned FSMap version 55 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55fbb895f2a0 con 0x55fbb895c680 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff09c02f620 con 0x55fbb895c680 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 55) ==== 8430+0+0 (secure 0 0 0) 0x7ff09c0394e0 con 0x55fbb895c680 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 10 client.26687 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 10 client.26687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 20 client.26687 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb896bdd0 0x55fbb896e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 4 client.26687 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.920+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbb88c4850 con 0x55fbb896bdd0 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.921+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb896bdd0 0x55fbb896e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.921+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb896bdd0 0x55fbb896e1c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f2045e4631a3e5 server_cookie=ad20e2bd5f8b420e in_seq=0 out_seq=0 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.921+0000 7ff083fff640 10 client.26687 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff083fff640 1 -- 172.21.15.19:0/932684595 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fbb88c4850 con 0x55fbb896bdd0 2024-09-12T03:28:58.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff083fff640 10 client.26687 handle_command_reply: tid=0 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 client.26687 shutdown 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 msgr2=0x55fbb8959780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb8959780 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff09c006010 tx=0x7ff09c0047b0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 shutdown_connections 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb896bdd0 0x55fbb896e1c0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb895c680 0x55fbb8959780 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb8951dc0 0x55fbb8959240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff083fff640 0 client.26687 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 --2- 172.21.15.19:0/932684595 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb89514a0 0x55fbb895bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 >> 172.21.15.19:0/932684595 conn(0x55fbb894da30 msgr2=0x55fbb894cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 shutdown_connections 2024-09-12T03:28:58.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.922+0000 7ff0ad061740 1 -- 172.21.15.19:0/932684595 wait complete. 2024-09-12T03:28:58.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.923+0000 7ff0ad061740 20 client.26687 trim_cache size 0 max 16384 2024-09-12T03:28:58.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 Processor -- start 2024-09-12T03:28:58.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 -- start start 2024-09-12T03:28:58.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb89739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8973ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41af860 con 0x55fbb894da30 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b15e0 con 0x55fbb89514a0 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0ad061740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41aff30 con 0x55fbb8951dc0 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58134/0 (socket says 172.21.15.19:58134) 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.940+0000 7ff0a8a24640 1 -- 172.21.15.19:0/811076965 learned_addr learned my addr 172.21.15.19:0/811076965 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a9225640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8973ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff09bfff640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb89739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 -- 172.21.15.19:0/811076965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 msgr2=0x55fbb8973ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8973ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 -- 172.21.15.19:0/811076965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 msgr2=0x55fbb89739a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb89739a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 -- 172.21.15.19:0/811076965 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbb895f0a0 con 0x55fbb894da30 2024-09-12T03:28:58.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff09bfff640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb89739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:28:58.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.941+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00985e0 tx=0x7ff0a00ccdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838620b82785efa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.942+0000 7ff08a7fc640 1 -- 172.21.15.19:0/811076965 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a00a0010 con 0x55fbb894da30 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.942+0000 7ff08a7fc640 1 -- 172.21.15.19:0/811076965 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a00b1020 con 0x55fbb894da30 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.942+0000 7ff08a7fc640 1 -- 172.21.15.19:0/811076965 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a009b310 con 0x55fbb894da30 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- 172.21.15.19:0/811076965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 msgr2=0x55fbb895b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7ff0a00985e0 tx=0x7ff0a00ccdd0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- 172.21.15.19:0/811076965 shutdown_connections 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8973ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb89739a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 --2- 172.21.15.19:0/811076965 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb895b1b0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- 172.21.15.19:0/811076965 >> 172.21.15.19:0/811076965 conn(0x55fbb89598e0 msgr2=0x55fbb895aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- 172.21.15.19:0/811076965 shutdown_connections 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- 172.21.15.19:0/811076965 wait complete. 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 -- start start 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.943+0000 7ff0ad061740 1 Processor -- start 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb8961fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8962500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b0e50 con 0x55fbb894da30 2024-09-12T03:28:58.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41b06c0 con 0x55fbb89514a0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff0a41ad7a0 con 0x55fbb8951dc0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0ad061740 10 client.0 _ll_register_callbacks cb 0x7ff09c04c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a8a24640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58138/0 (socket says 172.21.15.19:58138) 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a8a24640 1 -- 172.21.15.19:0/942593784 learned_addr learned my addr 172.21.15.19:0/942593784 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a9225640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8962500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb8961fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 -- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 msgr2=0x55fbb8962500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8962500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 -- 172.21.15.19:0/942593784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 msgr2=0x55fbb8964c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff09bfff640 1 -- 172.21.15.19:0/942593784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbb8962ba0 con 0x55fbb89514a0 2024-09-12T03:28:58.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.944+0000 7ff0a9225640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8962500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff09bfff640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb8961fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55fbb895a880 tx=0x7ff094010b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fef4e1bd0250cf4a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094020020 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff09400f8f0 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0a00b1020 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff09400faf0 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 10 client.26693 fetch_fsmap learned FSMap version 55 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55fbb89d6860 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0a00b1020 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 55) ==== 8430+0+0 (secure 0 0 0) 0x7ff09401c420 con 0x55fbb89514a0 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 10 client.26693 fetch_fsmap finished waiting for FSMap version 55 2024-09-12T03:28:58.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 10 client.26693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 20 client.26693 populate_metadata read hostname 'smithi019' 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb895c680 0x55fbb89d9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 4 client.26693 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.945+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fbb88bed60 con 0x55fbb895c680 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb895c680 0x55fbb89d9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0a8a24640 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb895c680 0x55fbb89d9100 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bda9fbbecefbe91 server_cookie=c45dd0df4503d75f in_seq=0 out_seq=0 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff083fff640 10 client.26693 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff083fff640 1 -- 172.21.15.19:0/942593784 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fbb88bed60 con 0x55fbb895c680 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff083fff640 10 client.26693 handle_command_reply: tid=0 2024-09-12T03:28:58.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 client.26693 shutdown 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 msgr2=0x55fbb8961fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb8961fc0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55fbb895a880 tx=0x7ff094010b40 comp rx=0 tx=0).stop 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 shutdown_connections 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fbb8951dc0 0x55fbb8962500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55fbb895c680 0x55fbb89d9100 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fbb89514a0 0x55fbb8961fc0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff0ad061740 1 --2- 172.21.15.19:0/942593784 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fbb894da30 0x55fbb8964c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.946+0000 7ff083fff640 0 client.26693 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 >> 172.21.15.19:0/942593784 conn(0x55fbb89598e0 msgr2=0x55fbb895b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 shutdown_connections 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0ad061740 1 -- 172.21.15.19:0/942593784 wait complete. 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0ad061740 20 client.26693 trim_cache size 0 max 16384 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:28:58.952 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:28:58.953 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:28:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 msgr2=0x7ff08c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 0x7ff08c0a50c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff09c047000 tx=0x7ff09c049000 comp rx=0 tx=0).stop 2024-09-12T03:28:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 msgr2=0x7ff0a40fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:28:58.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff0a000e9f0 tx=0x7ff0a000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 shutdown_connections 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.947+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0a4102ef0 0x7ff0a40fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff0a4102570 0x7ff0a40fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff08c0a2c00 0x7ff08c0a50c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 --2- 172.21.15.19:0/535747975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff0a4100e20 0x7ff0a40fcc80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 >> 172.21.15.19:0/535747975 conn(0x7ff0a40fc7d0 msgr2=0x7ff0a406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 shutdown_connections 2024-09-12T03:28:58.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:28:58.948+0000 7ff0aac88640 1 -- 172.21.15.19:0/535747975 wait complete. 2024-09-12T03:28:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:28:58 smithi135 ceph-mon[35474]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 451 op/s 2024-09-12T03:28:59.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:28:58 smithi121 ceph-mon[34963]: pgmap v635: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 451 op/s 2024-09-12T03:28:59.773 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:28:59.773 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:00.096 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:00.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 -- 172.21.15.19:0/3542827745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988100b20 msgr2=0x7f9988100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 --2- 172.21.15.19:0/3542827745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988100b20 0x7f9988100f00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f9978009900 tx=0x7f997802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 -- 172.21.15.19:0/3542827745 shutdown_connections 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 --2- 172.21.15.19:0/3542827745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f998810e500 0x7f9988110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 --2- 172.21.15.19:0/3542827745 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9988101440 0x7f998810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 --2- 172.21.15.19:0/3542827745 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988100b20 0x7f9988100f00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.603+0000 7f998de73640 1 -- 172.21.15.19:0/3542827745 >> 172.21.15.19:0/3542827745 conn(0x7f99880fc7d0 msgr2=0x7f99880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.604+0000 7f998de73640 1 -- 172.21.15.19:0/3542827745 shutdown_connections 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.604+0000 7f998de73640 1 -- 172.21.15.19:0/3542827745 wait complete. 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.604+0000 7f998de73640 1 Processor -- start 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.604+0000 7f998de73640 1 -- start start 2024-09-12T03:29:00.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 0x7f99881a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f998810e500 0x7f998819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9988114260 con 0x7f9988101440 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f99881140e0 con 0x7f9988100b20 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f99881143e0 con 0x7f998810e500 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35520/0 (socket says 172.21.15.19:35520) 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 -- 172.21.15.19:0/3693256840 learned_addr learned my addr 172.21.15.19:0/3693256840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9987fff640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f998810e500 0x7f998819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f99877fe640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 0x7f99881a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 -- 172.21.15.19:0/3693256840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f998810e500 msgr2=0x7f998819a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f998810e500 0x7f998819a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 -- 172.21.15.19:0/3693256840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 msgr2=0x7f99881a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 0x7f99881a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 -- 172.21.15.19:0/3693256840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f998819ae80 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f99877fe640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 0x7f99881a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9986ffd640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9c0 tx=0x7f997c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c00cd80 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f997c0035f0 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f998819b170 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.605+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f997c010690 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.606+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881a74e0 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.607+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998806a850 con 0x7f9988101440 2024-09-12T03:29:00.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.608+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f997c0037a0 con 0x7f9988101440 2024-09-12T03:29:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.608+0000 7f9984ff9640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 0x7f99700a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.608+0000 7f99877fe640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 0x7f99700a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.608+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f997c0cf440 con 0x7f9988101440 2024-09-12T03:29:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.609+0000 7f99877fe640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 0x7f99700a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9978005ab0 tx=0x7f9978005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:00.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.613+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f997c09c3b0 con 0x7f9988101440 2024-09-12T03:29:00.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.845+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f998819bb90 con 0x7f9988101440 2024-09-12T03:29:00.845 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:00 smithi019 ceph-mon[30953]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-12T03:29:00.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.847+0000 7f9984ff9640 1 -- 172.21.15.19:0/3693256840 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+12003 (secure 0 0 0) 0x7f998819bb90 con 0x7f9988101440 2024-09-12T03:29:00.847 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:29:00.848 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":55,"btime":"2024-09-12T03:28:56:701525+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:28:55.755042+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:29:00.848 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 55 2024-09-12T03:29:00.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.850+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 msgr2=0x7f99700a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.850+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 0x7f99700a50c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9978005ab0 tx=0x7f9978005a20 comp rx=0 tx=0).stop 2024-09-12T03:29:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.850+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 msgr2=0x7f99881a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:00.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.850+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f997c00e9c0 tx=0x7f997c00ee80 comp rx=0 tx=0).stop 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 shutdown_connections 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f998810e500 0x7f998819a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f99700a2c00 0x7f99700a50c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9988101440 0x7f99881a0a30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 --2- 172.21.15.19:0/3693256840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9988100b20 0x7f99881a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 >> 172.21.15.19:0/3693256840 conn(0x7f99880fc7d0 msgr2=0x7f99880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.851+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 shutdown_connections 2024-09-12T03:29:00.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:00.852+0000 7f998de73640 1 -- 172.21.15.19:0/3693256840 wait complete. 2024-09-12T03:29:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:00 smithi135 ceph-mon[35474]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-12T03:29:01.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:00 smithi121 ceph-mon[34963]: pgmap v636: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 386 op/s 2024-09-12T03:29:01.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:29:01.836 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:01.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3693256840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3693256840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3693256840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 -- 172.21.15.19:0/1702749225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 msgr2=0x7f3388100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 --2- 172.21.15.19:0/1702749225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f3388100f00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3378009900 tx=0x7f337802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:02.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 -- 172.21.15.19:0/1702749225 shutdown_connections 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 --2- 172.21.15.19:0/1702749225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f3388110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 --2- 172.21.15.19:0/1702749225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 --2- 172.21.15.19:0/1702749225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f3388100f00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 -- 172.21.15.19:0/1702749225 >> 172.21.15.19:0/1702749225 conn(0x7f33880fc7d0 msgr2=0x7f33880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 -- 172.21.15.19:0/1702749225 shutdown_connections 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.361+0000 7f338d1d8640 1 -- 172.21.15.19:0/1702749225 wait complete. 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.362+0000 7f338d1d8640 1 Processor -- start 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.362+0000 7f338d1d8640 1 -- start start 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f338819e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3388114280 con 0x7f3388100b20 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3388114100 con 0x7f338810e500 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338d1d8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3388114400 con 0x7f3388101440 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35532/0 (socket says 172.21.15.19:35532) 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 -- 172.21.15.19:0/418745611 learned_addr learned my addr 172.21.15.19:0/418745611 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338759e640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f338819e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338659c640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 -- 172.21.15.19:0/418745611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 msgr2=0x7f338819ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338819ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 -- 172.21.15.19:0/418745611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 msgr2=0x7f338819e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f338819e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 -- 172.21.15.19:0/418745611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f338819e730 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338759e640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f338819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f338659c640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f3386d9d640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.363+0000 7f336ffff640 1 -- 172.21.15.19:0/418745611 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337803f030 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.364+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f338819e9c0 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.364+0000 7f336ffff640 1 -- 172.21.15.19:0/418745611 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3378004440 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.364+0000 7f336ffff640 1 -- 172.21.15.19:0/418745611 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378035070 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.364+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33881a7390 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f336ffff640 1 -- 172.21.15.19:0/418745611 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3378042030 con 0x7f3388100b20 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 Processor -- start 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 -- start start 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 0x561a2fbff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a2fb582f0 con 0x561a2fbf25d0 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561a2faffcf0 con 0x561a2fbf3b40 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f338f5b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561a2fb542b0 con 0x561a2fbfd370 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.366+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f338759e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35540/0 (socket says 172.21.15.19:35540) 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 -- 172.21.15.19:0/3201825988 learned_addr learned my addr 172.21.15.19:0/3201825988 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f338659c640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 0x561a2fbff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 -- 172.21.15.19:0/3201825988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 msgr2=0x561a2fbff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 0x561a2fbff760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 -- 172.21.15.19:0/3201825988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 msgr2=0x561a2fbf2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 -- 172.21.15.19:0/3201825988 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2fae43a0 con 0x561a2fbf25d0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f338659c640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 0x561a2fbff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f338759e640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f336ffff640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 0x7f33700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f33780f7590 tx=0x7f33780f7680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf819bcd50b883 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f338759e640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 0x7f33700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f336ffff640 1 -- 172.21.15.19:0/418745611 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f33780ef410 con 0x7f3388100b20 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f336d7fa640 1 -- 172.21.15.19:0/3201825988 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780b7f70 con 0x561a2fbf25d0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f336d7fa640 1 -- 172.21.15.19:0/3201825988 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33780c4890 con 0x561a2fbf25d0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.367+0000 7f336d7fa640 1 -- 172.21.15.19:0/3201825988 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780c3350 con 0x561a2fbf25d0 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- 172.21.15.19:0/3201825988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 msgr2=0x561a2fbf2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338759e640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 0x7f33700a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3380009fd0 tx=0x7f3380009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f33780f7590 tx=0x7f33780f7680 comp rx=0 tx=0).stop 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- 172.21.15.19:0/3201825988 shutdown_connections 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbfd370 0x561a2fbff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fbf2a10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3201825988 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- 172.21.15.19:0/3201825988 >> 172.21.15.19:0/3201825988 conn(0x561a2fbbb310 msgr2=0x561a2fbeec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- 172.21.15.19:0/3201825988 shutdown_connections 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- 172.21.15.19:0/3201825988 wait complete. 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.368+0000 7f338f5b1740 1 -- start start 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 Processor -- start 2024-09-12T03:29:02.381 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 0x561a2fbfcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561a2fbffe30 con 0x561a2fbfd370 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561a2fb13060 con 0x561a2fbf25d0 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561a2fbfffb0 con 0x561a2fbf3b40 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338f5b1740 10 client.0 _ll_register_callbacks cb 0x561a2f8668c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 0x561a2fbfcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35556/0 (socket says 172.21.15.19:35556) 2024-09-12T03:29:02.382 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 -- 172.21.15.19:0/2593959953 learned_addr learned my addr 172.21.15.19:0/2593959953 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338659c640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 -- 172.21.15.19:0/2593959953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 msgr2=0x561a2fbf9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 -- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 msgr2=0x561a2fbfcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 0x561a2fbfcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338759e640 1 -- 172.21.15.19:0/2593959953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2f8c93c0 con 0x561a2fbfd370 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f3386d9d640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 0x561a2fbfcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.369+0000 7f338659c640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338759e640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3380012080 tx=0x7f3380004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2dacb538974b2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3380011c90 con 0x561a2fbfd370 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33780c4890 con 0x561a2fbfd370 2024-09-12T03:29:02.383 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f338001f420 con 0x561a2fbfd370 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f3380011e30 con 0x561a2fbfd370 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 10 client.17316 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x561a2fc0b8b0 con 0x561a2fbfd370 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33780c4890 con 0x561a2fbfd370 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f3380027070 con 0x561a2fbfd370 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 10 client.17316 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 10 client.17316 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 20 client.17316 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc0f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 4 client.17316 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.370+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a2fb65860 con 0x561a2fc0cd80 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.371+0000 7f3386d9d640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc0f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.384 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.371+0000 7f336aff5640 10 client.17316 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.371+0000 7f3386d9d640 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc0f200 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=534084661467d37a server_cookie=6b5ffe29a8d06ae1 in_seq=0 out_seq=0 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f336aff5640 1 -- 172.21.15.19:0/2593959953 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a2fb65860 con 0x561a2fc0cd80 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f336aff5640 10 client.17316 handle_command_reply: tid=0 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 client.17316 shutdown 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 msgr2=0x561a2fbfa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f3380012080 tx=0x7f3380004ab0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 shutdown_connections 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbfa5d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc0f200 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f336aff5640 0 client.17316 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.385 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 --2- 172.21.15.19:0/2593959953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf25d0 0x561a2fbfcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 >> 172.21.15.19:0/2593959953 conn(0x561a2fbbb310 msgr2=0x561a2fbf4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 shutdown_connections 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 1 -- 172.21.15.19:0/2593959953 wait complete. 2024-09-12T03:29:02.386 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.372+0000 7f338f5b1740 20 client.17316 trim_cache size 0 max 16384 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 Processor -- start 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 -- start start 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbfd370 0x561a2fc14b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33881af870 con 0x561a2fbf25d0 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33881b15f0 con 0x561a2fbfd370 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.385+0000 7f338f5b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33881aff40 con 0x561a2fbf3b40 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.390 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35566/0 (socket says 172.21.15.19:35566) 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 -- 172.21.15.19:0/1007392608 learned_addr learned my addr 172.21.15.19:0/1007392608 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f338659c640 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f338759e640 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbfd370 0x561a2fc14b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 -- 172.21.15.19:0/1007392608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 msgr2=0x561a2fbf8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 -- 172.21.15.19:0/1007392608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbfd370 msgr2=0x561a2fc14b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbfd370 0x561a2fc14b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 -- 172.21.15.19:0/1007392608 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a2fbf8550 con 0x561a2fbf25d0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f3386d9d640 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3378005d00 tx=0x7f3378005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7837580f92112689 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f336cff9640 1 -- 172.21.15.19:0/1007392608 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3378049040 con 0x561a2fbf25d0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f336cff9640 1 -- 172.21.15.19:0/1007392608 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33780bffd0 con 0x561a2fbf25d0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.386+0000 7f336cff9640 1 -- 172.21.15.19:0/1007392608 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33780bb630 con 0x561a2fbf25d0 2024-09-12T03:29:02.391 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- 172.21.15.19:0/1007392608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 msgr2=0x561a2fc09090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3378005d00 tx=0x7f3378005c90 comp rx=0 tx=0).stop 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- 172.21.15.19:0/1007392608 shutdown_connections 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbfd370 0x561a2fc14b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf3b40 0x561a2fbf8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 --2- 172.21.15.19:0/1007392608 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbf25d0 0x561a2fc09090 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- 172.21.15.19:0/1007392608 >> 172.21.15.19:0/1007392608 conn(0x561a2fbbb310 msgr2=0x561a2fbf88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- 172.21.15.19:0/1007392608 shutdown_connections 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- 172.21.15.19:0/1007392608 wait complete. 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 -- start start 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.387+0000 7f338f5b1740 1 Processor -- start 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf25d0 0x561a2fc05ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.392 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbf7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f33881b0e60 con 0x561a2fbfd370 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f33881b06d0 con 0x561a2fbf3b40 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f33881ad790 con 0x561a2fbf25d0 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338f5b1740 10 client.0 _ll_register_callbacks cb 0x7f3380012840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbf7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338759e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbf7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35580/0 (socket says 172.21.15.19:35580) 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338759e640 1 -- 172.21.15.19:0/3459817090 learned_addr learned my addr 172.21.15.19:0/3459817090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf25d0 0x561a2fc05ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 -- 172.21.15.19:0/3459817090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf25d0 msgr2=0x561a2fc05ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf25d0 0x561a2fc05ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.393 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 -- 172.21.15.19:0/3459817090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 msgr2=0x561a2fbf7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbf7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.388+0000 7f338659c640 1 -- 172.21.15.19:0/3459817090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a2fc76ae0 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338659c640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf6e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x561a2fbf9870 tx=0x7f337c00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5333155f30dfb81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c022060 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f337c004250 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33780bffd0 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f337c004450 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 10 client.26714 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x561a2fc77c80 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33780bffd0 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f337c01d7f0 con 0x561a2fbf3b40 2024-09-12T03:29:02.394 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 10 client.26714 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 10 client.26714 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 20 client.26714 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc7a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 4 client.26714 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.389+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a2fb5fd70 con 0x561a2fc0cd80 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc7a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f336aff5640 10 client.26714 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:02.395 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f3386d9d640 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc7a4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1eb84f6db1eca0f6 server_cookie=85ba1625e121ec0d in_seq=0 out_seq=0 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f336aff5640 1 -- 172.21.15.19:0/3459817090 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a2fb5fd70 con 0x561a2fc0cd80 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f336aff5640 10 client.26714 handle_command_reply: tid=0 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 client.26714 shutdown 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 msgr2=0x561a2fbf6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.396 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf6e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x561a2fbf9870 tx=0x7f337c00f790 comp rx=0 tx=0).stop 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 shutdown_connections 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561a2fbfd370 0x561a2fbf7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561a2fbf3b40 0x561a2fbf6e40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x561a2fc0cd80 0x561a2fc7a4d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 --2- 172.21.15.19:0/3459817090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561a2fbf25d0 0x561a2fc05ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 >> 172.21.15.19:0/3459817090 conn(0x561a2fbbb310 msgr2=0x561a2fbf90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f336aff5640 0 client.26714 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 shutdown_connections 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.390+0000 7f338f5b1740 1 -- 172.21.15.19:0/3459817090 wait complete. 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338f5b1740 20 client.26714 trim_cache size 0 max 16384 2024-09-12T03:29:02.397 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 msgr2=0x7f33700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 0x7f33700a5040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3380009fd0 tx=0x7f3380009500 comp rx=0 tx=0).stop 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 msgr2=0x7f338819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:02.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f3378006010 tx=0x7f3378004040 comp rx=0 tx=0).stop 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 shutdown_connections 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f338810e500 0x7f338819e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3388101440 0x7f338819ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f33700a2b80 0x7f33700a5040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 --2- 172.21.15.19:0/418745611 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3388100b20 0x7f338819a7e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 >> 172.21.15.19:0/418745611 conn(0x7f33880fc7d0 msgr2=0x7f33880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 shutdown_connections 2024-09-12T03:29:02.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:02.391+0000 7f338d1d8640 1 -- 172.21.15.19:0/418745611 wait complete. 2024-09-12T03:29:03.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:02 smithi135 ceph-mon[35474]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 467 op/s 2024-09-12T03:29:03.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:02 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:03.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:02 smithi121 ceph-mon[34963]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 467 op/s 2024-09-12T03:29:03.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:02 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:03.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:02 smithi019 ceph-mon[30953]: pgmap v637: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 467 op/s 2024-09-12T03:29:03.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:02 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:03.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:29:03.465 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1453318947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102ef0 msgr2=0x7f9f8410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1453318947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102ef0 0x7f9f8410da40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009ca0 tx=0x7f9f7c02a950 comp rx=0 tx=0).stop 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1453318947 shutdown_connections 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1453318947 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102ef0 0x7f9f8410da40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1453318947 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84102570 0x7f9f841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1453318947 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84100e20 0x7f9f84101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1453318947 >> 172.21.15.19:0/1453318947 conn(0x7f9f840fc7d0 msgr2=0x7f9f840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1453318947 shutdown_connections 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1453318947 wait complete. 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 Processor -- start 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.175+0000 7f9f8aeeb640 1 -- start start 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84100e20 0x7f9f8419a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84102ef0 0x7f9f8419f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84110f80 con 0x7f9f84102570 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84110e00 con 0x7f9f84102ef0 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f8aeeb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f84111100 con 0x7f9f84100e20 2024-09-12T03:29:04.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35600/0 (socket says 172.21.15.19:35600) 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 -- 172.21.15.19:0/1907873596 learned_addr learned my addr 172.21.15.19:0/1907873596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84100e20 0x7f9f8419a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f89488640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84102ef0 0x7f9f8419f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 -- 172.21.15.19:0/1907873596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84100e20 msgr2=0x7f9f8419a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84100e20 0x7f9f8419a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 -- 172.21.15.19:0/1907873596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84102ef0 msgr2=0x7f9f8419f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84102ef0 0x7f9f8419f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 -- 172.21.15.19:0/1907873596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8419f6f0 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.176+0000 7f9f83fff640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e9f0 tx=0x7f9f7000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.177+0000 7f9f81ffb640 1 -- 172.21.15.19:0/1907873596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7000cd80 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.177+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8419f9e0 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.177+0000 7f9f81ffb640 1 -- 172.21.15.19:0/1907873596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f70005500 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.177+0000 7f9f81ffb640 1 -- 172.21.15.19:0/1907873596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f70010690 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.177+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f841a72c0 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.179+0000 7f9f81ffb640 1 -- 172.21.15.19:0/1907873596 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f700046c0 con 0x7f9f84102570 2024-09-12T03:29:04.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.180+0000 7f9f81ffb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 0x7f9f740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.180+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 0x7f9f740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.180+0000 7f9f81ffb640 1 -- 172.21.15.19:0/1907873596 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f700cf3d0 con 0x7f9f84102570 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.180+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 0x7f9f740a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bd0 tx=0x7f9f78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 Processor -- start 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 -- start start 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557007fba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700806370 0x555700808760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5557007612f0 con 0x5557007fcb40 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x555700708cf0 con 0x5557007fb5d0 2024-09-12T03:29:04.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55570075d2b0 con 0x555700806370 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35614/0 (socket says 172.21.15.19:35614) 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 -- 172.21.15.19:0/1256602868 learned_addr learned my addr 172.21.15.19:0/1256602868 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f83fff640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557007fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f89488640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700806370 0x555700808760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 -- 172.21.15.19:0/1256602868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700806370 msgr2=0x555700808760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700806370 0x555700808760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 -- 172.21.15.19:0/1256602868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 msgr2=0x5557007fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557007fba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.181+0000 7f9f88c87640 1 -- 172.21.15.19:0/1256602868 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557006ed3a0 con 0x5557007fcb40 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.182+0000 7f9f83fff640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557007fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.182+0000 7f9f88c87640 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800e630 tx=0x7f9f78014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c95c8caa93b178 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.182+0000 7f9f6effd640 1 -- 172.21.15.19:0/1256602868 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78004020 con 0x5557007fcb40 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.182+0000 7f9f6effd640 1 -- 172.21.15.19:0/1256602868 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f780041c0 con 0x5557007fcb40 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.182+0000 7f9f6effd640 1 -- 172.21.15.19:0/1256602868 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7801c720 con 0x5557007fcb40 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/1256602868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 msgr2=0x5557007fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800e630 tx=0x7f9f78014000 comp rx=0 tx=0).stop 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/1256602868 shutdown_connections 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x555700806370 0x555700808760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557007fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/1256602868 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fb090 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/1256602868 >> 172.21.15.19:0/1256602868 conn(0x5557007c4310 msgr2=0x5557007f7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/1256602868 shutdown_connections 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.183+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/1256602868 wait complete. 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 -- start start 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 Processor -- start 2024-09-12T03:29:04.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700802ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557008035e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x555700809070 con 0x5557007fb5d0 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55570071c060 con 0x555700806370 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5557008091f0 con 0x5557007fcb40 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f8d2c4740 10 client.0 _ll_register_callbacks cb 0x55570046f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f88c87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f88c87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35630/0 (socket says 172.21.15.19:35630) 2024-09-12T03:29:04.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f88c87640 1 -- 172.21.15.19:0/2557089291 learned_addr learned my addr 172.21.15.19:0/2557089291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f89488640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557008035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.184+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700802ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 -- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 msgr2=0x555700802ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700802ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 -- 172.21.15.19:0/2557089291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 msgr2=0x5557008035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557008035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 -- 172.21.15.19:0/2557089291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557004d23c0 con 0x5557007fb5d0 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f89488640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557008035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800eaf0 tx=0x7f9f7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc7af148a532aee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78035060 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f78004370 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f780041c0 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f9f7801c480 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f8d2c4740 10 client.17334 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.185+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x555700814ae0 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f780041c0 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f9f78003a10 con 0x5557007fb5d0 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 10 client.17334 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:04.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 10 client.17334 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 20 client.17334 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x5557008182d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 4 client.17334 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55570076e860 con 0x555700815ee0 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.186+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x5557008182d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x5557008182d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d47d8620a7eed158 server_cookie=3172dba311989c23 in_seq=0 out_seq=0 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f63fff640 10 client.17334 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f63fff640 1 -- 172.21.15.19:0/2557089291 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55570076e860 con 0x555700815ee0 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f63fff640 10 client.17334 handle_command_reply: tid=0 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f8d2c4740 1 client.17334 shutdown 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 msgr2=0x555700805d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.201 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.187+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800eaf0 tx=0x7f9f7800edf0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 shutdown_connections 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x5557008182d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557008035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f63fff640 0 client.17334 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700802ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2557089291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fb5d0 0x555700805d40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 >> 172.21.15.19:0/2557089291 conn(0x5557007c4310 msgr2=0x5557007fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.202 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 shutdown_connections 2024-09-12T03:29:04.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2557089291 wait complete. 2024-09-12T03:29:04.203 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.188+0000 7f9f8d2c4740 20 client.17334 trim_cache size 0 max 16384 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 Processor -- start 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 -- start start 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fb5d0 0x5557007f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557007fe660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.209 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841af860 con 0x5557007fcb40 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b15e0 con 0x555700806370 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841aff30 con 0x5557007fb5d0 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35644/0 (socket says 172.21.15.19:35644) 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 -- 172.21.15.19:0/635366403 learned_addr learned my addr 172.21.15.19:0/635366403 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f88c87640 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fb5d0 0x5557007f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f89488640 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557007fe660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.210 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 -- 172.21.15.19:0/635366403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fb5d0 msgr2=0x5557007f7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fb5d0 0x5557007f7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 -- 172.21.15.19:0/635366403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 msgr2=0x5557007fe660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557007fe660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.202+0000 7f9f83fff640 1 -- 172.21.15.19:0/635366403 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557007f8da0 con 0x5557007fcb40 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f83fff640 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9f7009b940 tx=0x7f9f70097fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fce9f15cd54d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f6e7fc640 1 -- 172.21.15.19:0/635366403 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7009fff0 con 0x5557007fcb40 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f6e7fc640 1 -- 172.21.15.19:0/635366403 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f700a48b0 con 0x5557007fcb40 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f6e7fc640 1 -- 172.21.15.19:0/635366403 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7009bea0 con 0x5557007fcb40 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/635366403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 msgr2=0x5557007fe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.203+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9f7009b940 tx=0x7f9f70097fa0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/635366403 shutdown_connections 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x555700806370 0x5557007fe660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5557007fcb40 0x5557007fe120 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/635366403 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fb5d0 0x5557007f7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/635366403 >> 172.21.15.19:0/635366403 conn(0x5557007c4310 msgr2=0x555700805070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/635366403 shutdown_connections 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/635366403 wait complete. 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 -- start start 2024-09-12T03:29:04.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 Processor -- start 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557008000e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700800620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.204+0000 7f9f8d2c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 0x555700800b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b0e50 con 0x555700806370 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841b06c0 con 0x5557007fb5d0 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f8d2c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9f841ad780 con 0x5557007fcb40 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f8d2c4740 10 client.0 _ll_register_callbacks cb 0x7f9f70009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f89488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 0x555700800b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557008000e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f89488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 0x555700800b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35658/0 (socket says 172.21.15.19:35658) 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f89488640 1 -- 172.21.15.19:0/2398916177 learned_addr learned my addr 172.21.15.19:0/2398916177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700800620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 -- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 msgr2=0x555700800620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700800620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 -- 172.21.15.19:0/2398916177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 msgr2=0x555700800b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 0x555700800b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 -- 172.21.15.19:0/2398916177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55570087f9d0 con 0x5557007fb5d0 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700800620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f88c87640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557008000e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f78011940 tx=0x7f9f780054c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700e2356a02d35b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f63fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78040020 con 0x5557007fb5d0 2024-09-12T03:29:04.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9f78003800 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.205+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f700a48b0 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f9f7802a070 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 10 client.26732 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x555700880cb0 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f700a48b0 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f9f78026b50 con 0x5557007fb5d0 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 10 client.26732 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 10 client.26732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:29:04.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 20 client.26732 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x555700883350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 4 client.26732 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555700768d70 con 0x555700815ee0 2024-09-12T03:29:04.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.206+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x555700883350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f83fff640 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x555700883350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b45f4966e62771a server_cookie=2c11fed2729a10da in_seq=0 out_seq=0 2024-09-12T03:29:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f63fff640 10 client.26732 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f63fff640 1 -- 172.21.15.19:0/2398916177 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555700768d70 con 0x555700815ee0 2024-09-12T03:29:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f63fff640 10 client.26732 handle_command_reply: tid=0 2024-09-12T03:29:04.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 client.26732 shutdown 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 msgr2=0x5557008000e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557008000e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f78011940 tx=0x7f9f780054c0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 shutdown_connections 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x555700815ee0 0x555700883350 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x555700806370 0x555700800b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f63fff640 0 client.26732 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5557007fcb40 0x555700800620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 --2- 172.21.15.19:0/2398916177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5557007fb5d0 0x5557008000e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 >> 172.21.15.19:0/2398916177 conn(0x5557007c4310 msgr2=0x5557007f64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 shutdown_connections 2024-09-12T03:29:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 1 -- 172.21.15.19:0/2398916177 wait complete. 2024-09-12T03:29:04.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.207+0000 7f9f8d2c4740 20 client.26732 trim_cache size 0 max 16384 2024-09-12T03:29:04.219 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 msgr2=0x7f9f740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 0x7f9f740a5040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9f78009bd0 tx=0x7f9f78005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 msgr2=0x7f9f8419ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f9f7000e9f0 tx=0x7f9f7000edf0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 shutdown_connections 2024-09-12T03:29:04.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9f84102ef0 0x7f9f8419f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9f740a2b80 0x7f9f740a5040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9f84102570 0x7f9f8419ac80 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 --2- 172.21.15.19:0/1907873596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9f84100e20 0x7f9f8419a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 >> 172.21.15.19:0/1907873596 conn(0x7f9f840fc7d0 msgr2=0x7f9f84069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 shutdown_connections 2024-09-12T03:29:04.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:04.208+0000 7f9f8aeeb640 1 -- 172.21.15.19:0/1907873596 wait complete. 2024-09-12T03:29:04.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:04 smithi019 ceph-mon[30953]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-12T03:29:05.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:29:05.043 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:04 smithi135 ceph-mon[35474]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-12T03:29:05.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:04 smithi121 ceph-mon[34963]: pgmap v638: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 431 op/s 2024-09-12T03:29:05.314 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:05.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 -- 172.21.15.19:0/4029739168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c100e20 msgr2=0x7f768c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 --2- 172.21.15.19:0/4029739168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c100e20 0x7f768c101200 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f767c009900 tx=0x7f767c02a480 comp rx=0 tx=0).stop 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 -- 172.21.15.19:0/4029739168 shutdown_connections 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 --2- 172.21.15.19:0/4029739168 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c102ef0 0x7f768c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 --2- 172.21.15.19:0/4029739168 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102570 0x7f768c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 --2- 172.21.15.19:0/4029739168 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c100e20 0x7f768c101200 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 -- 172.21.15.19:0/4029739168 >> 172.21.15.19:0/4029739168 conn(0x7f768c0fc7d0 msgr2=0x7f768c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 -- 172.21.15.19:0/4029739168 shutdown_connections 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.843+0000 7f7691c39640 1 -- 172.21.15.19:0/4029739168 wait complete. 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 Processor -- start 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 -- start start 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 0x7f768c1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102ef0 0x7f768c19cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f768c110f80 con 0x7f768c102570 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f768c110e00 con 0x7f768c102ef0 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.844+0000 7f7691c39640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f768c111100 con 0x7f768c100e20 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 0x7f768c1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35676/0 (socket says 172.21.15.19:35676) 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 -- 172.21.15.19:0/2810841304 learned_addr learned my addr 172.21.15.19:0/2810841304 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768bfff640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102ef0 0x7f768c19cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 -- 172.21.15.19:0/2810841304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 msgr2=0x7f768c1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 0x7f768c1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 -- 172.21.15.19:0/2810841304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102ef0 msgr2=0x7f768c19cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.868 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102ef0 0x7f768c19cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 -- 172.21.15.19:0/2810841304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c19d4a0 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768b7fe640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 0x7f768c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f768affd640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f768000e9f0 tx=0x7f768000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f7688ff9640 1 -- 172.21.15.19:0/2810841304 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768000cd80 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c19d790 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f7688ff9640 1 -- 172.21.15.19:0/2810841304 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7680010910 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f7688ff9640 1 -- 172.21.15.19:0/2810841304 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76800188f0 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.845+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c1a5070 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7688ff9640 1 -- 172.21.15.19:0/2810841304 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7680010ac0 con 0x7f768c102570 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 Processor -- start 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 -- start start 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a958826a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.869 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7688ff9640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 0x7f76740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a958837b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a95878c2f0 con 0x55a958827b40 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a958733cf0 con 0x55a958835780 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f7694012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a9587882b0 con 0x55a9588265d0 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f768b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a958826a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35692/0 (socket says 172.21.15.19:35692) 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 -- 172.21.15.19:0/3732141684 learned_addr learned my addr 172.21.15.19:0/3732141684 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.848+0000 7f768b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a958826a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50800/0 (socket says 172.21.15.19:50800) 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f7688ff9640 1 -- 172.21.15.19:0/2810841304 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f76800cf3c0 con 0x7f768c102570 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768affd640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 0x7f76740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a958837b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 -- 172.21.15.19:0/3732141684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 msgr2=0x55a958826a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a958826a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 -- 172.21.15.19:0/3732141684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 msgr2=0x55a958837b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a958837b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 -- 172.21.15.19:0/3732141684 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9587183a0 con 0x55a958827b40 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768bfff640 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7684013930 tx=0x7f7684009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2412c1f586ad59c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f768affd640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 0x7f76740a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f768000e9c0 tx=0x7f768000cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f7671ffb640 1 -- 172.21.15.19:0/3732141684 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004b20 con 0x55a958827b40 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f7671ffb640 1 -- 172.21.15.19:0/3732141684 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684003e50 con 0x55a958827b40 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.849+0000 7f7671ffb640 1 -- 172.21.15.19:0/3732141684 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004230 con 0x55a958827b40 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- 172.21.15.19:0/3732141684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 msgr2=0x55a958826090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7684013930 tx=0x7f7684009be0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- 172.21.15.19:0/3732141684 shutdown_connections 2024-09-12T03:29:05.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a958837b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a958826a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 --2- 172.21.15.19:0/3732141684 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958827b40 0x55a958826090 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- 172.21.15.19:0/3732141684 >> 172.21.15.19:0/3732141684 conn(0x55a9587ef310 msgr2=0x55a958822c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- 172.21.15.19:0/3732141684 shutdown_connections 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- 172.21.15.19:0/3732141684 wait complete. 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.850+0000 7f7694012740 1 -- start start 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 Processor -- start 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a9588323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958835780 0x55a958832930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a958838480 con 0x55a9588265d0 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a958747060 con 0x55a958827b40 2024-09-12T03:29:05.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a958838600 con 0x55a958835780 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f7694012740 10 client.0 _ll_register_callbacks cb 0x55a95849a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35700/0 (socket says 172.21.15.19:35700) 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 -- 172.21.15.19:0/1227268244 learned_addr learned my addr 172.21.15.19:0/1227268244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768b7fe640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a9588323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958835780 0x55a958832930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 -- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958835780 msgr2=0x55a958832930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958835780 0x55a958832930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 -- 172.21.15.19:0/1227268244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 msgr2=0x55a9588323b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a9588323b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768bfff640 1 -- 172.21.15.19:0/1227268244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9584fd3c0 con 0x55a9588265d0 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.851+0000 7f768b7fe640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a9588323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:05.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f768bfff640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7684006010 tx=0x7f768400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1807a840dd8c003 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684004c40 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7684004de0 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7684003e50 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f7684035050 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 10 client.17352 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55a958843eb0 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684003e50 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f7684005130 con 0x55a9588265d0 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 10 client.17352 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 10 client.17352 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 20 client.17352 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:05.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.852+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a958845200 0x55a9588475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.853+0000 7f7694012740 4 client.17352 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.853+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a958799860 con 0x55a958845200 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.853+0000 7f768b7fe640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a958845200 0x55a9588475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.853+0000 7f766affd640 10 client.17352 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.853+0000 7f768b7fe640 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a958845200 0x55a9588475f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c446cbe2f6d4937 server_cookie=179de5e5a0493eb7 in_seq=0 out_seq=0 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f766affd640 1 -- 172.21.15.19:0/1227268244 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a958799860 con 0x55a958845200 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f766affd640 10 client.17352 handle_command_reply: tid=0 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 client.17352 shutdown 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 msgr2=0x55a958835100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7684006010 tx=0x7f768400f8c0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 shutdown_connections 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a958845200 0x55a9588475f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958835780 0x55a958832930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a9588323b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 --2- 172.21.15.19:0/1227268244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958835100 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f766affd640 0 client.17352 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 >> 172.21.15.19:0/1227268244 conn(0x55a9587ef310 msgr2=0x55a958828960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 shutdown_connections 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 1 -- 172.21.15.19:0/1227268244 wait complete. 2024-09-12T03:29:05.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.854+0000 7f7694012740 20 client.17352 trim_cache size 0 max 16384 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 Processor -- start 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 -- start start 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a95884d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1ab3d0 con 0x55a9588265d0 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1ad150 con 0x55a958835780 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f7694012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1abaa0 con 0x55a958827b40 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f768b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35706/0 (socket says 172.21.15.19:35706) 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f768bfff640 1 -- 172.21.15.19:0/3517826042 learned_addr learned my addr 172.21.15.19:0/3517826042 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.868+0000 7f768b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50822/0 (socket says 172.21.15.19:50822) 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a95884d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 -- 172.21.15.19:0/3517826042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 msgr2=0x55a95884ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 -- 172.21.15.19:0/3517826042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 msgr2=0x55a95884d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a95884d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 -- 172.21.15.19:0/3517826042 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a958832ca0 con 0x55a9588265d0 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768b7fe640 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f768bfff640 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7684009560 tx=0x7f76840021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ba5ef192786ffa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f76717fa640 1 -- 172.21.15.19:0/3517826042 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684005970 con 0x55a9588265d0 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f76717fa640 1 -- 172.21.15.19:0/3517826042 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684003e50 con 0x55a9588265d0 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.869+0000 7f76717fa640 1 -- 172.21.15.19:0/3517826042 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684020940 con 0x55a9588265d0 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- 172.21.15.19:0/3517826042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 msgr2=0x55a958841720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7684009560 tx=0x7f76840021d0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- 172.21.15.19:0/3517826042 shutdown_connections 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958835780 0x55a95884d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a958827b40 0x55a95884ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 --2- 172.21.15.19:0/3517826042 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a9588265d0 0x55a958841720 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- 172.21.15.19:0/3517826042 >> 172.21.15.19:0/3517826042 conn(0x55a9587ef310 msgr2=0x55a958830c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- 172.21.15.19:0/3517826042 shutdown_connections 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- 172.21.15.19:0/3517826042 wait complete. 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.870+0000 7f7694012740 1 -- start start 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 Processor -- start 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a95883dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958835780 0x55a958838df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1ac9c0 con 0x55a958835780 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1ac230 con 0x55a958827b40 2024-09-12T03:29:05.884 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f768c1a92f0 con 0x55a9588265d0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f7694012740 10 client.0 _ll_register_callbacks cb 0x7f76840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958835780 0x55a958838df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50838/0 (socket says 172.21.15.19:50838) 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 -- 172.21.15.19:0/499259812 learned_addr learned my addr 172.21.15.19:0/499259812 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768b7fe640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a95883dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 -- 172.21.15.19:0/499259812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 msgr2=0x55a95883dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a95883dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 -- 172.21.15.19:0/499259812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958835780 msgr2=0x55a958838df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958835780 0x55a958838df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.871+0000 7f768bfff640 1 -- 172.21.15.19:0/499259812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a958835470 con 0x55a9588265d0 2024-09-12T03:29:05.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f768bfff640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7684013930 tx=0x7f7684013890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e40515c883a1a17c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76840059a0 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7684003e50 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7684020660 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f76840052d0 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 10 client.26719 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55a9588af7a0 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7684003e50 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f76840227b0 con 0x55a9588265d0 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 10 client.26719 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 10 client.26719 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 20 client.26719 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:05.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.872+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a9588afe80 0x55a9588b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f7694012740 4 client.26719 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a958793d70 con 0x55a9588afe80 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f768b7fe640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a9588afe80 0x55a9588b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f766affd640 10 client.26719 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f768b7fe640 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a9588afe80 0x55a9588b22a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9b732879dd028c43 server_cookie=4490b803600006cb in_seq=0 out_seq=0 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f766affd640 1 -- 172.21.15.19:0/499259812 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a958793d70 con 0x55a9588afe80 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f766affd640 10 client.26719 handle_command_reply: tid=0 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f7694012740 1 client.26719 shutdown 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 msgr2=0x55a95883d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.873+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7684013930 tx=0x7f7684013890 comp rx=0 tx=0).stop 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 shutdown_connections 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x55a9588afe80 0x55a9588b22a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a958835780 0x55a958838df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a958827b40 0x55a95883dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f766affd640 0 client.26719 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 --2- 172.21.15.19:0/499259812 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a9588265d0 0x55a95883d960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 >> 172.21.15.19:0/499259812 conn(0x55a9587ef310 msgr2=0x55a958821500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 shutdown_connections 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 1 -- 172.21.15.19:0/499259812 wait complete. 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7694012740 20 client.26719 trim_cache size 0 max 16384 2024-09-12T03:29:05.888 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 msgr2=0x7f76740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 0x7f76740a5040 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f768000e9c0 tx=0x7f768000cf20 comp rx=0 tx=0).stop 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 msgr2=0x7f768c198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.874+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f768000e9f0 tx=0x7f768000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 shutdown_connections 2024-09-12T03:29:05.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f768c102ef0 0x7f768c19cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f76740a2b80 0x7f76740a5040 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f768c102570 0x7f768c198a30 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 --2- 172.21.15.19:0/2810841304 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f768c100e20 0x7f768c1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 >> 172.21.15.19:0/2810841304 conn(0x7f768c0fc7d0 msgr2=0x7f768c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 shutdown_connections 2024-09-12T03:29:05.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:05.875+0000 7f7691c39640 1 -- 172.21.15.19:0/2810841304 wait complete. 2024-09-12T03:29:06.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:29:07.029 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:07.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:06 smithi135 ceph-mon[35474]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2024-09-12T03:29:07.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:06 smithi121 ceph-mon[34963]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2024-09-12T03:29:07.152 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:06 smithi019 ceph-mon[30953]: pgmap v639: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 470 op/s 2024-09-12T03:29:07.562 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 -- 172.21.15.19:0/2384164528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e8102df0 msgr2=0x7f83e81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 --2- 172.21.15.19:0/2384164528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e8102df0 0x7f83e81031d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009900 tx=0x7f83e402a410 comp rx=0 tx=0).stop 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 -- 172.21.15.19:0/2384164528 shutdown_connections 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 --2- 172.21.15.19:0/2384164528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e810c2f0 0x7f83e810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 --2- 172.21.15.19:0/2384164528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e81037a0 0x7f83e810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 --2- 172.21.15.19:0/2384164528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e8102df0 0x7f83e81031d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.530+0000 7f83efa2c640 1 -- 172.21.15.19:0/2384164528 >> 172.21.15.19:0/2384164528 conn(0x7f83e80fc7d0 msgr2=0x7f83e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- 172.21.15.19:0/2384164528 shutdown_connections 2024-09-12T03:29:07.563 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- 172.21.15.19:0/2384164528 wait complete. 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 Processor -- start 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- start start 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 0x7f83e8198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e810c2f0 0x7f83e819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8112110 con 0x7f83e81037a0 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8111f90 con 0x7f83e810c2f0 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.531+0000 7f83efa2c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8112290 con 0x7f83e8102df0 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.564 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 0x7f83e8198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83edfc9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e810c2f0 0x7f83e819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35730/0 (socket says 172.21.15.19:35730) 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 -- 172.21.15.19:0/645462170 learned_addr learned my addr 172.21.15.19:0/645462170 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 0x7f83e8198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50868/0 (socket says 172.21.15.19:50868) 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 -- 172.21.15.19:0/645462170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 msgr2=0x7f83e8198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 0x7f83e8198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 -- 172.21.15.19:0/645462170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e810c2f0 msgr2=0x7f83e819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e810c2f0 0x7f83e819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 -- 172.21.15.19:0/645462170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819d5c0 con 0x7f83e81037a0 2024-09-12T03:29:07.565 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9f0 tx=0x7f83d800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83de7fc640 1 -- 172.21.15.19:0/645462170 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800ccc0 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83de7fc640 1 -- 172.21.15.19:0/645462170 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83d8010910 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83de7fc640 1 -- 172.21.15.19:0/645462170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8018920 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819d8b0 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.532+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a5190 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.535+0000 7f83de7fc640 1 -- 172.21.15.19:0/645462170 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83d8010ac0 con 0x7f83e81037a0 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.535+0000 7f83f1e05740 1 Processor -- start 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.535+0000 7f83f1e05740 1 -- start start 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83de7fc640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 0x7f83d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.566 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04dffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e0c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc04d652f0 con 0x55fc04e0a370 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fc04d0ccf0 con 0x55fc04e00b40 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83f1e05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fc04d612b0 con 0x55fc04dff5d0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:45124/0 (socket says 172.21.15.19:45124) 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04dffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 -- 172.21.15.19:0/1949564989 learned_addr learned my addr 172.21.15.19:0/1949564989 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83de7fc640 1 -- 172.21.15.19:0/645462170 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f83d80cf320 con 0x7f83e81037a0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83edfc9640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 0x7f83d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83edfc9640 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e0c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 -- 172.21.15.19:0/1949564989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 msgr2=0x55fc04dffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04dffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 -- 172.21.15.19:0/1949564989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 msgr2=0x55fc04e0c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e0c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83ed7c8640 1 -- 172.21.15.19:0/1949564989 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc04cf13a0 con 0x55fc04e00b40 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.536+0000 7f83edfc9640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 0x7f83d40a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55fc04e00070 tx=0x7f83e0009b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009900 tx=0x7f83e4005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d47cfb4160c7be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83d37fe640 1 -- 172.21.15.19:0/1949564989 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4040070 con 0x55fc04e00b40 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83d37fe640 1 -- 172.21.15.19:0/1949564989 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4031870 con 0x55fc04e00b40 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83d37fe640 1 -- 172.21.15.19:0/1949564989 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035040 con 0x55fc04e00b40 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83f1e05740 1 -- 172.21.15.19:0/1949564989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 msgr2=0x55fc04dff090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.537+0000 7f83f1e05740 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009900 tx=0x7f83e4005be0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- 172.21.15.19:0/1949564989 shutdown_connections 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e0c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04dffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- 172.21.15.19:0/1949564989 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04dff090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- 172.21.15.19:0/1949564989 >> 172.21.15.19:0/1949564989 conn(0x55fc04dc8310 msgr2=0x55fc04dfbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- 172.21.15.19:0/1949564989 shutdown_connections 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- 172.21.15.19:0/1949564989 wait complete. 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- start start 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 Processor -- start 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 0x55fc04e06fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 0x55fc04e07530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fc04e0d010 con 0x55fc04e00b40 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fc04d20060 con 0x55fc04dff5d0 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.538+0000 7f83f1e05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fc04e0d190 con 0x55fc04e0a370 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83f1e05740 10 client.0 _ll_register_callbacks cb 0x55fc04a738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:07.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 0x55fc04e06fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:45138/0 (socket says 172.21.15.19:45138) 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 -- 172.21.15.19:0/2308602688 learned_addr learned my addr 172.21.15.19:0/2308602688 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83edfc9640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 0x55fc04e07530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 -- 172.21.15.19:0/2308602688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 msgr2=0x55fc04e07530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 0x55fc04e07530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 -- 172.21.15.19:0/2308602688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 msgr2=0x55fc04e06fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 0x55fc04e06fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 -- 172.21.15.19:0/2308602688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc04ad63c0 con 0x55fc04dff5d0 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 0x55fc04e06fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:07.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83edfc9640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 0x55fc04e07530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83e4000c00 tx=0x7f83e402ab90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2b1b11c7517065 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e404b020 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f83e4004250 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.539+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e4031870 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f83e4035070 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 10 client.26762 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fc04e18a40 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4031870 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f83e40035f0 con 0x55fc04dff5d0 2024-09-12T03:29:07.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 10 client.26762 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 10 client.26762 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 20 client.26762 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e19d90 0x55fc04e1c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 4 client.26762 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc04d72860 con 0x55fc04e19d90 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.540+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e19d90 0x55fc04e1c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.541+0000 7f83d0ff9640 10 client.26762 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.541+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e19d90 0x55fc04e1c180 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa27c08c870849d7 server_cookie=6d49687c27b2253 in_seq=0 out_seq=0 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83d0ff9640 1 -- 172.21.15.19:0/2308602688 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc04d72860 con 0x55fc04e19d90 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83d0ff9640 10 client.26762 handle_command_reply: tid=0 2024-09-12T03:29:07.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 client.26762 shutdown 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 msgr2=0x55fc04e09d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f83e4000c00 tx=0x7f83e402ab90 comp rx=0 tx=0).stop 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 shutdown_connections 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e19d90 0x55fc04e1c180 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e0a370 0x55fc04e07530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e00b40 0x55fc04e06fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 --2- 172.21.15.19:0/2308602688 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e09d00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83d0ff9640 0 client.26762 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 >> 172.21.15.19:0/2308602688 conn(0x55fc04dc8310 msgr2=0x55fc04e01960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 shutdown_connections 2024-09-12T03:29:07.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 1 -- 172.21.15.19:0/2308602688 wait complete. 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.542+0000 7f83f1e05740 20 client.26762 trim_cache size 0 max 16384 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.555+0000 7f83f1e05740 1 Processor -- start 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.555+0000 7f83f1e05740 1 -- start start 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04e162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04e00300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e06840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ad580 con 0x55fc04e0a370 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81af300 con 0x55fc04e00b40 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83f1e05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81adc50 con 0x55fc04dff5d0 2024-09-12T03:29:07.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83edfc9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e06840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04e162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83edfc9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e06840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35758/0 (socket says 172.21.15.19:35758) 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83edfc9640 1 -- 172.21.15.19:0/3478373339 learned_addr learned my addr 172.21.15.19:0/3478373339 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04e00300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 -- 172.21.15.19:0/3478373339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 msgr2=0x55fc04e00300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04e00300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 -- 172.21.15.19:0/3478373339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 msgr2=0x55fc04e06840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e06840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 -- 172.21.15.19:0/3478373339 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc04e078a0 con 0x55fc04dff5d0 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04e00300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:07.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04e162b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83e4003c50 tx=0x7f83e4005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd2a8830b2177c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.556+0000 7f83d2ffd640 1 -- 172.21.15.19:0/3478373339 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e404b070 con 0x55fc04dff5d0 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83d2ffd640 1 -- 172.21.15.19:0/3478373339 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4003cd0 con 0x55fc04dff5d0 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83d2ffd640 1 -- 172.21.15.19:0/3478373339 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4040040 con 0x55fc04dff5d0 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 -- 172.21.15.19:0/3478373339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 msgr2=0x55fc04e162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04e162b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f83e4003c50 tx=0x7f83e4005e50 comp rx=0 tx=0).stop 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 -- 172.21.15.19:0/3478373339 shutdown_connections 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e06840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04e00b40 0x55fc04e00300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3478373339 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04dff5d0 0x55fc04e162b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 -- 172.21.15.19:0/3478373339 >> 172.21.15.19:0/3478373339 conn(0x55fc04dc8310 msgr2=0x55fc04df6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 -- 172.21.15.19:0/3478373339 shutdown_connections 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.557+0000 7f83f1e05740 1 -- 172.21.15.19:0/3478373339 wait complete. 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 -- start start 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 Processor -- start 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81aeb70 con 0x55fc04e0a370 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ae3e0 con 0x55fc04dff5d0 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81ab4a0 con 0x55fc04e00b40 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83f1e05740 10 client.0 _ll_register_callbacks cb 0x7f83e0011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:07.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83edfc9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83edfc9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35766/0 (socket says 172.21.15.19:35766) 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83ed7c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:45152/0 (socket says 172.21.15.19:45152) 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83edfc9640 1 -- 172.21.15.19:0/3140576026 learned_addr learned my addr 172.21.15.19:0/3140576026 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.558+0000 7f83ecfc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50914/0 (socket says 172.21.15.19:50914) 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 -- 172.21.15.19:0/3140576026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 msgr2=0x55fc04e0d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 -- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 msgr2=0x55fc04e0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 -- 172.21.15.19:0/3140576026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc04e0e260 con 0x55fc04e0a370 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83ecfc7640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83edfc9640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f83e000fef0 tx=0x7f83e0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91de84b508799fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004b20 con 0x55fc04e0a370 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f83e001a070 con 0x55fc04e0a370 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83e4003cd0 con 0x55fc04e0a370 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f83e0004cc0 con 0x55fc04e0a370 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83f1e05740 10 client.17376 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:07.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55fc04e83e40 con 0x55fc04e0a370 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.559+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4003cd0 con 0x55fc04e0a370 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f83e00051e0 con 0x55fc04e0a370 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 10 client.17376 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 10 client.17376 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 20 client.17376 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e846a0 0x55fc04e86ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 4 client.17376 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc04d6cd70 con 0x55fc04e846a0 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e846a0 0x55fc04e86ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83d0ff9640 10 client.17376 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:07.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.560+0000 7f83ed7c8640 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e846a0 0x55fc04e86ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55424dbd12f38daf server_cookie=ffb8374eaf914247 in_seq=0 out_seq=0 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.561+0000 7f83d0ff9640 1 -- 172.21.15.19:0/3140576026 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc04d6cd70 con 0x55fc04e846a0 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.561+0000 7f83d0ff9640 10 client.17376 handle_command_reply: tid=0 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.561+0000 7f83f1e05740 1 client.17376 shutdown 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.561+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 msgr2=0x55fc04e82830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.561+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f83e000fef0 tx=0x7f83e0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 shutdown_connections 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55fc04e846a0 0x55fc04e86ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fc04e0a370 0x55fc04e82830 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fc04e00b40 0x55fc04e0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 --2- 172.21.15.19:0/3140576026 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fc04dff5d0 0x55fc04e0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 >> 172.21.15.19:0/3140576026 conn(0x55fc04dc8310 msgr2=0x55fc04df6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83d0ff9640 0 client.17376 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 shutdown_connections 2024-09-12T03:29:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 1 -- 172.21.15.19:0/3140576026 wait complete. 2024-09-12T03:29:07.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83f1e05740 20 client.17376 trim_cache size 0 max 16384 2024-09-12T03:29:07.583 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 msgr2=0x7f83d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 0x7f83d40a5040 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x55fc04e00070 tx=0x7f83e0009b70 comp rx=0 tx=0).stop 2024-09-12T03:29:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 msgr2=0x7f83e8198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9f0 tx=0x7f83d800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 shutdown_connections 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83e810c2f0 0x7f83e819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83d40a2b80 0x7f83d40a5040 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f83e81037a0 0x7f83e8198b50 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.562+0000 7f83efa2c640 1 --2- 172.21.15.19:0/645462170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f83e8102df0 0x7f83e8198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.563+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 >> 172.21.15.19:0/645462170 conn(0x7f83e80fc7d0 msgr2=0x7f83e80fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.563+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 shutdown_connections 2024-09-12T03:29:07.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:07.563+0000 7f83efa2c640 1 -- 172.21.15.19:0/645462170 wait complete. 2024-09-12T03:29:08.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:29:09.047 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:08 smithi135 ceph-mon[35474]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-12T03:29:09.129 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:09.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:08 smithi121 ceph-mon[34963]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-12T03:29:09.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:08 smithi019 ceph-mon[30953]: pgmap v640: 97 pgs: 97 active+clean; 2.2 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 493 op/s 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 -- 172.21.15.19:0/593932710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 msgr2=0x7f2df810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 --2- 172.21.15.19:0/593932710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df810da40 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009ca0 tx=0x7f2df002a950 comp rx=0 tx=0).stop 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 -- 172.21.15.19:0/593932710 shutdown_connections 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 --2- 172.21.15.19:0/593932710 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df810da40 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 --2- 172.21.15.19:0/593932710 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8102570 0x7f2df81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 --2- 172.21.15.19:0/593932710 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8100e20 0x7f2df8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 -- 172.21.15.19:0/593932710 >> 172.21.15.19:0/593932710 conn(0x7f2df80fc7d0 msgr2=0x7f2df80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 -- 172.21.15.19:0/593932710 shutdown_connections 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.620+0000 7f2dfe779640 1 -- 172.21.15.19:0/593932710 wait complete. 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.621+0000 7f2dfe779640 1 Processor -- start 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.621+0000 7f2dfe779640 1 -- start start 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8100e20 0x7f2df819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 0x7f2df819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8110f80 con 0x7f2df8102ef0 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8110e00 con 0x7f2df8100e20 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfe779640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2df8111100 con 0x7f2df8102570 2024-09-12T03:29:09.633 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 0x7f2df819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35790/0 (socket says 172.21.15.19:35790) 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 -- 172.21.15.19:0/4131098464 learned_addr learned my addr 172.21.15.19:0/4131098464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2df7fff640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8100e20 0x7f2df819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 -- 172.21.15.19:0/4131098464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 msgr2=0x7f2df819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 0x7f2df819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 -- 172.21.15.19:0/4131098464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8100e20 msgr2=0x7f2df819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8100e20 0x7f2df819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 -- 172.21.15.19:0/4131098464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df819f660 con 0x7f2df8102ef0 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2df77fe640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 0x7f2df819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.622+0000 7f2dfcd16640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004af0 tx=0x7f2df002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.634 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.623+0000 7f2df57fa640 1 -- 172.21.15.19:0/4131098464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0034040 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.623+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df819f8f0 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.623+0000 7f2df57fa640 1 -- 172.21.15.19:0/4131098464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df0030920 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.623+0000 7f2df57fa640 1 -- 172.21.15.19:0/4131098464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0004630 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.623+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df81a7230 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.625+0000 7f2df57fa640 1 -- 172.21.15.19:0/4131098464 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2df0037020 con 0x7f2df8102ef0 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.625+0000 7f2e00b52740 1 Processor -- start 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.625+0000 7f2e00b52740 1 -- start start 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 0x56192250f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 0x56192251c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5619224752f0 con 0x56192251a370 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56192241ccf0 con 0x561922510b40 2024-09-12T03:29:09.635 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2e00b52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5619224712b0 con 0x56192250f5d0 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 0x56192250f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 0x56192250f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:45182/0 (socket says 172.21.15.19:45182) 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2df77fe640 1 -- 172.21.15.19:0/3275181236 learned_addr learned my addr 172.21.15.19:0/3275181236 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.626+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 0x56192251c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df57fa640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 -- 172.21.15.19:0/3275181236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 msgr2=0x56192250fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 -- 172.21.15.19:0/3275181236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 msgr2=0x56192250f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 0x56192250f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2dfcd16640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 -- 172.21.15.19:0/3275181236 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619224013a0 con 0x56192251a370 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df57fa640 1 -- 172.21.15.19:0/4131098464 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2df00efdf0 con 0x7f2df8102ef0 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 0x56192251c760 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011be0 tx=0x7f2dec005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2270f7de437ee6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2dde7fc640 1 -- 172.21.15.19:0/3275181236 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec011d90 con 0x56192251a370 2024-09-12T03:29:09.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2dde7fc640 1 -- 172.21.15.19:0/3275181236 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec0118a0 con 0x56192251a370 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.627+0000 7f2dde7fc640 1 -- 172.21.15.19:0/3275181236 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec003b50 con 0x56192251a370 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 -- 172.21.15.19:0/3275181236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 msgr2=0x56192251c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2dfcd16640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 0x7f2de00a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2df00b8fd0 tx=0x7f2df0087280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 0x56192251c760 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011be0 tx=0x7f2dec005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 -- 172.21.15.19:0/3275181236 shutdown_connections 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192251a370 0x56192251c760 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3275181236 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561922510b40 0x56192250f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 -- 172.21.15.19:0/3275181236 >> 172.21.15.19:0/3275181236 conn(0x5619224d8310 msgr2=0x56192250bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 -- 172.21.15.19:0/3275181236 shutdown_connections 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.628+0000 7f2e00b52740 1 -- 172.21.15.19:0/3275181236 wait complete. 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 -- start start 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 Processor -- start 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192250f5d0 0x561922519db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192251a370 0x5619225177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56192251d070 con 0x561922510b40 2024-09-12T03:29:09.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561922430060 con 0x56192250f5d0 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56192251d1f0 con 0x56192251a370 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2e00b52740 10 client.0 _ll_register_callbacks cb 0x5619221838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35808/0 (socket says 172.21.15.19:35808) 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192250f5d0 0x561922519db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2df7fff640 1 -- 172.21.15.19:0/3284774062 learned_addr learned my addr 172.21.15.19:0/3284774062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.629+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192251a370 0x5619225177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 -- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192251a370 msgr2=0x5619225177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192251a370 0x5619225177f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 -- 172.21.15.19:0/3284774062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192250f5d0 msgr2=0x561922519db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192250f5d0 0x561922519db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 -- 172.21.15.19:0/3284774062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619221e63c0 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2de8013710 tx=0x7f2de8013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1098870ad9344bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8020060 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2de8011dd0 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec0118a0 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f2de801b430 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2e00b52740 10 client.17388 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.630+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x561922528b30 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec0118a0 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f2de8026810 con 0x561922510b40 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 10 client.17388 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:09.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 20 client.17388 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56192252a440 0x56192252c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 4 client.17388 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561922482860 con 0x56192252a440 2024-09-12T03:29:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.631+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56192252a440 0x56192252c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.632+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56192252a440 0x56192252c8a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20a94d9693a515ce server_cookie=633b653893f8d9dd in_seq=0 out_seq=0 2024-09-12T03:29:09.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.632+0000 7f2dd77fe640 10 client.17388 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.633+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3284774062 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561922482860 con 0x56192252a440 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.633+0000 7f2dd77fe640 10 client.17388 handle_command_reply: tid=0 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.633+0000 7f2e00b52740 1 client.17388 shutdown 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 msgr2=0x561922517060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f2de8013710 tx=0x7f2de8013bd0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 shutdown_connections 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192251a370 0x5619225177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922510b40 0x561922517060 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x56192252a440 0x56192252c8a0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3284774062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192250f5d0 0x561922519db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 >> 172.21.15.19:0/3284774062 conn(0x5619224d8310 msgr2=0x561922511960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2dd77fe640 0 client.17388 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 shutdown_connections 2024-09-12T03:29:09.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 1 -- 172.21.15.19:0/3284774062 wait complete. 2024-09-12T03:29:09.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.634+0000 7f2e00b52740 20 client.17388 trim_cache size 0 max 16384 2024-09-12T03:29:09.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2e00b52740 1 Processor -- start 2024-09-12T03:29:09.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2e00b52740 1 -- start start 2024-09-12T03:29:09.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x561922516920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922512120 0x5619225125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.658+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50966/0 (socket says 172.21.15.19:50966) 2024-09-12T03:29:09.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x561922516920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x561922516920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:45206/0 (socket says 172.21.15.19:45206) 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2df77fe640 1 -- 172.21.15.19:0/3014681597 learned_addr learned my addr 172.21.15.19:0/3014681597 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81af6f0 con 0x561922512120 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81b1470 con 0x56192251a370 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81afdc0 con 0x56192250f5d0 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.659+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922512120 0x5619225125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 -- 172.21.15.19:0/3014681597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 msgr2=0x561922516920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x561922516920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 -- 172.21.15.19:0/3014681597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922512120 msgr2=0x5619225125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922512120 0x5619225125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 -- 172.21.15.19:0/3014681597 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192250c960 con 0x56192250f5d0 2024-09-12T03:29:09.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.660+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011be0 tx=0x7f2dec009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e3937830435e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.661+0000 7f2dddffb640 1 -- 172.21.15.19:0/3014681597 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec011800 con 0x56192250f5d0 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.661+0000 7f2dddffb640 1 -- 172.21.15.19:0/3014681597 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec034070 con 0x56192250f5d0 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.661+0000 7f2dddffb640 1 -- 172.21.15.19:0/3014681597 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec003890 con 0x56192250f5d0 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 msgr2=0x56192250be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2dec011be0 tx=0x7f2dec009f70 comp rx=0 tx=0).stop 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 shutdown_connections 2024-09-12T03:29:09.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561922512120 0x5619225125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x561922516920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3014681597 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56192250f5d0 0x56192250be10 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 >> 172.21.15.19:0/3014681597 conn(0x5619224d8310 msgr2=0x56192250b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 shutdown_connections 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- 172.21.15.19:0/3014681597 wait complete. 2024-09-12T03:29:09.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.662+0000 7f2e00b52740 1 -- start start 2024-09-12T03:29:09.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.663+0000 7f2e00b52740 1 Processor -- start 2024-09-12T03:29:09.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192250f5d0 0x561922522a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x56192251e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81b0ce0 con 0x56192250f5d0 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81b0550 con 0x56192251a370 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2df81ad610 con 0x561922512120 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2e00b52740 10 client.0 _ll_register_callbacks cb 0x7f2de8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:50976/0 (socket says 172.21.15.19:50976) 2024-09-12T03:29:09.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2df7fff640 1 -- 172.21.15.19:0/3995541711 learned_addr learned my addr 172.21.15.19:0/3995541711 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x56192251e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.664+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192250f5d0 0x561922522a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 -- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 msgr2=0x56192251ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 -- 172.21.15.19:0/3995541711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192250f5d0 msgr2=0x561922522a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192250f5d0 0x561922522a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 -- 172.21.15.19:0/3995541711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192251e960 con 0x56192251a370 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2df77fe640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x56192251e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006010 tx=0x7f2dec009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1876fc8ff5de493 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.665+0000 7f2dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dec005020 con 0x56192251a370 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2dec0051c0 con 0x56192251a370 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dec034070 con 0x56192251a370 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f2dec0053c0 con 0x56192251a370 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2e00b52740 10 client.26786 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:09.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x5619225ae810 con 0x56192251a370 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2dec034070 con 0x56192251a370 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.666+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f2dec01d020 con 0x56192251a370 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 10 client.26786 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 20 client.26786 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:09.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619225aed80 0x5619225b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 4 client.26786 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56192247cd70 con 0x5619225aed80 2024-09-12T03:29:09.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.667+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619225aed80 0x5619225b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.669+0000 7f2df7fff640 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619225aed80 0x5619225b11a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b9d2d1fe988d66a server_cookie=daafadb71cb08e5a in_seq=0 out_seq=0 2024-09-12T03:29:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.669+0000 7f2dd77fe640 10 client.26786 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.669+0000 7f2dd77fe640 1 -- 172.21.15.19:0/3995541711 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56192247cd70 con 0x5619225aed80 2024-09-12T03:29:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.669+0000 7f2dd77fe640 10 client.26786 handle_command_reply: tid=0 2024-09-12T03:29:09.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.669+0000 7f2e00b52740 1 client.26786 shutdown 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 msgr2=0x56192251e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x56192251e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006010 tx=0x7f2dec009f70 comp rx=0 tx=0).stop 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 shutdown_connections 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5619225aed80 0x5619225b11a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56192251a370 0x56192251e2f0 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2dec006010 tx=0x7f2dec009f70 comp rx=0 tx=0).stop 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561922512120 0x56192251ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 --2- 172.21.15.19:0/3995541711 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56192250f5d0 0x561922522a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 >> 172.21.15.19:0/3995541711 conn(0x5619224d8310 msgr2=0x56192250b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2dd77fe640 0 client.26786 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 shutdown_connections 2024-09-12T03:29:09.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 1 -- 172.21.15.19:0/3995541711 wait complete. 2024-09-12T03:29:09.672 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:09.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.670+0000 7f2e00b52740 20 client.26786 trim_cache size 0 max 16384 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 msgr2=0x7f2de00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 0x7f2de00a5040 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2df00b8fd0 tx=0x7f2df0087280 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 msgr2=0x7f2df819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004af0 tx=0x7f2df002f3a0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 shutdown_connections 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2de00a2b80 0x7f2de00a5040 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2df8102ef0 0x7f2df819ef80 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2df8102570 0x7f2df819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 --2- 172.21.15.19:0/4131098464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2df8100e20 0x7f2df819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.671+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 >> 172.21.15.19:0/4131098464 conn(0x7f2df80fc7d0 msgr2=0x7f2df8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:09.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.672+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 shutdown_connections 2024-09-12T03:29:09.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:09.672+0000 7f2dfe779640 1 -- 172.21.15.19:0/4131098464 wait complete. 2024-09-12T03:29:11.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:10 smithi135 ceph-mon[35474]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 433 op/s 2024-09-12T03:29:11.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:10 smithi121 ceph-mon[34963]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 433 op/s 2024-09-12T03:29:11.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:10 smithi019 ceph-mon[30953]: pgmap v641: 97 pgs: 97 active+clean; 2.2 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 433 op/s 2024-09-12T03:29:11.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:29:11.516 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:29:11.816 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 -- 172.21.15.19:0/1153545817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8101430 msgr2=0x7f09a810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 --2- 172.21.15.19:0/1153545817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8101430 0x7f09a810d960 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009cd0 tx=0x7f09a002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 -- 172.21.15.19:0/1153545817 shutdown_connections 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 --2- 172.21.15.19:0/1153545817 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8101430 0x7f09a810d960 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 --2- 172.21.15.19:0/1153545817 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 0x7f09a8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 --2- 172.21.15.19:0/1153545817 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8106a90 0x7f09a8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 -- 172.21.15.19:0/1153545817 >> 172.21.15.19:0/1153545817 conn(0x7f09a80fc7d0 msgr2=0x7f09a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 -- 172.21.15.19:0/1153545817 shutdown_connections 2024-09-12T03:29:12.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.310+0000 7f09af7b8640 1 -- 172.21.15.19:0/1153545817 wait complete. 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 Processor -- start 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 -- start start 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 0x7f09a819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 0x7f09a819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8110e90 con 0x7f09a8106a90 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8110d10 con 0x7f09a8100a90 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09af7b8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09a8111010 con 0x7f09a8101430 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09add55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 0x7f09a819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09add55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55244/0 (socket says 172.21.15.19:55244) 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09add55640 1 -- 172.21.15.19:0/2978426700 learned_addr learned my addr 172.21.15.19:0/2978426700 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09ad554640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 0x7f09a819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 -- 172.21.15.19:0/2978426700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 msgr2=0x7f09a819a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 0x7f09a819a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 -- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 msgr2=0x7f09a819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.311+0000 7f09acd53640 1 -- 172.21.15.19:0/2978426700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a819f5f0 con 0x7f09a8101430 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f09add55640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f09acd53640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 0x7f09a819ab80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9f0 tx=0x7f099800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f099e7fc640 1 -- 172.21.15.19:0/2978426700 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099800ccc0 con 0x7f09a8101430 2024-09-12T03:29:12.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a819f8e0 con 0x7f09a8101430 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f099e7fc640 1 -- 172.21.15.19:0/2978426700 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998010910 con 0x7f09a8101430 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f099e7fc640 1 -- 172.21.15.19:0/2978426700 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998018960 con 0x7f09a8101430 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.312+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a81a71c0 con 0x7f09a8101430 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.314+0000 7f099e7fc640 1 -- 172.21.15.19:0/2978426700 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0998010ac0 con 0x7f09a8101430 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 Processor -- start 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 -- start start 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbc480 0x563127cc6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cc9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563127c21280 con 0x563127cc6d40 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563127bc8b10 con 0x563127cbc480 2024-09-12T03:29:12.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09b1b91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563127c1d240 con 0x563127cbbb60 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09acd53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cc9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:54616/0 (socket says 172.21.15.19:54616) 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 -- 172.21.15.19:0/3390627846 learned_addr learned my addr 172.21.15.19:0/3390627846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f099e7fc640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 0x7f09940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09ad554640 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbc480 0x563127cc6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 -- 172.21.15.19:0/3390627846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbc480 msgr2=0x563127cc6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbc480 0x563127cc6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 -- 172.21.15.19:0/3390627846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 msgr2=0x563127cc9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.315+0000 7f09add55640 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cc9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09add55640 1 -- 172.21.15.19:0/3390627846 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563127bd51f0 con 0x563127cbbb60 2024-09-12T03:29:12.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09add55640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 0x7f09940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f099e7fc640 1 -- 172.21.15.19:0/2978426700 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f09980cf420 con 0x7f09a8101430 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09add55640 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09a0031860 tx=0x7f09a0005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d83178ffcf5dc53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09937fe640 1 -- 172.21.15.19:0/3390627846 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0041040 con 0x563127cbbb60 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09937fe640 1 -- 172.21.15.19:0/3390627846 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0043070 con 0x563127cbbb60 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09add55640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 0x7f09940a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09a004c000 tx=0x7f09a004d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.316+0000 7f09937fe640 1 -- 172.21.15.19:0/3390627846 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0047430 con 0x563127cbbb60 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- 172.21.15.19:0/3390627846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 msgr2=0x563127cbbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f09a0031860 tx=0x7f09a0005f70 comp rx=0 tx=0).stop 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- 172.21.15.19:0/3390627846 shutdown_connections 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cc9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbc480 0x563127cc6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3390627846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbbb60 0x563127cbbf40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- 172.21.15.19:0/3390627846 >> 172.21.15.19:0/3390627846 conn(0x563127cb80f0 msgr2=0x563127cb84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- 172.21.15.19:0/3390627846 shutdown_connections 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- 172.21.15.19:0/3390627846 wait complete. 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 -- start start 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.317+0000 7f09b1b91740 1 Processor -- start 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563127cc9910 con 0x563127cbc480 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563127bdbf10 con 0x563127cbbb60 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563127cc9a90 con 0x563127cc6d40 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09b1b91740 10 client.0 _ll_register_callbacks cb 0x563127ba4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:12.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09add55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09add55640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:54622/0 (socket says 172.21.15.19:54622) 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09add55640 1 -- 172.21.15.19:0/4290372146 learned_addr learned my addr 172.21.15.19:0/4290372146 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55266/0 (socket says 172.21.15.19:55266) 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09ad554640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 -- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 msgr2=0x563127cc46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 -- 172.21.15.19:0/4290372146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 msgr2=0x563127cc3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09acd53640 1 -- 172.21.15.19:0/4290372146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631279926a0 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09add55640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.318+0000 7f09ad554640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09acd53640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f099809b9e0 tx=0x7f0998010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d30562b6c4a163 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099809ffe0 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a0043070 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f099809b130 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f09980a48a0 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 10 client.17406 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x563127cd5570 con 0x563127cbc480 2024-09-12T03:29:12.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0043070 con 0x563127cbc480 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f0998097a30 con 0x563127cbc480 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 10 client.17406 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 20 client.17406 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127cd6980 0x563127cd8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 4 client.17406 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.319+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563127c2e7f0 con 0x563127cd6980 2024-09-12T03:29:12.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.320+0000 7f09ad554640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127cd6980 0x563127cd8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.320+0000 7f09ad554640 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127cd6980 0x563127cd8e00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=601fca058d92e0ef server_cookie=6ea1fb8cde8d977c in_seq=0 out_seq=0 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.320+0000 7f0990ff9640 10 client.17406 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f0990ff9640 1 -- 172.21.15.19:0/4290372146 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563127c2e7f0 con 0x563127cd6980 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f0990ff9640 10 client.17406 handle_command_reply: tid=0 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 client.17406 shutdown 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 msgr2=0x563127cc4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f099809b9e0 tx=0x7f0998010ef0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 shutdown_connections 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cc6d40 0x563127cc46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbc480 0x563127cc4170 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127cd6980 0x563127cd8e00 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 --2- 172.21.15.19:0/4290372146 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f0990ff9640 0 client.17406 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 >> 172.21.15.19:0/4290372146 conn(0x563127cb80f0 msgr2=0x563127cb7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 shutdown_connections 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 1 -- 172.21.15.19:0/4290372146 wait complete. 2024-09-12T03:29:12.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.321+0000 7f09b1b91740 20 client.17406 trim_cache size 0 max 16384 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 Processor -- start 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 -- start start 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cde710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cdec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81af840 con 0x563127cc6d40 2024-09-12T03:29:12.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b15c0 con 0x563127cbbb60 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09b1b91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81aff10 con 0x563127cbc480 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37114/0 (socket says 172.21.15.19:37114) 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09add55640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cdec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 -- 172.21.15.19:0/2072685256 learned_addr learned my addr 172.21.15.19:0/2072685256 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:12.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09acd53640 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cde710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 -- 172.21.15.19:0/2072685256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 msgr2=0x563127cde710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cde710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 -- 172.21.15.19:0/2072685256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 msgr2=0x563127cdec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cdec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09ad554640 1 -- 172.21.15.19:0/2072685256 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563127cbc2b0 con 0x563127cbbb60 2024-09-12T03:29:12.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.340+0000 7f09add55640 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cdec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09ad554640 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x563127cd96b0 tx=0x7f09a4005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c1cc03ad6ae208b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f0992ffd640 1 -- 172.21.15.19:0/2072685256 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4004030 con 0x563127cbbb60 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f0992ffd640 1 -- 172.21.15.19:0/2072685256 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a400fcd0 con 0x563127cbbb60 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f0992ffd640 1 -- 172.21.15.19:0/2072685256 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a40039e0 con 0x563127cbbb60 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 -- 172.21.15.19:0/2072685256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 msgr2=0x563127cc5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x563127cd96b0 tx=0x7f09a4005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 -- 172.21.15.19:0/2072685256 shutdown_connections 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cc6d40 0x563127cdec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cde710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 --2- 172.21.15.19:0/2072685256 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cbbb60 0x563127cc5a90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.341+0000 7f09b1b91740 1 -- 172.21.15.19:0/2072685256 >> 172.21.15.19:0/2072685256 conn(0x563127cb80f0 msgr2=0x563127cc5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 -- 172.21.15.19:0/2072685256 shutdown_connections 2024-09-12T03:29:12.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 -- 172.21.15.19:0/2072685256 wait complete. 2024-09-12T03:29:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 -- start start 2024-09-12T03:29:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 Processor -- start 2024-09-12T03:29:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 0x563127cc18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.342+0000 7f09b1b91740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b0e30 con 0x563127cbbb60 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09b1b91740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81b06a0 con 0x563127cc6d40 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09b1b91740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f09a81ad760 con 0x563127cbc480 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09b1b91740 10 client.0 _ll_register_callbacks cb 0x7f09980d2b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09acd53640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09add55640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 0x563127cc18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09acd53640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:54640/0 (socket says 172.21.15.19:54640) 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55286/0 (socket says 172.21.15.19:55286) 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09add55640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 0x563127cc18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37116/0 (socket says 172.21.15.19:37116) 2024-09-12T03:29:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09acd53640 1 -- 172.21.15.19:0/3010683954 learned_addr learned my addr 172.21.15.19:0/3010683954 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 -- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 msgr2=0x563127cc1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 -- 172.21.15.19:0/3010683954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 msgr2=0x563127cc18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 0x563127cc18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 -- 172.21.15.19:0/3010683954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563127d408e0 con 0x563127cbbb60 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09acd53640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09ad554640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f09a4014010 tx=0x7f09a4005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c97f14fa469271d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f0990ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a4038060 con 0x563127cbbb60 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f09a400f7f0 con 0x563127cbbb60 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.343+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a400fcd0 con 0x563127cbbb60 2024-09-12T03:29:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f09a4009e50 con 0x563127cbbb60 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 10 client.17412 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x563127d41bc0 con 0x563127cbbb60 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a400fcd0 con 0x563127cbbb60 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f09a4029400 con 0x563127cbbb60 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 10 client.17412 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 20 client.17412 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127d42310 0x563127d44730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 4 client.17412 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563127c28d00 con 0x563127d42310 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.344+0000 7f09acd53640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127d42310 0x563127d44730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.345+0000 7f09acd53640 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127d42310 0x563127d44730 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfd4baa6ef04cef2 server_cookie=d37f6443d3253117 in_seq=0 out_seq=0 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.345+0000 7f0990ff9640 10 client.17412 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f0990ff9640 1 -- 172.21.15.19:0/3010683954 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563127c28d00 con 0x563127d42310 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f0990ff9640 10 client.17412 handle_command_reply: tid=0 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 client.17412 shutdown 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 msgr2=0x563127cc0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f09a4014010 tx=0x7f09a4005760 comp rx=0 tx=0).stop 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 shutdown_connections 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563127d42310 0x563127d44730 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563127cc6d40 0x563127cc18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563127cbc480 0x563127cc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f0990ff9640 0 client.17412 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 --2- 172.21.15.19:0/3010683954 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563127cbbb60 0x563127cc0e40 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 >> 172.21.15.19:0/3010683954 conn(0x563127cb80f0 msgr2=0x563127cb7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 shutdown_connections 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 1 -- 172.21.15.19:0/3010683954 wait complete. 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.346+0000 7f09b1b91740 20 client.17412 trim_cache size 0 max 16384 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:29:12.353 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "ba1ab028-7bd3-4776-9788-9b914e4ba5ec", 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 msgr2=0x7f09940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 0x7f09940a5040 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f09a004c000 tx=0x7f09a004d000 comp rx=0 tx=0).stop 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 msgr2=0x7f09a819ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 0x7f09a819ab80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f099800e9f0 tx=0x7f099800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 shutdown_connections 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f09940a2b80 0x7f09940a5040 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f09a8106a90 0x7f09a819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f09a8101430 0x7f09a819ab80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 --2- 172.21.15.19:0/2978426700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a8100a90 0x7f09a819a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 >> 172.21.15.19:0/2978426700 conn(0x7f09a80fc7d0 msgr2=0x7f09a806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 shutdown_connections 2024-09-12T03:29:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:12.347+0000 7f09af7b8640 1 -- 172.21.15.19:0/2978426700 wait complete. 2024-09-12T03:29:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:12 smithi135 ceph-mon[35474]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-12T03:29:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:12 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:29:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:12 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:12 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:29:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:12 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:29:13.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:12 smithi019 ceph-mon[30953]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-12T03:29:13.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:12 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:29:13.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:12 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:13.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:12 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:29:13.116 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:12 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:29:13.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:12 smithi121 ceph-mon[34963]: pgmap v642: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 495 op/s 2024-09-12T03:29:13.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:12 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:29:13.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:12 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:13.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:12 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:29:13.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:12 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:29:13.162 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:29:13.524 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:13.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:13 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:29:13.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:13 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:29:14.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:13 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:29:14.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:13 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:29:14.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:13 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:29:14.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:13 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:29:14.488 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 -- 172.21.15.19:0/1157969372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 msgr2=0x7f54e81043a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 --2- 172.21.15.19:0/1157969372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e81043a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54d8009900 tx=0x7f54d802a410 comp rx=0 tx=0).stop 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 -- 172.21.15.19:0/1157969372 shutdown_connections 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 --2- 172.21.15.19:0/1157969372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e81090f0 0x7f54e8110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 --2- 172.21.15.19:0/1157969372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e8104970 0x7f54e81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 --2- 172.21.15.19:0/1157969372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e81043a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 -- 172.21.15.19:0/1157969372 >> 172.21.15.19:0/1157969372 conn(0x7f54e80ffa40 msgr2=0x7f54e8101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 -- 172.21.15.19:0/1157969372 shutdown_connections 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.104+0000 7f54ed820640 1 -- 172.21.15.19:0/1157969372 wait complete. 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.105+0000 7f54ed820640 1 Processor -- start 2024-09-12T03:29:14.489 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.105+0000 7f54ed820640 1 -- start start 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.105+0000 7f54ed820640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.105+0000 7f54ed820640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.105+0000 7f54ed820640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e81090f0 0x7f54e819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55304/0 (socket says 172.21.15.19:55304) 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54ed820640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8114140 con 0x7f54e8104970 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54ed820640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54e8113fc0 con 0x7f54e8103fc0 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54ed820640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81142c0 con 0x7f54e81090f0 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e6ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 -- 172.21.15.19:0/333341706 learned_addr learned my addr 172.21.15.19:0/333341706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e77fe640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e81090f0 0x7f54e819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.490 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 -- 172.21.15.19:0/333341706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e81090f0 msgr2=0x7f54e819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e81090f0 0x7f54e819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 -- 172.21.15.19:0/333341706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 msgr2=0x7f54e819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 -- 172.21.15.19:0/333341706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e819f5d0 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54e67fc640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9f0 tx=0x7f54dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.106+0000 7f54ec81e640 1 -- 172.21.15.19:0/333341706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc00cd80 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.107+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e819f8c0 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.107+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81a71a0 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.107+0000 7f54ec81e640 1 -- 172.21.15.19:0/333341706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54dc010910 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.107+0000 7f54ec81e640 1 -- 172.21.15.19:0/333341706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0188f0 con 0x7f54e8104970 2024-09-12T03:29:14.491 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.110+0000 7f54efbf9740 1 Processor -- start 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.110+0000 7f54ec81e640 1 -- 172.21.15.19:0/333341706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54dc010ac0 con 0x7f54e8104970 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.110+0000 7f54efbf9740 1 -- start start 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54ec81e640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 0x7f54d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37142/0 (socket says 172.21.15.19:37142) 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54e77fe640 1 -- 172.21.15.19:0/2127596509 learned_addr learned my addr 172.21.15.19:0/2127596509 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eebd6312e0 con 0x55eebd6d6680 2024-09-12T03:29:14.492 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55eebd5d8ce0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.111+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eebd62d2a0 con 0x55eebd6cbdc0 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54ec81e640 1 -- 172.21.15.19:0/333341706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f54dc0cf480 con 0x7f54e8104970 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e77fe640 1 -- 172.21.15.19:0/2127596509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 msgr2=0x55eebd6d6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e77fe640 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e77fe640 1 -- 172.21.15.19:0/2127596509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 msgr2=0x55eebd6d8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e77fe640 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e77fe640 1 -- 172.21.15.19:0/2127596509 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebd5bd440 con 0x55eebd6cb4a0 2024-09-12T03:29:14.493 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.112+0000 7f54e67fc640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 0x7f54d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.113+0000 7f54e77fe640 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54e0012440 tx=0x7f54e0012900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510c77114e89b94b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.113+0000 7f54cdffb640 1 -- 172.21.15.19:0/2127596509 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e000f480 con 0x55eebd6cb4a0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.113+0000 7f54cdffb640 1 -- 172.21.15.19:0/2127596509 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e0016070 con 0x55eebd6cb4a0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.113+0000 7f54cdffb640 1 -- 172.21.15.19:0/2127596509 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0010a60 con 0x55eebd6cb4a0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 msgr2=0x55eebd6cb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f54e0012440 tx=0x7f54e0012900 comp rx=0 tx=0).stop 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54e67fc640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 0x7f54d00a50c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00cfd0 tx=0x7f54dc00cf20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.495 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 shutdown_connections 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 --2- 172.21.15.19:0/2127596509 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6cb880 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.114+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 >> 172.21.15.19:0/2127596509 conn(0x55eebd6c7a30 msgr2=0x55eebd6c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.115+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 shutdown_connections 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.115+0000 7f54efbf9740 1 -- 172.21.15.19:0/2127596509 wait complete. 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.115+0000 7f54efbf9740 1 -- start start 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.115+0000 7f54efbf9740 1 Processor -- start 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55eebd6d9250 con 0x55eebd6d6680 2024-09-12T03:29:14.496 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37146/0 (socket says 172.21.15.19:37146) 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54e77fe640 1 -- 172.21.15.19:0/322854793 learned_addr learned my addr 172.21.15.19:0/322854793 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55eebd5ec050 con 0x55eebd6cb4a0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55eebd6d93d0 con 0x55eebd6cbdc0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.116+0000 7f54e77fe640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 -- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 msgr2=0x55eebd6d33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 -- 172.21.15.19:0/322854793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 msgr2=0x55eebd6d3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebd3a23c0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54efbf9740 10 client.0 _ll_register_callbacks cb 0x55eebd62c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e0016070 con 0x55eebd6cb4a0 2024-09-12T03:29:14.497 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54e77fe640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54e0010010 tx=0x7f54e0010600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390927d3ac07bd16 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54c6ffd640 10 client.26816 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.117+0000 7f54c6ffd640 1 -- 172.21.15.19:0/322854793 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0026ad0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.118+0000 7f54c6ffd640 1 -- 172.21.15.19:0/322854793 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f54e0026c70 con 0x55eebd6cb4a0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.118+0000 7f54c6ffd640 1 -- 172.21.15.19:0/322854793 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f54e00280d0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.118+0000 7f54efbf9740 10 client.26816 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.118+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({fsmap=56}) -- 0x55eebd6e4b70 con 0x55eebd6cb4a0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54c6ffd640 1 -- 172.21.15.19:0/322854793 <== mon.2 v2:172.21.15.121:3300/0 4 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f54e0028270 con 0x55eebd6cb4a0 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 10 client.26816 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 10 client.26816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 20 client.26816 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd6e5ec0 0x55eebd6e82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 4 client.26816 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:14.498 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.119+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eebd63e850 con 0x55eebd6e5ec0 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.120+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd6e5ec0 0x55eebd6e82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.121+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd6e5ec0 0x55eebd6e82b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b22f44afb1ec53b server_cookie=84b70f6889cf4162 in_seq=0 out_seq=0 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.121+0000 7f54c6ffd640 10 client.26816 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54c6ffd640 1 -- 172.21.15.19:0/322854793 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eebd63e850 con 0x55eebd6e5ec0 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54c6ffd640 10 client.26816 handle_command_reply: tid=0 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 client.26816 shutdown 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 msgr2=0x55eebd6d60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f54e0010010 tx=0x7f54e0010600 comp rx=0 tx=0).stop 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 shutdown_connections 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6d6680 0x55eebd6d3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd6e5ec0 0x55eebd6e82b0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.499 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 --2- 172.21.15.19:0/322854793 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d60b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 >> 172.21.15.19:0/322854793 conn(0x55eebd6c7a30 msgr2=0x55eebd6c6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54c6ffd640 0 client.26816 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.122+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 shutdown_connections 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.123+0000 7f54efbf9740 1 -- 172.21.15.19:0/322854793 wait complete. 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.123+0000 7f54efbf9740 20 client.26816 trim_cache size 0 max 16384 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 Processor -- start 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 -- start start 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6d6680 0x55eebd6ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81ab480 con 0x55eebd6cb4a0 2024-09-12T03:29:14.500 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81ad200 con 0x55eebd6cbdc0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54efbf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81abb50 con 0x55eebd6d6680 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.150+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:55324/0 (socket says 172.21.15.19:55324) 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 -- 172.21.15.19:0/282115877 learned_addr learned my addr 172.21.15.19:0/282115877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6d6680 0x55eebd6ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 -- 172.21.15.19:0/282115877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6d6680 msgr2=0x55eebd6ed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6d6680 0x55eebd6ed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 -- 172.21.15.19:0/282115877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cbdc0 msgr2=0x55eebd6d30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.151+0000 7f54e77fe640 1 -- 172.21.15.19:0/282115877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eebd6d9130 con 0x55eebd6cb4a0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.152+0000 7f54e77fe640 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009af0 tx=0x7f54e001d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1443b1487f49aee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.501 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.152+0000 7f54cd7fa640 1 -- 172.21.15.19:0/282115877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0026ad0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.152+0000 7f54cd7fa640 1 -- 172.21.15.19:0/282115877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e0016070 con 0x55eebd6cb4a0 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.152+0000 7f54cd7fa640 1 -- 172.21.15.19:0/282115877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0010670 con 0x55eebd6cb4a0 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 -- 172.21.15.19:0/282115877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 msgr2=0x55eebd6d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009af0 tx=0x7f54e001d070 comp rx=0 tx=0).stop 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 -- 172.21.15.19:0/282115877 shutdown_connections 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6d6680 0x55eebd6ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6cbdc0 0x55eebd6d30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 --2- 172.21.15.19:0/282115877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cb4a0 0x55eebd6d2b60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 -- 172.21.15.19:0/282115877 >> 172.21.15.19:0/282115877 conn(0x55eebd6c7a30 msgr2=0x55eebd6cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 -- 172.21.15.19:0/282115877 shutdown_connections 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.153+0000 7f54efbf9740 1 -- 172.21.15.19:0/282115877 wait complete. 2024-09-12T03:29:14.502 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.154+0000 7f54efbf9740 1 -- start start 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.154+0000 7f54efbf9740 1 Processor -- start 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.154+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cb4a0 0x55eebd6da560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.154+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cbdc0 0x55eebd6daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54efbf9740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54efbf9740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81aca70 con 0x55eebd6cbdc0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54efbf9740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81ac2e0 con 0x55eebd6d6680 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54efbf9740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f54e81a93a0 con 0x55eebd6cb4a0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54efbf9740 10 client.0 _ll_register_callbacks cb 0x7f54dc005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e6ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cbdc0 0x55eebd6daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:37154/0 (socket says 172.21.15.19:37154) 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 -- 172.21.15.19:0/1199251203 learned_addr learned my addr 172.21.15.19:0/1199251203 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cb4a0 0x55eebd6da560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.503 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 -- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cb4a0 msgr2=0x55eebd6da560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cb4a0 0x55eebd6da560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 -- 172.21.15.19:0/1199251203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cbdc0 msgr2=0x55eebd6daaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cbdc0 0x55eebd6daaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 -- 172.21.15.19:0/1199251203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eebd6c86c0 con 0x55eebd6d6680 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.155+0000 7f54e77fe640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e70 tx=0x7f54e0003ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d752c82f9a619996 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0026ce0 con 0x55eebd6d6680 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f54e00049e0 con 0x55eebd6d6680 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e0016070 con 0x55eebd6d6680 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f54e0004be0 con 0x55eebd6d6680 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54efbf9740 10 client.26822 fetch_fsmap learned FSMap version 56 2024-09-12T03:29:14.504 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.156+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) -- 0x55eebd6c86c0 con 0x55eebd6d6680 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54e0016070 con 0x55eebd6d6680 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 56) ==== 8442+0+0 (secure 0 0 0) 0x7f54e004d450 con 0x55eebd6d6680 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 10 client.26822 fetch_fsmap finished waiting for FSMap version 56 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 10 client.26822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 20 client.26822 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd750fa0 0x55eebd753390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 4 client.26822 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.157+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55eebd638d60 con 0x55eebd750fa0 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.158+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd750fa0 0x55eebd753390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.158+0000 7f54e6ffd640 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd750fa0 0x55eebd753390 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61313054c3f4e1dd server_cookie=a8045f07cde70833 in_seq=0 out_seq=0 2024-09-12T03:29:14.505 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.158+0000 7f54c6ffd640 10 client.26822 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.159+0000 7f54c6ffd640 1 -- 172.21.15.19:0/1199251203 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55eebd638d60 con 0x55eebd750fa0 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.159+0000 7f54c6ffd640 10 client.26822 handle_command_reply: tid=0 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.159+0000 7f54efbf9740 1 client.26822 shutdown 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 msgr2=0x55eebd6dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009e70 tx=0x7f54e0003ab0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 shutdown_connections 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55eebd750fa0 0x55eebd753390 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55eebd6d6680 0x55eebd6dafe0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55eebd6cbdc0 0x55eebd6daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 --2- 172.21.15.19:0/1199251203 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55eebd6cb4a0 0x55eebd6da560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.506 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 >> 172.21.15.19:0/1199251203 conn(0x55eebd6c7a30 msgr2=0x55eebd6c6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (7574 inodes in the stack)", 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "ba1ab028-7bd3-4776-9788-9b914e4ba5ec": { 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "ba1ab028-7bd3-4776-9788-9b914e4ba5ec", 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:29:14.507 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54c6ffd640 0 client.26822 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 shutdown_connections 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.160+0000 7f54efbf9740 1 -- 172.21.15.19:0/1199251203 wait complete. 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.161+0000 7f54efbf9740 20 client.26822 trim_cache size 0 max 16384 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.161+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 msgr2=0x7f54d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.161+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 0x7f54d00a50c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00cfd0 tx=0x7f54dc00cf20 comp rx=0 tx=0).stop 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.161+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 msgr2=0x7f54e819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.161+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00e9f0 tx=0x7f54dc00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 shutdown_connections 2024-09-12T03:29:14.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f54e81090f0 0x7f54e819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f54d00a2c00 0x7f54d00a50c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f54e8104970 0x7f54e819ab60 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 --2- 172.21.15.19:0/333341706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e8103fc0 0x7f54e819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 >> 172.21.15.19:0/333341706 conn(0x7f54e80ffa40 msgr2=0x7f54e8100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 shutdown_connections 2024-09-12T03:29:14.509 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:14.162+0000 7f54ed820640 1 -- 172.21.15.19:0/333341706 wait complete. 2024-09-12T03:29:15.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:14 smithi135 ceph-mon[35474]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-09-12T03:29:15.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:14 smithi121 ceph-mon[34963]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-09-12T03:29:15.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:14 smithi019 ceph-mon[30953]: pgmap v643: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-09-12T03:29:15.776 INFO:tasks.cephfs.filesystem:scrub status for tag:ba1ab028-7bd3-4776-9788-9b914e4ba5ec - {'path': '/', 'tag': 'ba1ab028-7bd3-4776-9788-9b914e4ba5ec', 'options': 'recursive,force'} 2024-09-12T03:29:16.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:15 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:16.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:15 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:16.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:15 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:16 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:16 smithi135 ceph-mon[35474]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-09-12T03:29:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:16 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:16 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:16 smithi121 ceph-mon[34963]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-09-12T03:29:17.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:16 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:16 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:17.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:16 smithi019 ceph-mon[30953]: pgmap v644: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 460 op/s 2024-09-12T03:29:17.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:16 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:18.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:17 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:18.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:17 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:18.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:17 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:29:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:18 smithi135 ceph-mon[35474]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 469 op/s 2024-09-12T03:29:19.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:18 smithi121 ceph-mon[34963]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 469 op/s 2024-09-12T03:29:19.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:18 smithi019 ceph-mon[30953]: pgmap v645: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 469 op/s 2024-09-12T03:29:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:20 smithi135 ceph-mon[35474]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2024-09-12T03:29:21.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:20 smithi121 ceph-mon[34963]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2024-09-12T03:29:21.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:20 smithi019 ceph-mon[30953]: pgmap v646: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 407 op/s 2024-09-12T03:29:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:22 smithi135 ceph-mon[35474]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 455 op/s 2024-09-12T03:29:23.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:22 smithi121 ceph-mon[34963]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 455 op/s 2024-09-12T03:29:23.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:22 smithi019 ceph-mon[30953]: pgmap v647: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 455 op/s 2024-09-12T03:29:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:24 smithi135 ceph-mon[35474]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 430 op/s 2024-09-12T03:29:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:29:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:29:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:29:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:29:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:24 smithi121 ceph-mon[34963]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 430 op/s 2024-09-12T03:29:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:29:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:29:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:29:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:29:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:24 smithi019 ceph-mon[30953]: pgmap v648: 97 pgs: 97 active+clean; 2.1 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 430 op/s 2024-09-12T03:29:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:29:25.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:29:25.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:29:25.165 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:29:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:26 smithi135 ceph-mon[35474]: pgmap v649: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 431 op/s 2024-09-12T03:29:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:26 smithi121 ceph-mon[34963]: pgmap v649: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 431 op/s 2024-09-12T03:29:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:26 smithi019 ceph-mon[30953]: pgmap v649: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 431 op/s 2024-09-12T03:29:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:28 smithi135 ceph-mon[35474]: pgmap v650: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-09-12T03:29:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:28 smithi121 ceph-mon[34963]: pgmap v650: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-09-12T03:29:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:28 smithi019 ceph-mon[30953]: pgmap v650: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 467 op/s 2024-09-12T03:29:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:30 smithi135 ceph-mon[35474]: pgmap v651: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 425 op/s 2024-09-12T03:29:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:30 smithi121 ceph-mon[34963]: pgmap v651: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 425 op/s 2024-09-12T03:29:31.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:30 smithi019 ceph-mon[30953]: pgmap v651: 97 pgs: 97 active+clean; 2.0 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 425 op/s 2024-09-12T03:29:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:32 smithi135 ceph-mon[35474]: pgmap v652: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-09-12T03:29:33.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:32 smithi121 ceph-mon[34963]: pgmap v652: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-09-12T03:29:33.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:32 smithi019 ceph-mon[30953]: pgmap v652: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 468 op/s 2024-09-12T03:29:35.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:35 smithi135 ceph-mon[35474]: pgmap v653: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 444 op/s 2024-09-12T03:29:35.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:35 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:35 smithi121 ceph-mon[34963]: pgmap v653: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 444 op/s 2024-09-12T03:29:35.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:35 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:35 smithi019 ceph-mon[30953]: pgmap v653: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 444 op/s 2024-09-12T03:29:35.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:35 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:37 smithi135 ceph-mon[35474]: pgmap v654: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-09-12T03:29:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:37 smithi121 ceph-mon[34963]: pgmap v654: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-09-12T03:29:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:37 smithi019 ceph-mon[30953]: pgmap v654: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 425 op/s 2024-09-12T03:29:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:39 smithi135 ceph-mon[35474]: pgmap v655: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-09-12T03:29:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:39 smithi121 ceph-mon[34963]: pgmap v655: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-09-12T03:29:39.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:39 smithi019 ceph-mon[30953]: pgmap v655: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-09-12T03:29:40.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:40 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:40.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:40 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:40.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:40 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:29:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:41 smithi135 ceph-mon[35474]: pgmap v656: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 393 op/s 2024-09-12T03:29:41.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:41 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:41.338 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:41 smithi121 ceph-mon[34963]: pgmap v656: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 393 op/s 2024-09-12T03:29:41.338 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:41 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:41 smithi019 ceph-mon[30953]: pgmap v656: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 393 op/s 2024-09-12T03:29:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:41 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: pgmap v657: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:43.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:43.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: pgmap v657: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-09-12T03:29:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:43.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: pgmap v657: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 429 op/s 2024-09-12T03:29:43.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:29:43.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:29:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:45 smithi121 ceph-mon[34963]: pgmap v658: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-12T03:29:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:45 smithi019 ceph-mon[30953]: pgmap v658: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-12T03:29:45.777 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:29:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:45 smithi135 ceph-mon[35474]: pgmap v658: 97 pgs: 97 active+clean; 2.0 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 407 op/s 2024-09-12T03:29:46.043 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:47.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 -- 172.21.15.19:0/594977850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 msgr2=0x7f7fd4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/594977850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd4074a00 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009900 tx=0x7f7fc402a480 comp rx=0 tx=0).stop 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 -- 172.21.15.19:0/594977850 shutdown_connections 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/594977850 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd410e3d0 0x7f7fd4110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/594977850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd4074f40 0x7f7fd410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/594977850 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd4074a00 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 -- 172.21.15.19:0/594977850 >> 172.21.15.19:0/594977850 conn(0x7f7fd40fd8f0 msgr2=0x7f7fd40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 -- 172.21.15.19:0/594977850 shutdown_connections 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.287+0000 7f7fd9c58640 1 -- 172.21.15.19:0/594977850 wait complete. 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 Processor -- start 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 -- start start 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd4074f40 0x7f7fd419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd410e3d0 0x7f7fd419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41141b0 con 0x7f7fd4074620 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4114030 con 0x7f7fd410e3d0 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd9c58640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4114330 con 0x7f7fd4074f40 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44108/0 (socket says 172.21.15.19:44108) 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 -- 172.21.15.19:0/3300569643 learned_addr learned my addr 172.21.15.19:0/3300569643 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd4074f40 0x7f7fd419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd410e3d0 0x7f7fd419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 -- 172.21.15.19:0/3300569643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd4074f40 msgr2=0x7f7fd419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd4074f40 0x7f7fd419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 -- 172.21.15.19:0/3300569643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd410e3d0 msgr2=0x7f7fd419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd410e3d0 0x7f7fd419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.288+0000 7f7fd37fe640 1 -- 172.21.15.19:0/3300569643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd419e6c0 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004770 tx=0x7f7fc4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd0ff9640 1 -- 172.21.15.19:0/3300569643 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc403f070 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd0ff9640 1 -- 172.21.15.19:0/3300569643 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc4031ca0 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd419e950 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd0ff9640 1 -- 172.21.15.19:0/3300569643 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4035030 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.289+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd41a7320 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.292+0000 7f7fd0ff9640 1 -- 172.21.15.19:0/3300569643 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fc4042070 con 0x7f7fd4074620 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.292+0000 7f7fdc031740 1 Processor -- start 2024-09-12T03:29:47.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.292+0000 7f7fdc031740 1 -- start start 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e67a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5650b5dc02e0 con 0x5650b5e5adc0 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5650b5d67ce0 con 0x5650b5e65680 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fdc031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650b5dbc2a0 con 0x5650b5e5a4a0 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd0ff9640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 0x7f7fbc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44122/0 (socket says 172.21.15.19:44122) 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 -- 172.21.15.19:0/315539177 learned_addr learned my addr 172.21.15.19:0/315539177 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:47.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 0x7f7fbc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e67a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd0ff9640 1 -- 172.21.15.19:0/3300569643 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fc40ef7d0 con 0x7f7fd4074620 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 -- 172.21.15.19:0/315539177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 msgr2=0x5650b5e5a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e5a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 -- 172.21.15.19:0/315539177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 msgr2=0x5650b5e67a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e67a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 -- 172.21.15.19:0/315539177 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b5d4c440 con 0x5650b5e5adc0 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e5a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e67a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:47.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40bc020 tx=0x7f7fc4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee39de72f992ecbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.293+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 0x7f7fbc0a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009fd0 tx=0x7f7fcc0096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fb9ffb640 1 -- 172.21.15.19:0/315539177 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4049070 con 0x5650b5e5adc0 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fb9ffb640 1 -- 172.21.15.19:0/315539177 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fc40dc0c0 con 0x5650b5e5adc0 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fb9ffb640 1 -- 172.21.15.19:0/315539177 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40c3d80 con 0x5650b5e5adc0 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 -- 172.21.15.19:0/315539177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 msgr2=0x5650b5e65140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40bc020 tx=0x7f7fc4047000 comp rx=0 tx=0).stop 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 -- 172.21.15.19:0/315539177 shutdown_connections 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e67a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e65140 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 --2- 172.21.15.19:0/315539177 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e5a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 -- 172.21.15.19:0/315539177 >> 172.21.15.19:0/315539177 conn(0x5650b5e56a30 msgr2=0x5650b5e56e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 -- 172.21.15.19:0/315539177 shutdown_connections 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.294+0000 7f7fdc031740 1 -- 172.21.15.19:0/315539177 wait complete. 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 -- start start 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 Processor -- start 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e64f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5adc0 0x5650b5e621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5650b5e68010 con 0x5650b5e65680 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5650b5d7b050 con 0x5650b5e5adc0 2024-09-12T03:29:47.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5650b5e68190 con 0x5650b5e5a4a0 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fdc031740 10 client.0 _ll_register_callbacks cb 0x5650b5dbb160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44134/0 (socket says 172.21.15.19:44134) 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fd3fff640 1 -- 172.21.15.19:0/1458939048 learned_addr learned my addr 172.21.15.19:0/1458939048 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.295+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e64f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5adc0 0x5650b5e621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 -- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 msgr2=0x5650b5e64f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e64f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 -- 172.21.15.19:0/1458939048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5adc0 msgr2=0x5650b5e621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5adc0 0x5650b5e621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 -- 172.21.15.19:0/1458939048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b5b313c0 con 0x5650b5e65680 2024-09-12T03:29:47.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e64f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00fcc0 tx=0x7f7fcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22d46b02c4542d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc02e060 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fcc00fdb0 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5650b5dcd850 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7fcc01fc80 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fdc031740 10 client.17439 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.296+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5650b5e73900 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5650b5dcd850 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f7fcc0103d0 con 0x5650b5e65680 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 10 client.17439 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:47.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 10 client.17439 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 20 client.17439 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5e74e10 0x5650b5e772b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 4 client.17439 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650b5dc7d60 con 0x5650b5e74e10 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5e74e10 0x5650b5e772b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fb2ffd640 10 client.17439 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.297+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5e74e10 0x5650b5e772b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fda4d5ac2b00dfad server_cookie=dbbd2b24d1c5d995 in_seq=0 out_seq=0 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1458939048 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650b5dc7d60 con 0x5650b5e74e10 2024-09-12T03:29:47.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fb2ffd640 10 client.17439 handle_command_reply: tid=0 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 client.17439 shutdown 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 msgr2=0x5650b5e62720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00fcc0 tx=0x7f7fcc004ab0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 shutdown_connections 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e65680 0x5650b5e62720 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5e74e10 0x5650b5e772b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5adc0 0x5650b5e621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1458939048 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5a4a0 0x5650b5e64f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 >> 172.21.15.19:0/1458939048 conn(0x5650b5e56a30 msgr2=0x5650b5e5c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fb2ffd640 0 client.17439 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 shutdown_connections 2024-09-12T03:29:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 1 -- 172.21.15.19:0/1458939048 wait complete. 2024-09-12T03:29:47.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.299+0000 7f7fdc031740 20 client.17439 trim_cache size 0 max 16384 2024-09-12T03:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.317+0000 7f7fdc031740 1 Processor -- start 2024-09-12T03:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.317+0000 7f7fdc031740 1 -- start start 2024-09-12T03:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 0x5650b5e7cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e7d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41af860 con 0x5650b5e5a4a0 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41b15e0 con 0x5650b5e65680 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fdc031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41aff30 con 0x5650b5e5adc0 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 0x5650b5e7cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e7d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44150/0 (socket says 172.21.15.19:44150) 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 0x5650b5e7cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:54782/0 (socket says 172.21.15.19:54782) 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd37fe640 1 -- 172.21.15.19:0/2640816618 learned_addr learned my addr 172.21.15.19:0/2640816618 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 -- 172.21.15.19:0/2640816618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 msgr2=0x5650b5e7cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 0x5650b5e7cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 -- 172.21.15.19:0/2640816618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 msgr2=0x5650b5e711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 -- 172.21.15.19:0/2640816618 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650b5e5abf0 con 0x5650b5e65680 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.318+0000 7f7fd3fff640 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e7d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010870 tx=0x7f7fcc009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40cff6c02762dc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fb97fa640 1 -- 172.21.15.19:0/2640816618 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc038070 con 0x5650b5e65680 2024-09-12T03:29:47.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fb97fa640 1 -- 172.21.15.19:0/2640816618 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc00fdb0 con 0x5650b5e65680 2024-09-12T03:29:47.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fb97fa640 1 -- 172.21.15.19:0/2640816618 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc03d850 con 0x5650b5e65680 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fdc031740 1 -- 172.21.15.19:0/2640816618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 msgr2=0x5650b5e7d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fdc031740 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e7d0c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc010870 tx=0x7f7fcc009730 comp rx=0 tx=0).stop 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.321+0000 7f7fdc031740 1 -- 172.21.15.19:0/2640816618 shutdown_connections 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e65680 0x5650b5e7d0c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e5adc0 0x5650b5e7cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- 172.21.15.19:0/2640816618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5a4a0 0x5650b5e711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- 172.21.15.19:0/2640816618 >> 172.21.15.19:0/2640816618 conn(0x5650b5e56a30 msgr2=0x5650b5e5d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- 172.21.15.19:0/2640816618 shutdown_connections 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- 172.21.15.19:0/2640816618 wait complete. 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- start start 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 Processor -- start 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 0x5650b5e68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 0x5650b5eddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41b0e50 con 0x5650b5e5adc0 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41b06c0 con 0x5650b5e5a4a0 2024-09-12T03:29:47.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd41ad780 con 0x5650b5e65680 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.322+0000 7f7fdc031740 10 client.0 _ll_register_callbacks cb 0x7f7fcc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 0x5650b5eddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44158/0 (socket says 172.21.15.19:44158) 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 0x5650b5eddb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:54786/0 (socket says 172.21.15.19:54786) 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 -- 172.21.15.19:0/1983015346 learned_addr learned my addr 172.21.15.19:0/1983015346 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 0x5650b5e68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 -- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 msgr2=0x5650b5eddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 0x5650b5eddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 -- 172.21.15.19:0/1983015346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 msgr2=0x5650b5e68610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 0x5650b5e68610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 -- 172.21.15.19:0/1983015346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650b5ede080 con 0x5650b5e5adc0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 0x5650b5e68610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fd2ffd640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005d00 tx=0x7f7fc8012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f6b1d3f4514bba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8021030 con 0x5650b5e5adc0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7fc8010c30 con 0x5650b5e5adc0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.323+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc00fdb0 con 0x5650b5e5adc0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7fc8010e70 con 0x5650b5e5adc0 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 10 client.17445 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:47.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5650b5edf320 con 0x5650b5e5adc0 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fcc00fdb0 con 0x5650b5e5adc0 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f7fc8026450 con 0x5650b5e5adc0 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 10 client.17445 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 10 client.17445 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 20 client.17445 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5edfc10 0x5650b5ee2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 4 client.17445 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650b5d44210 con 0x5650b5edfc10 2024-09-12T03:29:47.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.324+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5edfc10 0x5650b5ee2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.325+0000 7f7fd37fe640 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5edfc10 0x5650b5ee2030 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78762b0eca32e325 server_cookie=1d6900283b6f2507 in_seq=0 out_seq=0 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.325+0000 7f7fb2ffd640 10 client.17445 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fb2ffd640 1 -- 172.21.15.19:0/1983015346 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5650b5d44210 con 0x5650b5edfc10 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fb2ffd640 10 client.17445 handle_command_reply: tid=0 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 client.17445 shutdown 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 msgr2=0x5650b5e68b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8005d00 tx=0x7f7fc8012b60 comp rx=0 tx=0).stop 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 shutdown_connections 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5650b5e65680 0x5650b5eddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5650b5e5adc0 0x5650b5e68b50 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5650b5edfc10 0x5650b5ee2030 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 --2- 172.21.15.19:0/1983015346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5650b5e5a4a0 0x5650b5e68610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 >> 172.21.15.19:0/1983015346 conn(0x5650b5e56a30 msgr2=0x5650b5e60ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fb2ffd640 0 client.17445 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 shutdown_connections 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 1 -- 172.21.15.19:0/1983015346 wait complete. 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.326+0000 7f7fdc031740 20 client.17445 trim_cache size 0 max 16384 2024-09-12T03:29:47.334 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:29:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:29:47.335 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:29:47.335 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:29:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 msgr2=0x7f7fbc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 0x7f7fbc0a5040 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009fd0 tx=0x7f7fcc0096a0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 msgr2=0x7f7fd419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4004770 tx=0x7f7fc4003c10 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 shutdown_connections 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7fd410e3d0 0x7f7fd419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7fd4074f40 0x7f7fd419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7fbc0a2b80 0x7f7fbc0a5040 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 --2- 172.21.15.19:0/3300569643 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7fd4074620 0x7f7fd419a770 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 >> 172.21.15.19:0/3300569643 conn(0x7f7fd40fd8f0 msgr2=0x7f7fd40fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 shutdown_connections 2024-09-12T03:29:47.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:47.327+0000 7f7fd9c58640 1 -- 172.21.15.19:0/3300569643 wait complete. 2024-09-12T03:29:47.644 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:47 smithi019 ceph-mon[30953]: pgmap v659: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-09-12T03:29:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:47 smithi121 ceph-mon[34963]: pgmap v659: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-09-12T03:29:47.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:47 smithi135 ceph-mon[35474]: pgmap v659: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-09-12T03:29:48.167 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:29:48.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:48.432 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4266884948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0100e20 msgr2=0x7f9ad0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4266884948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0100e20 0x7f9ad0101200 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009900 tx=0x7f9acc02a480 comp rx=0 tx=0).stop 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4266884948 shutdown_connections 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4266884948 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0102ef0 0x7f9ad010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4266884948 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 0x7f9ad01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4266884948 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0100e20 0x7f9ad0101200 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4266884948 >> 172.21.15.19:0/4266884948 conn(0x7f9ad00fc7d0 msgr2=0x7f9ad00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4266884948 shutdown_connections 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.243+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4266884948 wait complete. 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 Processor -- start 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 -- start start 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 0x7f9ad0103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 0x7f9ad0103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0110f80 con 0x7f9ad0102ef0 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0110e00 con 0x7f9ad0100e20 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad73a7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad0111100 con 0x7f9ad0102570 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad5944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad5944640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:44172/0 (socket says 172.21.15.19:44172) 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad5944640 1 -- 172.21.15.19:0/4037819690 learned_addr learned my addr 172.21.15.19:0/4037819690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad4942640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 0x7f9ad0103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.244+0000 7f9ad5143640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 0x7f9ad0103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 -- 172.21.15.19:0/4037819690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 msgr2=0x7f9ad0103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 0x7f9ad0103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 -- 172.21.15.19:0/4037819690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 msgr2=0x7f9ad0103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 0x7f9ad0103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 -- 172.21.15.19:0/4037819690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad01a69a0 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5143640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 0x7f9ad0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad5944640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800eaa0 tx=0x7f9ac800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800cdb0 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ac8005020 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad01a6c90 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80116b0 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.245+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad01a71d0 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.246+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ad0101200 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.247+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ac8005560 con 0x7f9ad0102ef0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.248+0000 7f9ac67fc640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 0x7f9abc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.248+0000 7f9ad5143640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 0x7f9abc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.248+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9ac80cf110 con 0x7f9ad0102ef0 2024-09-12T03:29:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.249+0000 7f9ad5143640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 0x7f9abc0a50c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009230 tx=0x7f9acc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.253+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9ac8016030 con 0x7f9ad0102ef0 2024-09-12T03:29:49.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:49 smithi019 ceph-mon[30953]: pgmap v660: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-09-12T03:29:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.472+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ad0104c80 con 0x7f9ad0102ef0 2024-09-12T03:29:49.472 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.472+0000 7f9ac67fc640 1 -- 172.21.15.19:0/4037819690 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7f9ad0104c80 con 0x7f9ad0102ef0 2024-09-12T03:29:49.473 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:29:49.473 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:29:49.473 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:29:49.474 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 msgr2=0x7f9abc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 0x7f9abc0a50c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009230 tx=0x7f9acc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 msgr2=0x7f9ad01041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800eaa0 tx=0x7f9ac800ef60 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 shutdown_connections 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9abc0a2c00 0x7f9abc0a50c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9ad0102ef0 0x7f9ad01041d0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9ad0102570 0x7f9ad0103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 --2- 172.21.15.19:0/4037819690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ad0100e20 0x7f9ad0103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.474+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 >> 172.21.15.19:0/4037819690 conn(0x7f9ad00fc7d0 msgr2=0x7f9ad006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.475+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 shutdown_connections 2024-09-12T03:29:49.475 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:49.475+0000 7f9ad73a7640 1 -- 172.21.15.19:0/4037819690 wait complete. 2024-09-12T03:29:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:49 smithi121 ceph-mon[34963]: pgmap v660: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-09-12T03:29:49.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:49 smithi135 ceph-mon[35474]: pgmap v660: 97 pgs: 97 active+clean; 1.9 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 421 op/s 2024-09-12T03:29:50.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:29:50.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:50 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4037819690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:50.417 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:50 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4037819690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:50.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:50 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4037819690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/1629255004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41082a0 msgr2=0x7f0fc410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/1629255004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41082a0 0x7f0fc410ec40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009cd0 tx=0x7f0fbc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/1629255004 shutdown_connections 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/1629255004 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41082a0 0x7f0fc410ec40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/1629255004 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc41036e0 0x7f0fc4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/1629255004 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc4102d30 0x7f0fc4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.936+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/1629255004 >> 172.21.15.19:0/1629255004 conn(0x7f0fc40fc7d0 msgr2=0x7f0fc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.937+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/1629255004 shutdown_connections 2024-09-12T03:29:50.938 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.937+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/1629255004 wait complete. 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 Processor -- start 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 -- start start 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41036e0 0x7f0fc419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc41082a0 0x7f0fc419ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4101910 con 0x7f0fc4102d30 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4101790 con 0x7f0fc41036e0 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.938+0000 7f0fcbd9f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4101a90 con 0x7f0fc41082a0 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54516/0 (socket says 172.21.15.19:54516) 2024-09-12T03:29:50.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3557984505 learned_addr learned my addr 172.21.15.19:0/3557984505 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fca33c640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc41082a0 0x7f0fc419ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41036e0 0x7f0fc419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3557984505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc41082a0 msgr2=0x7f0fc419ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc41082a0 0x7f0fc419ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3557984505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41036e0 msgr2=0x7f0fc419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41036e0 0x7f0fc419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3557984505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc419f560 con 0x7f0fc4102d30 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.939+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000e9f0 tx=0x7f0fc000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.940+0000 7f0fbaffd640 1 -- 172.21.15.19:0/3557984505 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000cd80 con 0x7f0fc4102d30 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.940+0000 7f0fbaffd640 1 -- 172.21.15.19:0/3557984505 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fc0010910 con 0x7f0fc4102d30 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.940+0000 7f0fbaffd640 1 -- 172.21.15.19:0/3557984505 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00188f0 con 0x7f0fc4102d30 2024-09-12T03:29:50.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.941+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc419f850 con 0x7f0fc4102d30 2024-09-12T03:29:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.942+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc41a7130 con 0x7f0fc4102d30 2024-09-12T03:29:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.944+0000 7f0fbaffd640 1 -- 172.21.15.19:0/3557984505 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0fc0010ac0 con 0x7f0fc4102d30 2024-09-12T03:29:50.944 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.944+0000 7f0fbaffd640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 0x7f0fb00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.944+0000 7f0fbaffd640 1 -- 172.21.15.19:0/3557984505 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0fc00cf450 con 0x7f0fc4102d30 2024-09-12T03:29:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.944+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 0x7f0fb00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.944+0000 7f0fce178740 1 Processor -- start 2024-09-12T03:29:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 -- start start 2024-09-12T03:29:50.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 0x5578a6991090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a69915d0 0x5578a6991a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578a68f72f0 con 0x5578a6992b40 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5578a689ecf0 con 0x5578a699c370 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fce178740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578a68f32b0 con 0x5578a69915d0 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 0x5578a6991090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 0x7f0fb00a5190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4108020 tx=0x7f0fb4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fca33c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fc933a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a69915d0 0x5578a6991a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fca33c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:52976/0 (socket says 172.21.15.19:52976) 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 0x5578a6991090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54526/0 (socket says 172.21.15.19:54526) 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.945+0000 7f0fca33c640 1 -- 172.21.15.19:0/2114653893 learned_addr learned my addr 172.21.15.19:0/2114653893 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 -- 172.21.15.19:0/2114653893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a69915d0 msgr2=0x5578a6991a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a69915d0 0x5578a6991a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 -- 172.21.15.19:0/2114653893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 msgr2=0x5578a6991090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 0x5578a6991090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 -- 172.21.15.19:0/2114653893 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a68833a0 con 0x5578a699c370 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0fca33c640 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009cd0 tx=0x7f0fbc0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2695a977bd4c96 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0faffff640 1 -- 172.21.15.19:0/2114653893 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc048070 con 0x5578a699c370 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0faffff640 1 -- 172.21.15.19:0/2114653893 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc036070 con 0x5578a699c370 2024-09-12T03:29:50.948 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.946+0000 7f0faffff640 1 -- 172.21.15.19:0/2114653893 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc004280 con 0x5578a699c370 2024-09-12T03:29:50.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- 172.21.15.19:0/2114653893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 msgr2=0x5578a699e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009cd0 tx=0x7f0fbc0093e0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- 172.21.15.19:0/2114653893 shutdown_connections 2024-09-12T03:29:50.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a699c370 0x5578a699e760 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a69915d0 0x5578a6991a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 --2- 172.21.15.19:0/2114653893 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a6992b40 0x5578a6991090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- 172.21.15.19:0/2114653893 >> 172.21.15.19:0/2114653893 conn(0x5578a695a310 msgr2=0x5578a698dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- 172.21.15.19:0/2114653893 shutdown_connections 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- 172.21.15.19:0/2114653893 wait complete. 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 -- start start 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.947+0000 7f0fce178740 1 Processor -- start 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a69915d0 0x5578a699bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a6992b40 0x5578a6998ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.952 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5578a699f010 con 0x5578a699c370 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5578a68b2060 con 0x5578a69915d0 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5578a699f190 con 0x5578a6992b40 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fce178740 10 client.0 _ll_register_callbacks cb 0x5578a66058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54534/0 (socket says 172.21.15.19:54534) 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 -- 172.21.15.19:0/3377501746 learned_addr learned my addr 172.21.15.19:0/3377501746 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fc933a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a6992b40 0x5578a6998ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a69915d0 0x5578a699bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 -- 172.21.15.19:0/3377501746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a6992b40 msgr2=0x5578a6998ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a6992b40 0x5578a6998ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 -- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a69915d0 msgr2=0x5578a699bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.953 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a69915d0 0x5578a699bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.948+0000 7f0fca33c640 1 -- 172.21.15.19:0/3377501746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a66683c0 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.949+0000 7f0fca33c640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc046040 tx=0x7f0fbc0010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70641395b32b42c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.950+0000 7f0fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.950+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc052020 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.950+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0fbc003970 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.950+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fbc036070 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.950+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0fbc048040 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 10 client.17463 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5578a699f010 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fbc036070 con 0x5578a699c370 2024-09-12T03:29:50.954 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f0fbc04b030 con 0x5578a699c370 2024-09-12T03:29:50.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 10 client.17463 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:50.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 10 client.17463 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 20 client.17463 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.951+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a69abc30 0x5578a69ae020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.952+0000 7f0fce178740 4 client.17463 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.952+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a6904860 con 0x5578a69abc30 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.952+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a69abc30 0x5578a69ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fad7fa640 10 client.17463 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a69abc30 0x5578a69ae020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb233ec4a151c118 server_cookie=976e5c87bf42be30 in_seq=0 out_seq=0 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3377501746 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578a6904860 con 0x5578a69abc30 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fad7fa640 10 client.17463 handle_command_reply: tid=0 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fce178740 1 client.17463 shutdown 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.953+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 msgr2=0x5578a69995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc046040 tx=0x7f0fbc0010b0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 shutdown_connections 2024-09-12T03:29:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a699c370 0x5578a69995e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a69abc30 0x5578a69ae020 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a6992b40 0x5578a6998ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 --2- 172.21.15.19:0/3377501746 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a69915d0 0x5578a699bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 >> 172.21.15.19:0/3377501746 conn(0x5578a695a310 msgr2=0x5578a6993930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fad7fa640 0 client.17463 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 shutdown_connections 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 1 -- 172.21.15.19:0/3377501746 wait complete. 2024-09-12T03:29:50.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.954+0000 7f0fce178740 20 client.17463 trim_cache size 0 max 16384 2024-09-12T03:29:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 Processor -- start 2024-09-12T03:29:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 -- start start 2024-09-12T03:29:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a698de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a69a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a6994120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41af820 con 0x5578a69915d0 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b15a0 con 0x5578a6992b40 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fce178740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41afef0 con 0x5578a699c370 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a698de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a698de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54542/0 (socket says 172.21.15.19:54542) 2024-09-12T03:29:50.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/1527676978 learned_addr learned my addr 172.21.15.19:0/1527676978 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fc933a640 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a69a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.967+0000 7f0fca33c640 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a6994120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 -- 172.21.15.19:0/1527676978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 msgr2=0x5578a6994120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a6994120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 -- 172.21.15.19:0/1527676978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 msgr2=0x5578a698de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a698de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 -- 172.21.15.19:0/1527676978 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a699b960 con 0x5578a6992b40 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0fc933a640 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a69a82e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5578a6998880 tx=0x7f0fb4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=599092b8dee076f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0faf7fe640 1 -- 172.21.15.19:0/1527676978 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4003ac0 con 0x5578a6992b40 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0faf7fe640 1 -- 172.21.15.19:0/1527676978 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4005500 con 0x5578a6992b40 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.968+0000 7f0faf7fe640 1 -- 172.21.15.19:0/1527676978 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4004d80 con 0x5578a6992b40 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- 172.21.15.19:0/1527676978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 msgr2=0x5578a69a82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.982 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a69a82e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x5578a6998880 tx=0x7f0fb4014000 comp rx=0 tx=0).stop 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- 172.21.15.19:0/1527676978 shutdown_connections 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a6994120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a69a82e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 --2- 172.21.15.19:0/1527676978 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a698de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- 172.21.15.19:0/1527676978 >> 172.21.15.19:0/1527676978 conn(0x5578a695a310 msgr2=0x5578a698d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- 172.21.15.19:0/1527676978 shutdown_connections 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- 172.21.15.19:0/1527676978 wait complete. 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 -- start start 2024-09-12T03:29:50.983 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.969+0000 7f0fce178740 1 Processor -- start 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a6a0f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a699f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b0e10 con 0x5578a69915d0 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41b0680 con 0x5578a6992b40 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc41ad740 con 0x5578a699c370 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fce178740 10 client.0 _ll_register_callbacks cb 0x7f0fbc046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:50.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc933a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a6a0f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54558/0 (socket says 172.21.15.19:54558) 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3421263955 learned_addr learned my addr 172.21.15.19:0/3421263955 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fca33c640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a699f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3421263955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 msgr2=0x5578a699f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a699f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 msgr2=0x5578a6a0f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a6a0f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 -- 172.21.15.19:0/3421263955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a69a00c0 con 0x5578a69915d0 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a6a0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fc9b3b640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c6a0 tx=0x7f0fc0009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50aa8dca97dcf560 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:50.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.970+0000 7f0fad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00aa070 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f0fc0099fc0 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb4005500 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f0fc0081050 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 10 client.17469 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5578a6a162e0 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fb4005500 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f0fc00864a0 con 0x5578a69915d0 2024-09-12T03:29:50.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 10 client.17469 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 10 client.17469 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 20 client.17469 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a6a16a70 0x5578a6a18e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 4 client.17469 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.971+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578a68fed70 con 0x5578a6a16a70 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a6a16a70 0x5578a6a18e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fc933a640 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a6a16a70 0x5578a6a18e90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=734a3ae44f99b20e server_cookie=5f93a9d7b09a0913 in_seq=0 out_seq=0 2024-09-12T03:29:50.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fad7fa640 10 client.17469 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fad7fa640 1 -- 172.21.15.19:0/3421263955 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578a68fed70 con 0x5578a6a16a70 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fad7fa640 10 client.17469 handle_command_reply: tid=0 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 client.17469 shutdown 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 msgr2=0x5578a6a0eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c6a0 tx=0x7f0fc0009790 comp rx=0 tx=0).stop 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 shutdown_connections 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5578a6a16a70 0x5578a6a18e90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5578a699c370 0x5578a699f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5578a6992b40 0x5578a6a0f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 --2- 172.21.15.19:0/3421263955 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5578a69915d0 0x5578a6a0eb10 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fad7fa640 0 client.17469 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 >> 172.21.15.19:0/3421263955 conn(0x5578a695a310 msgr2=0x5578a699a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 shutdown_connections 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.972+0000 7f0fce178740 1 -- 172.21.15.19:0/3421263955 wait complete. 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.973+0000 7f0fce178740 20 client.17469 trim_cache size 0 max 16384 2024-09-12T03:29:50.989 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:50.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.973+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 msgr2=0x7f0fb00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 0x7f0fb00a5190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4108020 tx=0x7f0fb4005f90 comp rx=0 tx=0).stop 2024-09-12T03:29:50.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 msgr2=0x7f0fc419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:50.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000e9f0 tx=0x7f0fc000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.990 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 shutdown_connections 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f0fc41082a0 0x7f0fc419ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0fc41036e0 0x7f0fc419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f0fb00a2cd0 0x7f0fb00a5190 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 --2- 172.21.15.19:0/3557984505 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f0fc4102d30 0x7f0fc419a5b0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 >> 172.21.15.19:0/3557984505 conn(0x7f0fc40fc7d0 msgr2=0x7f0fc40fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 shutdown_connections 2024-09-12T03:29:50.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:50.974+0000 7f0fcbd9f640 1 -- 172.21.15.19:0/3557984505 wait complete. 2024-09-12T03:29:51.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:51 smithi121 ceph-mon[34963]: pgmap v661: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 362 op/s 2024-09-12T03:29:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:51 smithi019 ceph-mon[30953]: pgmap v661: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 362 op/s 2024-09-12T03:29:51.691 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:51 smithi135 ceph-mon[35474]: pgmap v661: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 362 op/s 2024-09-12T03:29:52.188 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:29:52.449 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/3137846720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b281008d0 msgr2=0x7f5b28100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/3137846720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b281008d0 0x7f5b28100cb0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5b24009900 tx=0x7f5b2402a410 comp rx=0 tx=0).stop 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/3137846720 shutdown_connections 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/3137846720 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b2810e480 0x7f5b28110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/3137846720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b28101280 0x7f5b2810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/3137846720 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b281008d0 0x7f5b28100cb0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/3137846720 >> 172.21.15.19:0/3137846720 conn(0x7f5b280fc7a0 msgr2=0x7f5b280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/3137846720 shutdown_connections 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.108+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/3137846720 wait complete. 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 Processor -- start 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 -- start start 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 0x7f5b2819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2810e480 0x7f5b2819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281141b0 con 0x7f5b28101280 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28114030 con 0x7f5b2810e480 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2f8c2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b28114330 con 0x7f5b281008d0 2024-09-12T03:29:53.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54574/0 (socket says 172.21.15.19:54574) 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/1711467913 learned_addr learned my addr 172.21.15.19:0/1711467913 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 0x7f5b2819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.109+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2810e480 0x7f5b2819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/1711467913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 msgr2=0x7f5b2819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 0x7f5b2819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/1711467913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2810e480 msgr2=0x7f5b2819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2810e480 0x7f5b2819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/1711467913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b2819f680 con 0x7f5b28101280 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 0x7f5b2819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b1e7fc640 1 -- 172.21.15.19:0/1711467913 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1800cd80 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b1e7fc640 1 -- 172.21.15.19:0/1711467913 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b18010910 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b2819f970 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b1e7fc640 1 -- 172.21.15.19:0/1711467913 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18018990 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.110+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b281a7250 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.113+0000 7f5b31c9b740 1 Processor -- start 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.113+0000 7f5b1e7fc640 1 -- 172.21.15.19:0/1711467913 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b18010ac0 con 0x7f5b28101280 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.113+0000 7f5b31c9b740 1 -- start start 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c182b682f0 con 0x55c182c03b40 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c182b0fcf0 con 0x55c182c025d0 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c182b642b0 con 0x55c182c0d370 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54576/0 (socket says 172.21.15.19:54576) 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 -- 172.21.15.19:0/3936610325 learned_addr learned my addr 172.21.15.19:0/3936610325 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 -- 172.21.15.19:0/3936610325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 msgr2=0x55c182c0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 -- 172.21.15.19:0/3936610325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 msgr2=0x55c182c02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 -- 172.21.15.19:0/3936610325 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c182af43a0 con 0x55c182c03b40 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b1e7fc640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 0x7f5b140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 0x7f5b140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009f30 tx=0x7f5b200118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2ed4ef7a8183f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.114+0000 7f5b1e7fc640 1 -- 172.21.15.19:0/1711467913 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5b180cf340 con 0x7f5b28101280 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b137fe640 1 -- 172.21.15.19:0/3936610325 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004d20 con 0x55c182c03b40 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b137fe640 1 -- 172.21.15.19:0/3936610325 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b20004050 con 0x55c182c03b40 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b137fe640 1 -- 172.21.15.19:0/3936610325 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20005260 con 0x55c182c03b40 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 -- 172.21.15.19:0/3936610325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 msgr2=0x55c182c02090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 0x7f5b140a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55c182bffac0 tx=0x7f5b24005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009f30 tx=0x7f5b200118d0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 -- 172.21.15.19:0/3936610325 shutdown_connections 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/3936610325 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c02090 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.115+0000 7f5b31c9b740 1 -- 172.21.15.19:0/3936610325 >> 172.21.15.19:0/3936610325 conn(0x55c182bcb310 msgr2=0x55c182bfec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.116+0000 7f5b31c9b740 1 -- 172.21.15.19:0/3936610325 shutdown_connections 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.116+0000 7f5b31c9b740 1 -- 172.21.15.19:0/3936610325 wait complete. 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.116+0000 7f5b31c9b740 1 -- start start 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.116+0000 7f5b31c9b740 1 Processor -- start 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 0x55c182c09fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c182c10070 con 0x55c182c025d0 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c182b23060 con 0x55c182c0d370 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c182c101f0 con 0x55c182c03b40 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b31c9b740 10 client.0 _ll_register_callbacks cb 0x55c1828768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 0x55c182c09fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54584/0 (socket says 172.21.15.19:54584) 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 -- 172.21.15.19:0/905950865 learned_addr learned my addr 172.21.15.19:0/905950865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2de5f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 -- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 msgr2=0x55c182c09fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 0x55c182c09fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 -- 172.21.15.19:0/905950865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 msgr2=0x55c182c0a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c0a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 -- 172.21.15.19:0/905950865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1828d93c0 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 0x55c182c09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5b24033000 tx=0x7f5b24009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e479646ba97dc611 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.117+0000 7f5b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2404c070 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5b24005460 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b20004050 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5b24058050 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 10 client.17481 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c182c1bad0 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b20004050 con 0x55c182c025d0 2024-09-12T03:29:53.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f5b240035f0 con 0x55c182c025d0 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 10 client.17481 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 10 client.17481 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 20 client.17481 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c1cfd0 0x55c182c1f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 4 client.17481 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.118+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c182b75860 con 0x55c182c1cfd0 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.119+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c1cfd0 0x55c182c1f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.119+0000 7f5b10ff9640 10 client.17481 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:53.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.119+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c1cfd0 0x55c182c1f430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5679255c7d248189 server_cookie=4cf457a1477a0d12 in_seq=0 out_seq=0 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b10ff9640 1 -- 172.21.15.19:0/905950865 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c182b75860 con 0x55c182c1cfd0 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b10ff9640 10 client.17481 handle_command_reply: tid=0 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 client.17481 shutdown 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 msgr2=0x55c182c0cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5b24033000 tx=0x7f5b24009260 comp rx=0 tx=0).stop 2024-09-12T03:29:53.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 shutdown_connections 2024-09-12T03:29:53.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c1cfd0 0x55c182c1f430 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c0a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c03b40 0x55c182c09fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b10ff9640 0 client.17481 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:53.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/905950865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c025d0 0x55c182c0cd30 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 >> 172.21.15.19:0/905950865 conn(0x55c182bcb310 msgr2=0x55c182c04930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 shutdown_connections 2024-09-12T03:29:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 1 -- 172.21.15.19:0/905950865 wait complete. 2024-09-12T03:29:53.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.120+0000 7f5b31c9b740 20 client.17481 trim_cache size 0 max 16384 2024-09-12T03:29:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 Processor -- start 2024-09-12T03:29:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 -- start start 2024-09-12T03:29:53.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c05120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c24d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281af8e0 con 0x55c182c03b40 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.134+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281b1660 con 0x55c182c025d0 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281affb0 con 0x55c182c0d370 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54586/0 (socket says 172.21.15.19:54586) 2024-09-12T03:29:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/994644518 learned_addr learned my addr 172.21.15.19:0/994644518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c24d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c05120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/994644518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 msgr2=0x55c182c24d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.148 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c24d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/994644518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 msgr2=0x55c182c05120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c05120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/994644518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c182c02d50 con 0x55c182c03b40 2024-09-12T03:29:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c05120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5b18098580 tx=0x7f5b180d2e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19dcc75b740918e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b12ffd640 1 -- 172.21.15.19:0/994644518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b1809b510 con 0x55c182c03b40 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b12ffd640 1 -- 172.21.15.19:0/994644518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b1809c2b0 con 0x55c182c03b40 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.135+0000 7f5b12ffd640 1 -- 172.21.15.19:0/994644518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b18018d90 con 0x55c182c03b40 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- 172.21.15.19:0/994644518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 msgr2=0x55c182c05660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f5b18098580 tx=0x7f5b180d2e20 comp rx=0 tx=0).stop 2024-09-12T03:29:53.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- 172.21.15.19:0/994644518 shutdown_connections 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c0d370 0x55c182c24d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c05660 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/994644518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c025d0 0x55c182c05120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- 172.21.15.19:0/994644518 >> 172.21.15.19:0/994644518 conn(0x55c182bcb310 msgr2=0x55c182bfdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- 172.21.15.19:0/994644518 shutdown_connections 2024-09-12T03:29:53.151 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- 172.21.15.19:0/994644518 wait complete. 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.136+0000 7f5b31c9b740 1 -- start start 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 Processor -- start 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c025d0 0x55c182c15c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c11530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281b0ed0 con 0x55c182c03b40 2024-09-12T03:29:53.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281b0740 con 0x55c182c0d370 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5b281ad800 con 0x55c182c025d0 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b31c9b740 10 client.0 _ll_register_callbacks cb 0x7f5b200093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54588/0 (socket says 172.21.15.19:54588) 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/2488576416 learned_addr learned my addr 172.21.15.19:0/2488576416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c025d0 0x55c182c15c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c11530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c025d0 msgr2=0x55c182c15c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c025d0 0x55c182c15c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/2488576416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 msgr2=0x55c182c11530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c11530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2ce5d640 1 -- 172.21.15.19:0/2488576416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c182c11d60 con 0x55c182c03b40 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.137+0000 7f5b2de5f640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b2ce5d640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5b18012010 tx=0x7f5b18097820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac628d9e4301c9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:53.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b180b2020 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5b18018b30 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b1809c2b0 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5b1809bbb0 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 10 client.17487 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c182c87ce0 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b1809c2b0 con 0x55c182c03b40 2024-09-12T03:29:53.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f5b18097900 con 0x55c182c03b40 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 10 client.17487 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 10 client.17487 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 20 client.17487 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c884c0 0x55c182c8a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 4 client.17487 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:53.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.138+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c182b6fd70 con 0x55c182c884c0 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c884c0 0x55c182c8a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b2d65e640 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c884c0 0x55c182c8a8e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4f412e9743b05630 server_cookie=25aa7c24a0cf39fc in_seq=0 out_seq=0 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b10ff9640 10 client.17487 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b10ff9640 1 -- 172.21.15.19:0/2488576416 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c182b6fd70 con 0x55c182c884c0 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b10ff9640 10 client.17487 handle_command_reply: tid=0 2024-09-12T03:29:53.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b31c9b740 1 client.17487 shutdown 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 msgr2=0x55c182c10fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f5b18012010 tx=0x7f5b18097820 comp rx=0 tx=0).stop 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 shutdown_connections 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.139+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55c182c884c0 0x55c182c8a8e0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c182c0d370 0x55c182c11530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c182c03b40 0x55c182c10fb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b10ff9640 0 client.17487 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 --2- 172.21.15.19:0/2488576416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c182c025d0 0x55c182c15c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 >> 172.21.15.19:0/2488576416 conn(0x55c182bcb310 msgr2=0x55c182c0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 shutdown_connections 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 1 -- 172.21.15.19:0/2488576416 wait complete. 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b31c9b740 20 client.17487 trim_cache size 0 max 16384 2024-09-12T03:29:53.159 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:53.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 msgr2=0x7f5b140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 0x7f5b140a50c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55c182bffac0 tx=0x7f5b24005b50 comp rx=0 tx=0).stop 2024-09-12T03:29:53.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 msgr2=0x7f5b2819ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:53.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5b1800e9f0 tx=0x7f5b1800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 shutdown_connections 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b2810e480 0x7f5b2819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5b140a2c00 0x7f5b140a50c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5b28101280 0x7f5b2819ac10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 --2- 172.21.15.19:0/1711467913 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5b281008d0 0x7f5b2819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.140+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 >> 172.21.15.19:0/1711467913 conn(0x7f5b280fc7a0 msgr2=0x7f5b280feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.141+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 shutdown_connections 2024-09-12T03:29:53.161 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:53.141+0000 7f5b2f8c2640 1 -- 172.21.15.19:0/1711467913 wait complete. 2024-09-12T03:29:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:53 smithi135 ceph-mon[35474]: pgmap v662: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-09-12T03:29:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:53 smithi121 ceph-mon[34963]: pgmap v662: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-09-12T03:29:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:53 smithi019 ceph-mon[30953]: pgmap v662: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 421 op/s 2024-09-12T03:29:53.734 INFO:tasks.quiescer.fs.[cephfs]:Total duration (45.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:29:53.734 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T03:29:53.735 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:29:53.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:29:54.061 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.692+0000 7ff27bfff640 1 -- 172.21.15.19:0/4102053170 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff280031940 con 0x7ff284100e20 2024-09-12T03:29:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.693+0000 7ff28b960640 1 -- 172.21.15.19:0/4102053170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284100e20 msgr2=0x7ff284101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.693+0000 7ff28b960640 1 --2- 172.21.15.19:0/4102053170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284100e20 0x7ff284101200 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff280009900 tx=0x7ff28002a4a0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.694+0000 7ff28b960640 1 -- 172.21.15.19:0/4102053170 shutdown_connections 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.694+0000 7ff28b960640 1 --2- 172.21.15.19:0/4102053170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102ef0 0x7ff28410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.694+0000 7ff28b960640 1 --2- 172.21.15.19:0/4102053170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff284102570 0x7ff2841029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.694+0000 7ff28b960640 1 --2- 172.21.15.19:0/4102053170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284100e20 0x7ff284101200 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.694+0000 7ff28b960640 1 -- 172.21.15.19:0/4102053170 >> 172.21.15.19:0/4102053170 conn(0x7ff2840fc7d0 msgr2=0x7ff2840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.695+0000 7ff28b960640 1 -- 172.21.15.19:0/4102053170 shutdown_connections 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.695+0000 7ff28b960640 1 -- 172.21.15.19:0/4102053170 wait complete. 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.696+0000 7ff28b960640 1 Processor -- start 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.696+0000 7ff28b960640 1 -- start start 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff284100e20 0x7ff284073da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 0x7ff28406fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406c660 con 0x7ff284102ef0 2024-09-12T03:29:54.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406c4e0 con 0x7ff284100e20 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff28b960640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406c7e0 con 0x7ff284102570 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff288efb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 0x7ff28406fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54604/0 (socket says 172.21.15.19:54604) 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff288efb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 0x7ff28406fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:59048/0 (socket says 172.21.15.19:59048) 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 -- 172.21.15.19:0/1862617516 learned_addr learned my addr 172.21.15.19:0/1862617516 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 -- 172.21.15.19:0/1862617516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 msgr2=0x7ff28406fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 0x7ff28406fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 -- 172.21.15.19:0/1862617516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff284100e20 msgr2=0x7ff284073da0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff284100e20 0x7ff284073da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.697+0000 7ff289efd640 1 -- 172.21.15.19:0/1862617516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284070c60 con 0x7ff284102ef0 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.698+0000 7ff289efd640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00b990 tx=0x7ff27c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:54.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.699+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c019070 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.699+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff27c00cba0 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.699+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c007650 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.699+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2841af500 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.700+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841afac0 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.701+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2841021b0 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.703+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff27c01c070 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.704+0000 7ff27a7fc640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 0x7ff2700a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.704+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff27c0ce9e0 con 0x7ff284102ef0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.704+0000 7ff2896fc640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 0x7ff2700a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:54.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.704+0000 7ff2896fc640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 0x7ff2700a5190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff2800314e0 tx=0x7ff280000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:54.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.712+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff27c09b320 con 0x7ff284102ef0 2024-09-12T03:29:54.766 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:54.998 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.998+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff284101200 con 0x7ff284102ef0 2024-09-12T03:29:54.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:54.999+0000 7ff27a7fc640 1 -- 172.21.15.19:0/1862617516 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7ff27c09f620 con 0x7ff284102ef0 2024-09-12T03:29:55.000 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:29:55.000 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:29:55.000 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.002+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 msgr2=0x7ff2700a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.002+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 0x7ff2700a5190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff2800314e0 tx=0x7ff280000bc0 comp rx=0 tx=0).stop 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.002+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 msgr2=0x7ff2840703a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.002+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7ff27c00b990 tx=0x7ff27c00be50 comp rx=0 tx=0).stop 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 shutdown_connections 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff2700a2cd0 0x7ff2700a5190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff284102ef0 0x7ff2840703a0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:55.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff284102570 0x7ff28406fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 --2- 172.21.15.19:0/1862617516 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff284100e20 0x7ff284073da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 >> 172.21.15.19:0/1862617516 conn(0x7ff2840fc7d0 msgr2=0x7ff284105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 shutdown_connections 2024-09-12T03:29:55.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:55.003+0000 7ff28b960640 1 -- 172.21.15.19:0/1862617516 wait complete. 2024-09-12T03:29:55.653 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:55 smithi121 ceph-mon[34963]: pgmap v663: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-09-12T03:29:55.653 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:55 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1862617516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:55.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:55 smithi019 ceph-mon[30953]: pgmap v663: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-09-12T03:29:55.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:55 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1862617516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:55.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:55 smithi135 ceph-mon[35474]: pgmap v663: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 403 op/s 2024-09-12T03:29:55.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:55 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1862617516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:29:55.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T03:29:55.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 -- 172.21.15.19:0/3905961908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014100de0 msgr2=0x7f4014104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 --2- 172.21.15.19:0/3905961908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014100de0 0x7f4014104e30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4004009900 tx=0x7f400402a400 comp rx=0 tx=0).stop 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 -- 172.21.15.19:0/3905961908 shutdown_connections 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 --2- 172.21.15.19:0/3905961908 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4014105560 0x7f401410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 --2- 172.21.15.19:0/3905961908 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014100de0 0x7f4014104e30 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4004009900 tx=0x7f400402a400 comp rx=0 tx=0).stop 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 --2- 172.21.15.19:0/3905961908 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f4014100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.031+0000 7f401be81640 1 -- 172.21.15.19:0/3905961908 >> 172.21.15.19:0/3905961908 conn(0x7f4014077230 msgr2=0x7f40140fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.032+0000 7f401be81640 1 -- 172.21.15.19:0/3905961908 shutdown_connections 2024-09-12T03:29:56.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.032+0000 7f401be81640 1 -- 172.21.15.19:0/3905961908 wait complete. 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.033+0000 7f401be81640 1 Processor -- start 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 -- start start 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 0x7f4014198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401419cf80 0x7f401419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111f90 con 0x7f401419cf80 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f401be81640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4014111e10 con 0x7f4014100430 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53072/0 (socket says 172.21.15.19:53072) 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.034+0000 7f4019c1d640 1 -- 172.21.15.19:0/3358715453 learned_addr learned my addr 172.21.15.19:0/3358715453 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:56.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f4014112110 con 0x7f4014105560 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f401941c640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 0x7f4014198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f4019c1d640 1 -- 172.21.15.19:0/3358715453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 msgr2=0x7f4014198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 0x7f4014198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f4019c1d640 1 -- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401419cf80 msgr2=0x7f401419d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401419cf80 0x7f401419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f4019c1d640 1 -- 172.21.15.19:0/3358715453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401419da40 con 0x7f4014100430 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.035+0000 7f401941c640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 0x7f4014198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.036+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f401000d950 tx=0x7f401000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.036+0000 7f400affd640 1 -- 172.21.15.19:0/3358715453 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010017070 con 0x7f4014100430 2024-09-12T03:29:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.036+0000 7f400affd640 1 -- 172.21.15.19:0/3358715453 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4010009c30 con 0x7f4014100430 2024-09-12T03:29:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.036+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401419dd30 con 0x7f4014100430 2024-09-12T03:29:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.036+0000 7f400affd640 1 -- 172.21.15.19:0/3358715453 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401000f6b0 con 0x7f4014100430 2024-09-12T03:29:56.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.037+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a5400 con 0x7f4014100430 2024-09-12T03:29:56.041 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.041+0000 7f400affd640 1 -- 172.21.15.19:0/3358715453 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40100109d0 con 0x7f4014100430 2024-09-12T03:29:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.042+0000 7f401e25a740 1 Processor -- start 2024-09-12T03:29:56.042 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.042+0000 7f400affd640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 0x7f40000a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.042+0000 7f400affd640 1 -- 172.21.15.19:0/3358715453 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40100cec90 con 0x7f4014100430 2024-09-12T03:29:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 -- start start 2024-09-12T03:29:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401a41e640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 0x7f40000a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.043 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f186c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563cd8e742f0 con 0x563cd8f18280 2024-09-12T03:29:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563cd8e1bcf0 con 0x563cd8f1d5d0 2024-09-12T03:29:56.044 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.043+0000 7f401e25a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cd8e702b0 con 0x563cd8f18c00 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53082/0 (socket says 172.21.15.19:53082) 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 -- 172.21.15.19:0/1406089787 learned_addr learned my addr 172.21.15.19:0/1406089787 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f401941c640 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 -- 172.21.15.19:0/1406089787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 msgr2=0x563cd8f24330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 -- 172.21.15.19:0/1406089787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 msgr2=0x563cd8f186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f186c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.044+0000 7f4019c1d640 1 -- 172.21.15.19:0/1406089787 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd8e003a0 con 0x563cd8f1d5d0 2024-09-12T03:29:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.045+0000 7f401a41e640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 0x7f40000a5250 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f400c009370 tx=0x7f400c009730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.045+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f401009b1a0 tx=0x7f4010009f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=791eee47d6e17d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.045+0000 7f3ffffff640 1 -- 172.21.15.19:0/1406089787 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010020030 con 0x563cd8f1d5d0 2024-09-12T03:29:56.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.045+0000 7f3ffffff640 1 -- 172.21.15.19:0/1406089787 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401009f850 con 0x563cd8f1d5d0 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.046+0000 7f3ffffff640 1 -- 172.21.15.19:0/1406089787 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f401009a360 con 0x563cd8f1d5d0 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- 172.21.15.19:0/1406089787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 msgr2=0x563cd8f1d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f401009b1a0 tx=0x7f4010009f20 comp rx=0 tx=0).stop 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- 172.21.15.19:0/1406089787 shutdown_connections 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f24330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.047 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 --2- 172.21.15.19:0/1406089787 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f1d5d0 0x563cd8f1d9b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- 172.21.15.19:0/1406089787 >> 172.21.15.19:0/1406089787 conn(0x563cd8ed7310 msgr2=0x563cd8f0a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- 172.21.15.19:0/1406089787 shutdown_connections 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- 172.21.15.19:0/1406089787 wait complete. 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.047+0000 7f401e25a740 1 -- start start 2024-09-12T03:29:56.048 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 Processor -- start 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563cd8f24930 con 0x563cd8f1d5d0 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563cd8e2f060 con 0x563cd8f18280 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563cd8f24ab0 con 0x563cd8f18c00 2024-09-12T03:29:56.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f401e25a740 10 client.0 _ll_register_callbacks cb 0x563cd8b828c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.048+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53090/0 (socket says 172.21.15.19:53090) 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 -- 172.21.15.19:0/2808738923 learned_addr learned my addr 172.21.15.19:0/2808738923 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f401941c640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f401a41e640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 -- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 msgr2=0x563cd8f21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 -- 172.21.15.19:0/2808738923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 msgr2=0x563cd8f219d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f219d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f4019c1d640 1 -- 172.21.15.19:0/2808738923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd8be53c0 con 0x563cd8f18280 2024-09-12T03:29:56.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.049+0000 7f401941c640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.050+0000 7f4019c1d640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f401009e040 tx=0x7f4010010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59dbccd773bf9b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.050+0000 7f3ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.051+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40100aa060 con 0x563cd8f18280 2024-09-12T03:29:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.051+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40100b4430 con 0x563cd8f18280 2024-09-12T03:29:56.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.051+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f401009f850 con 0x563cd8f18280 2024-09-12T03:29:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.052+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4010020050 con 0x563cd8f18280 2024-09-12T03:29:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.052+0000 7f401e25a740 10 client.26894 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.052+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563cd8f30350 con 0x563cd8f18280 2024-09-12T03:29:56.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.053+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f401009f850 con 0x563cd8f18280 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.054+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f401009a8f0 con 0x563cd8f18280 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.054+0000 7f401e25a740 10 client.26894 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.054+0000 7f401e25a740 10 client.26894 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.054+0000 7f401e25a740 20 client.26894 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.054+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f33d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.055+0000 7f401e25a740 4 client.26894 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:56.055 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.055+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cd8e81860 con 0x563cd8f318f0 2024-09-12T03:29:56.056 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.055+0000 7f401941c640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f33d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.056+0000 7f3ffd7fa640 10 client.26894 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.056+0000 7f401941c640 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f33d50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ff389158b8be66a8 server_cookie=39b145bd63eedfc1 in_seq=0 out_seq=0 2024-09-12T03:29:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/2808738923 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cd8e81860 con 0x563cd8f318f0 2024-09-12T03:29:56.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f3ffd7fa640 10 client.26894 handle_command_reply: tid=0 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f401e25a740 1 client.26894 shutdown 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 msgr2=0x563cd8f241c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f401009e040 tx=0x7f4010010cc0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.058+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 shutdown_connections 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f33d50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f219d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18c00 0x563cd8f21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f3ffd7fa640 0 client.26894 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 --2- 172.21.15.19:0/2808738923 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18280 0x563cd8f241c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 >> 172.21.15.19:0/2808738923 conn(0x563cd8ed7310 msgr2=0x563cd8f17a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 shutdown_connections 2024-09-12T03:29:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 1 -- 172.21.15.19:0/2808738923 wait complete. 2024-09-12T03:29:56.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.059+0000 7f401e25a740 20 client.26894 trim_cache size 0 max 16384 2024-09-12T03:29:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.081+0000 7f401e25a740 1 Processor -- start 2024-09-12T03:29:56.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.081+0000 7f401e25a740 1 -- start start 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 0x563cd8f2dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f09500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f09a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40141abb10 con 0x563cd8f1d5d0 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40141ad890 con 0x563cd8f18c00 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401e25a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40141ac1e0 con 0x563cd8f18280 2024-09-12T03:29:56.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401941c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f09500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401941c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f09500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53100/0 (socket says 172.21.15.19:53100) 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401941c640 1 -- 172.21.15.19:0/1686763514 learned_addr learned my addr 172.21.15.19:0/1686763514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 0x563cd8f2dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.082+0000 7f401a41e640 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f09a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 -- 172.21.15.19:0/1686763514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 msgr2=0x563cd8f09500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f09500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 -- 172.21.15.19:0/1686763514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 msgr2=0x563cd8f09a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f09a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 -- 172.21.15.19:0/1686763514 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cd8f24ab0 con 0x563cd8f18280 2024-09-12T03:29:56.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.083+0000 7f4019c1d640 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 0x563cd8f2dbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4010097180 tx=0x7f4010005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d36fe7d562cda0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.084+0000 7f3fff7fe640 1 -- 172.21.15.19:0/1686763514 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40100aa070 con 0x563cd8f18280 2024-09-12T03:29:56.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.084+0000 7f3fff7fe640 1 -- 172.21.15.19:0/1686763514 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4010099860 con 0x563cd8f18280 2024-09-12T03:29:56.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.084+0000 7f3fff7fe640 1 -- 172.21.15.19:0/1686763514 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40100b4490 con 0x563cd8f18280 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.085+0000 7f401e25a740 1 -- 172.21.15.19:0/1686763514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 msgr2=0x563cd8f2dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.085+0000 7f401e25a740 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 0x563cd8f2dbe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4010097180 tx=0x7f4010005880 comp rx=0 tx=0).stop 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 -- 172.21.15.19:0/1686763514 shutdown_connections 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f1d5d0 0x563cd8f09a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f09500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 --2- 172.21.15.19:0/1686763514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f18280 0x563cd8f2dbe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 -- 172.21.15.19:0/1686763514 >> 172.21.15.19:0/1686763514 conn(0x563cd8ed7310 msgr2=0x563cd8f211d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 -- 172.21.15.19:0/1686763514 shutdown_connections 2024-09-12T03:29:56.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 -- 172.21.15.19:0/1686763514 wait complete. 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.086+0000 7f401e25a740 1 -- start start 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 Processor -- start 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f2a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 0x563cd8f38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40141ad100 con 0x563cd8f18280 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.087+0000 7f401e25a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40141ac970 con 0x563cd8f18c00 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401e25a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40141a9a30 con 0x563cd8f1d5d0 2024-09-12T03:29:56.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401a41e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 0x563cd8f38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f4019c1d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54668/0 (socket says 172.21.15.19:54668) 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f4019c1d640 1 -- 172.21.15.19:0/3618015658 learned_addr learned my addr 172.21.15.19:0/3618015658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401e25a740 10 client.0 _ll_register_callbacks cb 0x7f4004009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401a41e640 1 -- 172.21.15.19:0/3618015658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 msgr2=0x563cd8f2a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401a41e640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.088+0000 7f401a41e640 1 -- 172.21.15.19:0/3618015658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 msgr2=0x563cd8f29fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f401a41e640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f401a41e640 1 -- 172.21.15.19:0/3618015658 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cd8f39440 con 0x563cd8f1d5d0 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f401a41e640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 0x563cd8f38c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x563cd8f1fba0 tx=0x7f400c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a402e266e2d7465 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:56.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f3ffd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f400c01f860 con 0x563cd8f1d5d0 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f400c01fa00 con 0x563cd8f1d5d0 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.089+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563cd8e7bd70 con 0x563cd8f1d5d0 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f400c01fc00 con 0x563cd8f1d5d0 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 10 client.26863 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:56.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563cd8f9c1b0 con 0x563cd8f1d5d0 2024-09-12T03:29:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563cd8e7bd70 con 0x563cd8f1d5d0 2024-09-12T03:29:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f400c005710 con 0x563cd8f1d5d0 2024-09-12T03:29:56.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 10 client.26863 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 10 client.26863 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 20 client.26863 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 4 client.26863 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563cd8df8170 con 0x563cd8f318f0 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.090+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f4019c1d640 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f9eaa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d22b15d0cca1a97c server_cookie=5713adda3da87094 in_seq=0 out_seq=0 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f3ffd7fa640 10 client.26863 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f3ffd7fa640 1 -- 172.21.15.19:0/3618015658 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563cd8df8170 con 0x563cd8f318f0 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f3ffd7fa640 10 client.26863 handle_command_reply: tid=0 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 client.26863 shutdown 2024-09-12T03:29:56.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 msgr2=0x563cd8f38c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 0x563cd8f38c10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x563cd8f1fba0 tx=0x7f400c004a40 comp rx=0 tx=0).stop 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 shutdown_connections 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x563cd8f318f0 0x563cd8f9eaa0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563cd8f1d5d0 0x563cd8f38c10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563cd8f18c00 0x563cd8f2a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 --2- 172.21.15.19:0/3618015658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563cd8f18280 0x563cd8f29fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f3ffd7fa640 0 client.26863 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.091+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 >> 172.21.15.19:0/3618015658 conn(0x563cd8ed7310 msgr2=0x563cd8f17270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 shutdown_connections 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401e25a740 1 -- 172.21.15.19:0/3618015658 wait complete. 2024-09-12T03:29:56.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401e25a740 20 client.26863 trim_cache size 0 max 16384 2024-09-12T03:29:56.095 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 msgr2=0x7f40000a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 0x7f40000a5250 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f400c009370 tx=0x7f400c009730 comp rx=0 tx=0).stop 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 msgr2=0x7f40141986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.092+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f401000d950 tx=0x7f401000de10 comp rx=0 tx=0).stop 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 shutdown_connections 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40000a2d90 0x7f40000a5250 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.096 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f401419cf80 0x7f401419d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4014105560 0x7f4014198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 --2- 172.21.15.19:0/3358715453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4014100430 0x7f40141986b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:56.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 >> 172.21.15.19:0/3358715453 conn(0x7f4014077230 msgr2=0x7f4014103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:56.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 shutdown_connections 2024-09-12T03:29:56.097 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:56.093+0000 7f401be81640 1 -- 172.21.15.19:0/3358715453 wait complete. 2024-09-12T03:29:56.823 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:29:56.882 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:57 smithi121 ceph-mon[34963]: pgmap v664: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 397 op/s 2024-09-12T03:29:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:57 smithi019 ceph-mon[30953]: pgmap v664: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 397 op/s 2024-09-12T03:29:57.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:57 smithi135 ceph-mon[35474]: pgmap v664: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 397 op/s 2024-09-12T03:29:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 -- 172.21.15.19:0/3936435854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 msgr2=0x7fe5a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.808 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 --2- 172.21.15.19:0/3936435854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a01029b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe590009900 tx=0x7fe59002a480 comp rx=0 tx=0).stop 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 -- 172.21.15.19:0/3936435854 shutdown_connections 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 --2- 172.21.15.19:0/3936435854 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 --2- 172.21.15.19:0/3936435854 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a01029b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 --2- 172.21.15.19:0/3936435854 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 -- 172.21.15.19:0/3936435854 >> 172.21.15.19:0/3936435854 conn(0x7fe5a00fc7d0 msgr2=0x7fe5a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 -- 172.21.15.19:0/3936435854 shutdown_connections 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.800+0000 7fe5a732d640 1 -- 172.21.15.19:0/3936435854 wait complete. 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 Processor -- start 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 -- start start 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a010e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.809 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01120a0 con 0x7fe5a0102570 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0111f20 con 0x7fe5a0100e20 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a732d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0112220 con 0x7fe5a0102ef0 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a48c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a48c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54676/0 (socket says 172.21.15.19:54676) 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a48c8640 1 -- 172.21.15.19:0/641461307 learned_addr learned my addr 172.21.15.19:0/641461307 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a010e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.801+0000 7fe5a58ca640 1 -- 172.21.15.19:0/641461307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 msgr2=0x7fe5a010e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a010e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.810 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a58ca640 1 -- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 msgr2=0x7fe5a010ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a58ca640 1 -- 172.21.15.19:0/641461307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a01a69a0 con 0x7fe5a0102ef0 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010ef50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe598009fd0 tx=0x7fe59800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.811 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5967fc640 1 -- 172.21.15.19:0/641461307 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800ce20 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a01a6c90 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5967fc640 1 -- 172.21.15.19:0/641461307 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe598005020 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5967fc640 1 -- 172.21.15.19:0/641461307 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598010720 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.802+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01a71d0 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.804+0000 7fe5a9706740 1 Processor -- start 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.804+0000 7fe5967fc640 1 -- 172.21.15.19:0/641461307 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe598005540 con 0x7fe5a0102ef0 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 -- start start 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffb3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffbe4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 0x5623fffc0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5623fff18160 con 0x5623fffbea30 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5623ffebf720 con 0x5623fffb3850 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a9706740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623fff14120 con 0x5623fffb4170 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a48c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 0x5623fffc0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffbe4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a48c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 0x5623fffc0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54688/0 (socket says 172.21.15.19:54688) 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a48c8640 1 -- 172.21.15.19:0/45033225 learned_addr learned my addr 172.21.15.19:0/45033225 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffb3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 -- 172.21.15.19:0/45033225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 msgr2=0x5623fffb3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffb3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 -- 172.21.15.19:0/45033225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 msgr2=0x5623fffc0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 0x5623fffc0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5a50c9640 1 -- 172.21.15.19:0/45033225 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623ffc89ba0 con 0x5623fffb4170 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.805+0000 7fe5967fc640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 0x7fe5880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 0x7fe5880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffbe4f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c30 tx=0x7fe59c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb91c10b4ded7d6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5967fc640 1 -- 172.21.15.19:0/641461307 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe5980cf6d0 con 0x7fe5a0102ef0 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5877fe640 1 -- 172.21.15.19:0/45033225 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c012450 con 0x5623fffb4170 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5877fe640 1 -- 172.21.15.19:0/45033225 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c012ad0 con 0x5623fffb4170 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.806+0000 7fe5877fe640 1 -- 172.21.15.19:0/45033225 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c00fab0 con 0x5623fffb4170 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- 172.21.15.19:0/45033225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 msgr2=0x5623fffbe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 0x7fe5880a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe590005770 tx=0x7fe590005b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffbe4f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009c30 tx=0x7fe59c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- 172.21.15.19:0/45033225 shutdown_connections 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffbea30 0x5623fffc0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffbe4f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 --2- 172.21.15.19:0/45033225 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffb3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- 172.21.15.19:0/45033225 >> 172.21.15.19:0/45033225 conn(0x5623fffafde0 msgr2=0x5623fffb01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- 172.21.15.19:0/45033225 shutdown_connections 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- 172.21.15.19:0/45033225 wait complete. 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 -- start start 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.807+0000 7fe5a9706740 1 Processor -- start 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffbb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb4170 0x5623fffbbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffbc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5623fffc1600 con 0x5623fffb4170 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5623ffea0900 con 0x5623fffb3850 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5623fffc1780 con 0x5623fffbea30 2024-09-12T03:29:57.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a9706740 10 client.0 _ll_register_callbacks cb 0x5623ffe9e780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a58ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffbc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a48c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb4170 0x5623fffbbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffbb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a58ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffbc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:59148/0 (socket says 172.21.15.19:59148) 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a58ca640 1 -- 172.21.15.19:0/3604352051 learned_addr learned my addr 172.21.15.19:0/3604352051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 -- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 msgr2=0x5623fffbc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffbc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 -- 172.21.15.19:0/3604352051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb4170 msgr2=0x5623fffbbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb4170 0x5623fffbbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.808+0000 7fe5a50c9640 1 -- 172.21.15.19:0/3604352051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623ffee0c50 con 0x5623fffb3850 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffbb890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe59c01c040 tx=0x7fe59c012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9b4f4bf67901d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c029020 con 0x5623fffb3850 2024-09-12T03:29:57.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe59c012660 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c012ad0 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe59c012880 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 10 client.26909 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5623fffcd260 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c012ad0 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fe59c00fd10 con 0x5623fffb3850 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 10 client.26909 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 10 client.26909 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 20 client.26909 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623fffce5b0 0x5623fffd09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 4 client.26909 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:57.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.809+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623fff25890 con 0x5623fffce5b0 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.810+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623fffce5b0 0x5623fffd09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.810+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623fffce5b0 0x5623fffd09a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab34d0cd8423963 server_cookie=cda4a2aeb0d9b55e in_seq=0 out_seq=0 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.810+0000 7fe584ff9640 10 client.26909 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe584ff9640 1 -- 172.21.15.19:0/3604352051 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623fff25890 con 0x5623fffce5b0 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe584ff9640 10 client.26909 handle_command_reply: tid=0 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 client.26909 shutdown 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 msgr2=0x5623fffbb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffbb890 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe59c01c040 tx=0x7fe59c012eb0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 shutdown_connections 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffbc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb4170 0x5623fffbbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5623fffce5b0 0x5623fffd09a0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 --2- 172.21.15.19:0/3604352051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb3850 0x5623fffbb890 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 >> 172.21.15.19:0/3604352051 conn(0x5623fffafde0 msgr2=0x5623fffb5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe584ff9640 0 client.26909 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 shutdown_connections 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 1 -- 172.21.15.19:0/3604352051 wait complete. 2024-09-12T03:29:57.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.812+0000 7fe5a9706740 20 client.26909 trim_cache size 0 max 16384 2024-09-12T03:29:57.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 Processor -- start 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 -- start start 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffd60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffbea30 0x5623fffd6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01af880 con 0x5623fffb3850 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b1600 con 0x5623fffbea30 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.831+0000 7fe5a9706740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01aff50 con 0x5623fffb4170 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54700/0 (socket says 172.21.15.19:54700) 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 -- 172.21.15.19:0/1000950726 learned_addr learned my addr 172.21.15.19:0/1000950726 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:57.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffd60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffbea30 0x5623fffd6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 -- 172.21.15.19:0/1000950726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 msgr2=0x5623fffd60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffd60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 -- 172.21.15.19:0/1000950726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffbea30 msgr2=0x5623fffd6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffbea30 0x5623fffd6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 -- 172.21.15.19:0/1000950726 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623fffab3f0 con 0x5623fffb3850 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0095f0 tx=0x7fe59c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12b63464a5a36be server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe586ffd640 1 -- 172.21.15.19:0/1000950726 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c029040 con 0x5623fffb3850 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe586ffd640 1 -- 172.21.15.19:0/1000950726 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c009a80 con 0x5623fffb3850 2024-09-12T03:29:57.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.832+0000 7fe586ffd640 1 -- 172.21.15.19:0/1000950726 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c0038e0 con 0x5623fffb3850 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 -- 172.21.15.19:0/1000950726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 msgr2=0x5623fffba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe59c0095f0 tx=0x7fe59c009620 comp rx=0 tx=0).stop 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 -- 172.21.15.19:0/1000950726 shutdown_connections 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffbea30 0x5623fffd6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffb4170 0x5623fffd60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 --2- 172.21.15.19:0/1000950726 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffba740 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 -- 172.21.15.19:0/1000950726 >> 172.21.15.19:0/1000950726 conn(0x5623fffafde0 msgr2=0x5623fffba030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 -- 172.21.15.19:0/1000950726 shutdown_connections 2024-09-12T03:29:57.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.833+0000 7fe5a9706740 1 -- 172.21.15.19:0/1000950726 wait complete. 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 -- start start 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 Processor -- start 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb4170 0x5623fffc26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffc66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b0e70 con 0x5623fffb3850 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01b06e0 con 0x5623fffb4170 2024-09-12T03:29:57.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a01ad7a0 con 0x5623fffbea30 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a9706740 10 client.0 _ll_register_callbacks cb 0x7fe5980093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a50c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54706/0 (socket says 172.21.15.19:54706) 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a50c9640 1 -- 172.21.15.19:0/886672295 learned_addr learned my addr 172.21.15.19:0/886672295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffc66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.834+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb4170 0x5623fffc26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 -- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 msgr2=0x5623fffc66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffc66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 -- 172.21.15.19:0/886672295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb4170 msgr2=0x5623fffc26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb4170 0x5623fffc26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 -- 172.21.15.19:0/886672295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623fffc6d90 con 0x5623fffb3850 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a58ca640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffc66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:57.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a50c9640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe59c03a040 tx=0x7fe59c0126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab78b5055588e12b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe584ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c029040 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe59c0035f0 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c009a80 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fe59c009810 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a9706740 10 client.17526 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x562400038ad0 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.835+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe59c009a80 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fe59c04a070 con 0x5623fffb3850 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 10 client.17526 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 10 client.17526 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:29:57.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 20 client.17526 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562400039150 0x56240003b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 4 client.17526 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x5623fff1fbe0 con 0x562400039150 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562400039150 0x56240003b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe5a48c8640 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562400039150 0x56240003b570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d344d9fd4d7724fa server_cookie=1dd3c5eefd80e455 in_seq=0 out_seq=0 2024-09-12T03:29:57.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:57.836+0000 7fe584ff9640 10 client.17526 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:29:57.966 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 -- 172.21.15.19:0/4268147675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec101280 msgr2=0x7fa0ec10df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/4268147675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec101280 0x7fa0ec10df40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009900 tx=0x7fa0dc02a410 comp rx=0 tx=0).stop 2024-09-12T03:29:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 -- 172.21.15.19:0/4268147675 shutdown_connections 2024-09-12T03:29:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/4268147675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec10e480 0x7fa0ec110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/4268147675 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec101280 0x7fa0ec10df40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/4268147675 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 -- 172.21.15.19:0/4268147675 >> 172.21.15.19:0/4268147675 conn(0x7fa0ec0fc7a0 msgr2=0x7fa0ec0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 -- 172.21.15.19:0/4268147675 shutdown_connections 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.554+0000 7fa0f37f4640 1 -- 172.21.15.19:0/4268147675 wait complete. 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 Processor -- start 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 -- start start 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec101280 0x7fa0ec19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec114190 con 0x7fa0ec10e480 2024-09-12T03:29:58.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec114010 con 0x7fa0ec1008d0 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f37f4640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec114310 con 0x7fa0ec101280 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f1590640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54726/0 (socket says 172.21.15.19:54726) 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f1d91640 1 -- 172.21.15.19:0/3983839252 learned_addr learned my addr 172.21.15.19:0/3983839252 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.555+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec101280 0x7fa0ec19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 -- 172.21.15.19:0/3983839252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec101280 msgr2=0x7fa0ec19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec101280 0x7fa0ec19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 -- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 msgr2=0x7fa0ec19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 -- 172.21.15.19:0/3983839252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ec19f640 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f1590640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec19a690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8005fd0 tx=0x7fa0e800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0e27fc640 1 -- 172.21.15.19:0/3983839252 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8009c30 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ec19f930 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0e27fc640 1 -- 172.21.15.19:0/3983839252 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e8009dd0 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0e27fc640 1 -- 172.21.15.19:0/3983839252 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8010460 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.556+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ec1a7210 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 Processor -- start 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 -- start start 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0e27fc640 1 -- 172.21.15.19:0/3983839252 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa0e8003f50 con 0x7fa0ec1008d0 2024-09-12T03:29:58.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a774975d0 0x564a77497a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a773fd2f0 con 0x564a774a2370 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a773a4cf0 con 0x564a77498b40 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a773f92b0 con 0x564a774975d0 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f0d8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f0d8f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54742/0 (socket says 172.21.15.19:54742) 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f0d8f640 1 -- 172.21.15.19:0/2425574145 learned_addr learned my addr 172.21.15.19:0/2425574145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:58.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53204/0 (socket says 172.21.15.19:53204) 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.559+0000 7fa0f1590640 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a774975d0 0x564a77497a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 -- 172.21.15.19:0/2425574145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a774975d0 msgr2=0x564a77497a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a774975d0 0x564a77497a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 -- 172.21.15.19:0/2425574145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 msgr2=0x564a774a4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 -- 172.21.15.19:0/2425574145 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a773893a0 con 0x564a77498b40 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0e27fc640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 0x7fa0d80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011760 tx=0x7fa0e4011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa5b6cf607666d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0f1590640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 0x7fa0d80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0d77fe640 1 -- 172.21.15.19:0/2425574145 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4004d20 con 0x564a77498b40 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0d77fe640 1 -- 172.21.15.19:0/2425574145 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4005020 con 0x564a77498b40 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0d77fe640 1 -- 172.21.15.19:0/2425574145 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4019690 con 0x564a77498b40 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.560+0000 7fa0e27fc640 1 -- 172.21.15.19:0/3983839252 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa0e80ceec0 con 0x7fa0ec1008d0 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/2425574145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 msgr2=0x564a77497090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f1590640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 0x7fa0d80a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ccd40 tx=0x7fa0e80d29a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4011760 tx=0x7fa0e4011c20 comp rx=0 tx=0).stop 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/2425574145 shutdown_connections 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774a2370 0x564a774a4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a774975d0 0x564a77497a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/2425574145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a77498b40 0x564a77497090 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/2425574145 >> 172.21.15.19:0/2425574145 conn(0x564a77460310 msgr2=0x564a77493c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/2425574145 shutdown_connections 2024-09-12T03:29:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/2425574145 wait complete. 2024-09-12T03:29:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 -- start start 2024-09-12T03:29:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.561+0000 7fa0f5bcd740 1 Processor -- start 2024-09-12T03:29:58.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564a774a5070 con 0x564a774975d0 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564a773b8060 con 0x564a774a2370 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564a774a51f0 con 0x564a77498b40 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f5bcd740 10 client.0 _ll_register_callbacks cb 0x564a7710b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54752/0 (socket says 172.21.15.19:54752) 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 -- 172.21.15.19:0/110445059 learned_addr learned my addr 172.21.15.19:0/110445059 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:58.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 -- 172.21.15.19:0/110445059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 msgr2=0x564a7749ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.582 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 -- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 msgr2=0x564a7749f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 -- 172.21.15.19:0/110445059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7716e3c0 con 0x564a774975d0 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005980 tx=0x7fa0e40096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a12025ea06d63fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.562+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400fa00 con 0x564a774975d0 2024-09-12T03:29:58.583 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa0e400fba0 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e4005020 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0e400fda0 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 10 client.17538 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564a774b0a60 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4005020 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fa0e4025040 con 0x564a774975d0 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 10 client.17538 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:58.584 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 10 client.17538 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 20 client.17538 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a774b1e70 0x564a774b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 4 client.17538 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a7740a860 con 0x564a774b1e70 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.563+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a774b1e70 0x564a774b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.564+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a774b1e70 0x564a774b4260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f9c4b27dda35e4c2 server_cookie=86f448cd03e0483b in_seq=0 out_seq=0 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.564+0000 7fa0d4ff9640 10 client.17538 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:58.585 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.564+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/110445059 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564a7740a860 con 0x564a774b1e70 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.564+0000 7fa0d4ff9640 10 client.17538 handle_command_reply: tid=0 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.564+0000 7fa0f5bcd740 1 client.17538 shutdown 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 msgr2=0x564a774a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005980 tx=0x7fa0e40096c0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 shutdown_connections 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a774b1e70 0x564a774b4260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0d4ff9640 0 client.17538 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/110445059 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774a1ca0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.586 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 >> 172.21.15.19:0/110445059 conn(0x564a77460310 msgr2=0x564a77499960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 shutdown_connections 2024-09-12T03:29:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/110445059 wait complete. 2024-09-12T03:29:58.587 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.565+0000 7fa0f5bcd740 20 client.17538 trim_cache size 0 max 16384 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.578+0000 7fa0f5bcd740 1 Processor -- start 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.578+0000 7fa0f5bcd740 1 -- start start 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1af890 con 0x564a774975d0 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b1610 con 0x564a774a2370 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1aff60 con 0x564a77498b40 2024-09-12T03:29:58.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54762/0 (socket says 172.21.15.19:54762) 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 -- 172.21.15.19:0/3511005054 learned_addr learned my addr 172.21.15.19:0/3511005054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 -- 172.21.15.19:0/3511005054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 msgr2=0x564a7749a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 -- 172.21.15.19:0/3511005054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 msgr2=0x564a7749a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 -- 172.21.15.19:0/3511005054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a774b47f0 con 0x564a774975d0 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005d70 tx=0x7fa0e40111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b58f7d1ce8a3189 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.579+0000 7fa0d6ffd640 1 -- 172.21.15.19:0/3511005054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4011430 con 0x564a774975d0 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0d6ffd640 1 -- 172.21.15.19:0/3511005054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e400f790 con 0x564a774975d0 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0d6ffd640 1 -- 172.21.15.19:0/3511005054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400fa00 con 0x564a774975d0 2024-09-12T03:29:58.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/3511005054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 msgr2=0x564a774ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4005d70 tx=0x7fa0e40111d0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/3511005054 shutdown_connections 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a7749a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a7749a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/3511005054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a774ae310 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.580+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/3511005054 >> 172.21.15.19:0/3511005054 conn(0x564a77460310 msgr2=0x564a77492d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.581+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/3511005054 shutdown_connections 2024-09-12T03:29:58.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.581+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/3511005054 wait complete. 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.581+0000 7fa0f5bcd740 1 -- start start 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.581+0000 7fa0f5bcd740 1 Processor -- start 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a7749cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a774aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b0e80 con 0x564a774975d0 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1b06f0 con 0x564a774a2370 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ec1ad7b0 con 0x564a77498b40 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.582+0000 7fa0f5bcd740 10 client.0 _ll_register_callbacks cb 0x7fa0e4004de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:29:58.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.583+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.583+0000 7fa0f1d91640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53244/0 (socket says 172.21.15.19:53244) 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.583+0000 7fa0f1d91640 1 -- 172.21.15.19:0/72238665 learned_addr learned my addr 172.21.15.19:0/72238665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.583+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a7749cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.589+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a774aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.590+0000 7fa0f1d91640 1 -- 172.21.15.19:0/72238665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 msgr2=0x564a774aa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.590+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a774aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.590+0000 7fa0f1d91640 1 -- 172.21.15.19:0/72238665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 msgr2=0x564a7749cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.590+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a7749cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.590+0000 7fa0f1d91640 1 -- 172.21.15.19:0/72238665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a7749b2c0 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.593+0000 7fa0f1d91640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009230 tx=0x7fa0e4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a07c9b482855dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.594+0000 7fa0d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.594+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400fc00 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.594+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fa0e4028980 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.594+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0e400f790 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.594+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa0e4028b80 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.595+0000 7fa0f5bcd740 10 client.26933 fetch_fsmap learned FSMap version 58 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.595+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564a7751bde0 con 0x564a774a2370 2024-09-12T03:29:58.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.595+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e400f790 con 0x564a774a2370 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.595+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fa0e4032af0 con 0x564a774a2370 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 10 client.26933 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 10 client.26933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 20 client.26933 populate_metadata read hostname 'smithi019' 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a7751c280 0x564a7751e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 4 client.26933 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a77404d70 con 0x564a7751c280 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.596+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a7751c280 0x564a7751e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0d4ff9640 10 client.26933 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f0d8f640 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a7751c280 0x564a7751e6a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6974d8e7c6c6ddd5 server_cookie=a9022cd17792e39c in_seq=0 out_seq=0 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0d4ff9640 1 -- 172.21.15.19:0/72238665 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a77404d70 con 0x564a7751c280 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0d4ff9640 10 client.26933 handle_command_reply: tid=0 2024-09-12T03:29:58.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 client.26933 shutdown 2024-09-12T03:29:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 msgr2=0x564a774aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4009230 tx=0x7fa0e4005720 comp rx=0 tx=0).stop 2024-09-12T03:29:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 shutdown_connections 2024-09-12T03:29:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x564a7751c280 0x564a7751e6a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0d4ff9640 0 client.26933 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:29:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564a774a2370 0x564a774aac10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564a77498b40 0x564a774aa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.603 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 --2- 172.21.15.19:0/72238665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564a774975d0 0x564a7749cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.598+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 >> 172.21.15.19:0/72238665 conn(0x564a77460310 msgr2=0x564a77492500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.599+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 shutdown_connections 2024-09-12T03:29:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.599+0000 7fa0f5bcd740 1 -- 172.21.15.19:0/72238665 wait complete. 2024-09-12T03:29:58.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.599+0000 7fa0f5bcd740 20 client.26933 trim_cache size 0 max 16384 2024-09-12T03:29:58.604 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 msgr2=0x7fa0d80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 0x7fa0d80a5040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80ccd40 tx=0x7fa0e80d29a0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 msgr2=0x7fa0ec19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec19a690 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8005fd0 tx=0x7fa0e800dde0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 shutdown_connections 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa0d80a2b80 0x7fa0d80a5040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa0ec10e480 0x7fa0ec19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa0ec101280 0x7fa0ec19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 --2- 172.21.15.19:0/3983839252 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0ec1008d0 0x7fa0ec19a690 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:29:58.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 >> 172.21.15.19:0/3983839252 conn(0x7fa0ec0fc7a0 msgr2=0x7fa0ec0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:29:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 shutdown_connections 2024-09-12T03:29:58.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:29:58.600+0000 7fa0f37f4640 1 -- 172.21.15.19:0/3983839252 wait complete. 2024-09-12T03:29:59.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:29:59.561 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:29:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:29:59 smithi121 ceph-mon[34963]: pgmap v665: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-09-12T03:29:59.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:29:59 smithi019 ceph-mon[30953]: pgmap v665: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-09-12T03:29:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:29:59 smithi135 ceph-mon[35474]: pgmap v665: 97 pgs: 97 active+clean; 1.9 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-09-12T03:30:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 -- 172.21.15.19:0/2305484388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 msgr2=0x7f6b0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 --2- 172.21.15.19:0/2305484388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c10ec40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009930 tx=0x7f6b0402a420 comp rx=0 tx=0).stop 2024-09-12T03:30:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 -- 172.21.15.19:0/2305484388 shutdown_connections 2024-09-12T03:30:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 --2- 172.21.15.19:0/2305484388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c10ec40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 --2- 172.21.15.19:0/2305484388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 --2- 172.21.15.19:0/2305484388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 0x7f6b0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 -- 172.21.15.19:0/2305484388 >> 172.21.15.19:0/2305484388 conn(0x7f6b0c0fc7d0 msgr2=0x7f6b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 -- 172.21.15.19:0/2305484388 shutdown_connections 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.050+0000 7f6b10bae640 1 -- 172.21.15.19:0/2305484388 wait complete. 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 Processor -- start 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 -- start start 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 0x7f6b0c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1120a0 con 0x7f6b0c1082a0 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c111f20 con 0x7f6b0c102d30 2024-09-12T03:30:00.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b10bae640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c112220 con 0x7f6b0c1036e0 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 0x7f6b0c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54782/0 (socket says 172.21.15.19:54782) 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 -- 172.21.15.19:0/3791417653 learned_addr learned my addr 172.21.15.19:0/3791417653 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 -- 172.21.15.19:0/3791417653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 msgr2=0x7f6b0c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 -- 172.21.15.19:0/3791417653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 msgr2=0x7f6b0c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 0x7f6b0c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b0ad9d640 1 -- 172.21.15.19:0/3791417653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c19d510 con 0x7f6b0c1082a0 2024-09-12T03:30:00.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.051+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6b0ad9d640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005ab0 tx=0x7f6b04031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6afb7fe640 1 -- 172.21.15.19:0/3791417653 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0403f030 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6afb7fe640 1 -- 172.21.15.19:0/3791417653 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b040043d0 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c19d7a0 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6afb7fe640 1 -- 172.21.15.19:0/3791417653 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04035070 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.052+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c1a50e0 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6afb7fe640 1 -- 172.21.15.19:0/3791417653 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b04042030 con 0x7f6b0c1082a0 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 Processor -- start 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 -- start start 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9f96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fa3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9efc2f0 con 0x55bbc9fa1370 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9ea3cf0 con 0x55bbc9f97b40 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b12f87740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9ef82b0 con 0x55bbc9f965d0 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fa3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:59230/0 (socket says 172.21.15.19:59230) 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b0a59c640 1 -- 172.21.15.19:0/868723221 learned_addr learned my addr 172.21.15.19:0/868723221 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.055+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9f96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 -- 172.21.15.19:0/868723221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 msgr2=0x55bbc9f96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 -- 172.21.15.19:0/868723221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 msgr2=0x55bbc9f96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6afb7fe640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 0x7f6af40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9f96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 -- 172.21.15.19:0/868723221 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc9e883a0 con 0x55bbc9fa1370 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b0ad9d640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 0x7f6af40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fa3760 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009710 tx=0x7f6b000096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b14de3e41c40010 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6afb7fe640 1 -- 172.21.15.19:0/3791417653 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6b040ef260 con 0x7f6b0c1082a0 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6af8ff9640 1 -- 172.21.15.19:0/868723221 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00004050 con 0x55bbc9fa1370 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6af8ff9640 1 -- 172.21.15.19:0/868723221 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0000fdf0 con 0x55bbc9fa1370 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.056+0000 7f6af8ff9640 1 -- 172.21.15.19:0/868723221 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00005580 con 0x55bbc9fa1370 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- 172.21.15.19:0/868723221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 msgr2=0x55bbc9fa3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b0ad9d640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 0x7f6af40a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b040b8550 tx=0x7f6b0408ba30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fa3760 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6b00009710 tx=0x7f6b000096a0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- 172.21.15.19:0/868723221 shutdown_connections 2024-09-12T03:30:00.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fa3760 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f965d0 0x55bbc9f96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 --2- 172.21.15.19:0/868723221 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9f96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- 172.21.15.19:0/868723221 >> 172.21.15.19:0/868723221 conn(0x55bbc9f5f310 msgr2=0x55bbc9f92c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- 172.21.15.19:0/868723221 shutdown_connections 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- 172.21.15.19:0/868723221 wait complete. 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.057+0000 7f6b12f87740 1 -- start start 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 Processor -- start 2024-09-12T03:30:00.073 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f9e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9fa4070 con 0x55bbc9fa1370 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9eb7060 con 0x55bbc9f965d0 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bbc9fa41f0 con 0x55bbc9f97b40 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b12f87740 10 client.0 _ll_register_callbacks cb 0x55bbc9c0a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.074 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54790/0 (socket says 172.21.15.19:54790) 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f9e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b09d9b640 1 -- 172.21.15.19:0/124792548 learned_addr learned my addr 172.21.15.19:0/124792548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 -- 172.21.15.19:0/124792548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 msgr2=0x55bbc9fa0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 -- 172.21.15.19:0/124792548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 msgr2=0x55bbc9f9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b0a59c640 1 -- 172.21.15.19:0/124792548 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbc9c6d3c0 con 0x55bbc9f97b40 2024-09-12T03:30:00.075 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.058+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f9e060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6afc011cc0 tx=0x7f6afc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b090a3c50d9f650 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc004b20 con 0x55bbc9f97b40 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0000fdf0 con 0x55bbc9f97b40 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6afc004cc0 con 0x55bbc9f97b40 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6afc003970 con 0x55bbc9f97b40 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 10 client.26908 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bbc9fafb60 con 0x55bbc9f97b40 2024-09-12T03:30:00.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b0000fdf0 con 0x55bbc9f97b40 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6afc025450 con 0x55bbc9f97b40 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 10 client.26908 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 20 client.26908 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbc9fb1060 0x55bbc9fb3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 4 client.26908 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.059+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbc9f09860 con 0x55bbc9fb1060 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.060+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbc9fb1060 0x55bbc9fb3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.060+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbc9fb1060 0x55bbc9fb3480 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b22445f90e4d68e5 server_cookie=33c756bbf901fec1 in_seq=0 out_seq=0 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.060+0000 7f6af1ffb640 10 client.26908 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:00.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6af1ffb640 1 -- 172.21.15.19:0/124792548 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbc9f09860 con 0x55bbc9fb1060 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6af1ffb640 10 client.26908 handle_command_reply: tid=0 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 client.26908 shutdown 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 msgr2=0x55bbc9f9e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f9e060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6afc011cc0 tx=0x7f6afc00f790 comp rx=0 tx=0).stop 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 shutdown_connections 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9f9e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f9e060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbc9fb1060 0x55bbc9fb3480 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 --2- 172.21.15.19:0/124792548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6af1ffb640 0 client.26908 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 >> 172.21.15.19:0/124792548 conn(0x55bbc9f5f310 msgr2=0x55bbc9f98960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 shutdown_connections 2024-09-12T03:30:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 1 -- 172.21.15.19:0/124792548 wait complete. 2024-09-12T03:30:00.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.061+0000 7f6b12f87740 20 client.26908 trim_cache size 0 max 16384 2024-09-12T03:30:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 Processor -- start 2024-09-12T03:30:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 -- start start 2024-09-12T03:30:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9f9e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.085 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1ad640 con 0x55bbc9fa1370 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1af3c0 con 0x55bbc9f965d0 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b12f87740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1add10 con 0x55bbc9f97b40 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b0a59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:59248/0 (socket says 172.21.15.19:59248) 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9f9e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.074+0000 7f6b0a59c640 1 -- 172.21.15.19:0/3995618043 learned_addr learned my addr 172.21.15.19:0/3995618043 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 -- 172.21.15.19:0/3995618043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 msgr2=0x55bbc9f9e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9f9e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 -- 172.21.15.19:0/3995618043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 msgr2=0x55bbc9fb8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fb8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 -- 172.21.15.19:0/3995618043 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbc9f99120 con 0x55bbc9f97b40 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6afc025a60 tx=0x7f6afc005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9d15c5258edb45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6af3fff640 1 -- 172.21.15.19:0/3995618043 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc025450 con 0x55bbc9f97b40 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6af3fff640 1 -- 172.21.15.19:0/3995618043 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6afc0255f0 con 0x55bbc9f97b40 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.075+0000 7f6af3fff640 1 -- 172.21.15.19:0/3995618043 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01ac00 con 0x55bbc9f97b40 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- 172.21.15.19:0/3995618043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 msgr2=0x55bbc9f97300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6afc025a60 tx=0x7f6afc005d30 comp rx=0 tx=0).stop 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- 172.21.15.19:0/3995618043 shutdown_connections 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9fa1370 0x55bbc9fb8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9f97b40 0x55bbc9f97300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- 172.21.15.19:0/3995618043 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f965d0 0x55bbc9f9e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- 172.21.15.19:0/3995618043 >> 172.21.15.19:0/3995618043 conn(0x55bbc9f5f310 msgr2=0x55bbc9f9d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- 172.21.15.19:0/3995618043 shutdown_connections 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- 172.21.15.19:0/3995618043 wait complete. 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- start start 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 Processor -- start 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9fa4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9fa1370 0x55bbca019d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1aec30 con 0x55bbc9f965d0 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1ae4a0 con 0x55bbc9f97b40 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c1ab560 con 0x55bbc9fa1370 2024-09-12T03:30:00.088 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.076+0000 7f6b12f87740 10 client.0 _ll_register_callbacks cb 0x7f6afc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:54808/0 (socket says 172.21.15.19:54808) 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 -- 172.21.15.19:0/1512817521 learned_addr learned my addr 172.21.15.19:0/1512817521 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9fa4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9fa1370 0x55bbca019d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 -- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9fa1370 msgr2=0x55bbca019d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9fa1370 0x55bbca019d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 -- 172.21.15.19:0/1512817521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 msgr2=0x55bbc9fa4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9fa4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 -- 172.21.15.19:0/1512817521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbca01a250 con 0x55bbc9f965d0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b09d9b640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6b000058c0 tx=0x7f6b000096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18538820c6ce9725 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:00.089 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6af1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b00033040 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6b0000fc70 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.077+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6afc0255f0 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6b00043050 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 10 client.17562 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bbca01b460 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6afc0255f0 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6b00039040 con 0x55bbc9f965d0 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 10 client.17562 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 20 client.17562 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:00.090 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbca01ba50 0x55bbca01de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 4 client.17562 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbc9f03d70 con 0x55bbca01ba50 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.078+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbca01ba50 0x55bbca01de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6af1ffb640 10 client.17562 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b0a59c640 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbca01ba50 0x55bbca01de70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c6ff34e81dfec33 server_cookie=1a7a502bbe620d88 in_seq=0 out_seq=0 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6af1ffb640 1 -- 172.21.15.19:0/1512817521 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbc9f03d70 con 0x55bbca01ba50 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6af1ffb640 10 client.17562 handle_command_reply: tid=0 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 client.17562 shutdown 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 msgr2=0x55bbc9fa49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6b000058c0 tx=0x7f6b000096a0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 shutdown_connections 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bbc9fa1370 0x55bbca019d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.091 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bbca01ba50 0x55bbca01de70 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6af1ffb640 0 client.17562 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bbc9f97b40 0x55bbc9fa4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 --2- 172.21.15.19:0/1512817521 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bbc9f965d0 0x55bbc9fa49e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 >> 172.21.15.19:0/1512817521 conn(0x55bbc9f5f310 msgr2=0x55bbc9f98930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 shutdown_connections 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 1 -- 172.21.15.19:0/1512817521 wait complete. 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.079+0000 7f6b12f87740 20 client.17562 trim_cache size 0 max 16384 2024-09-12T03:30:00.092 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 msgr2=0x7f6af40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 0x7f6af40a50c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6b040b8550 tx=0x7f6b0408ba30 comp rx=0 tx=0).stop 2024-09-12T03:30:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 msgr2=0x7f6b0c19ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6b04005ab0 tx=0x7f6b04031b50 comp rx=0 tx=0).stop 2024-09-12T03:30:00.093 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 shutdown_connections 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6af40a2c00 0x7f6af40a50c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c19ce30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 --2- 172.21.15.19:0/3791417653 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6b0c102d30 0x7f6b0c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 >> 172.21.15.19:0/3791417653 conn(0x7f6b0c0fc7d0 msgr2=0x7f6b0c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 shutdown_connections 2024-09-12T03:30:00.094 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:00.080+0000 7f6b10bae640 1 -- 172.21.15.19:0/3791417653 wait complete. 2024-09-12T03:30:00.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:00 smithi121 ceph-mon[34963]: overall HEALTH_OK 2024-09-12T03:30:00.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:00 smithi019 ceph-mon[30953]: overall HEALTH_OK 2024-09-12T03:30:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:00 smithi135 ceph-mon[35474]: overall HEALTH_OK 2024-09-12T03:30:01.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:01 smithi121 ceph-mon[34963]: pgmap v666: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:30:01.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:01 smithi019 ceph-mon[30953]: pgmap v666: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:30:01.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:01 smithi135 ceph-mon[35474]: pgmap v666: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 409 op/s 2024-09-12T03:30:01.861 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:30:01.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:30:02.133 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:02.652 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 -- 172.21.15.19:0/2797278819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc102790 msgr2=0x7f17fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 --2- 172.21.15.19:0/2797278819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc102790 0x7f17fc1067e0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009860 tx=0x7f17e402a340 comp rx=0 tx=0).stop 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 -- 172.21.15.19:0/2797278819 shutdown_connections 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 --2- 172.21.15.19:0/2797278819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc106f10 0x7f17fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 --2- 172.21.15.19:0/2797278819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc102790 0x7f17fc1067e0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 --2- 172.21.15.19:0/2797278819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc101de0 0x7f17fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 -- 172.21.15.19:0/2797278819 >> 172.21.15.19:0/2797278819 conn(0x7f17fc0fd8c0 msgr2=0x7f17fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 -- 172.21.15.19:0/2797278819 shutdown_connections 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.640+0000 7f18026a2640 1 -- 172.21.15.19:0/2797278819 wait complete. 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 Processor -- start 2024-09-12T03:30:02.653 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 -- start start 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc101de0 0x7f17fc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 0x7f17fc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111f30 con 0x7f17fc106f10 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc111db0 con 0x7f17fc102790 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f18026a2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1120b0 con 0x7f17fc101de0 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60802/0 (socket says 172.21.15.19:60802) 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f1800c3f640 1 -- 172.21.15.19:0/3696597870 learned_addr learned my addr 172.21.15.19:0/3696597870 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 0x7f17fc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.654 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.641+0000 7f17fbfff640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc101de0 0x7f17fc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 -- 172.21.15.19:0/3696597870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc101de0 msgr2=0x7f17fc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc101de0 0x7f17fc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 -- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 msgr2=0x7f17fc19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 -- 172.21.15.19:0/3696597870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc19d4c0 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f1800c3f640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 0x7f17fc198a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17e4035040 tx=0x7f17e4033b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f9ffb640 1 -- 172.21.15.19:0/3696597870 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e403f020 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17fc19d750 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f9ffb640 1 -- 172.21.15.19:0/3696597870 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4030550 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f17f9ffb640 1 -- 172.21.15.19:0/3696597870 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4037040 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.642+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17fc1a5090 con 0x7f17fc102790 2024-09-12T03:30:02.655 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.644+0000 7f17f9ffb640 1 -- 172.21.15.19:0/3696597870 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17e4042020 con 0x7f17fc102790 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 Processor -- start 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 -- start start 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 0x5610d5a70f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 0x5610d5a7b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610d59d6280 con 0x5610d5a7bd40 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5610d597db10 con 0x5610d5a71480 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1804a7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d59d2240 con 0x5610d5a70b60 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 0x5610d5a70f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 0x5610d5a7b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.656 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60808/0 (socket says 172.21.15.19:60808) 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 0x5610d5a70f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47290/0 (socket says 172.21.15.19:47290) 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 -- 172.21.15.19:0/3142173842 learned_addr learned my addr 172.21.15.19:0/3142173842 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f9ffb640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 0x7f17e80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f1800c3f640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 0x7f17e80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 -- 172.21.15.19:0/3142173842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 msgr2=0x5610d5a70f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 0x5610d5a70f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 -- 172.21.15.19:0/3142173842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 msgr2=0x5610d5a7b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 0x5610d5a7b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.645+0000 7f17f3fff640 1 -- 172.21.15.19:0/3142173842 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d598a1f0 con 0x5610d5a7bd40 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17fbfff640 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 0x5610d5a7b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17f9ffb640 1 -- 172.21.15.19:0/3696597870 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17e40ef100 con 0x7f17fc102790 2024-09-12T03:30:02.657 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17f3fff640 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f17e402a860 tx=0x7f17e4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ac4ca0ed08cdb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f1800c3f640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 0x7f17e80a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009330 tx=0x7f17f4004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17f27fc640 1 -- 172.21.15.19:0/3142173842 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40bc010 con 0x5610d5a7bd40 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17f27fc640 1 -- 172.21.15.19:0/3142173842 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e40bb910 con 0x5610d5a7bd40 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.646+0000 7f17f27fc640 1 -- 172.21.15.19:0/3142173842 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40b7f80 con 0x5610d5a7bd40 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- 172.21.15.19:0/3142173842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 msgr2=0x5610d5a7e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f17e402a860 tx=0x7f17e4005f90 comp rx=0 tx=0).stop 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- 172.21.15.19:0/3142173842 shutdown_connections 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a7e130 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a71480 0x5610d5a7b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 --2- 172.21.15.19:0/3142173842 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a70b60 0x5610d5a70f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- 172.21.15.19:0/3142173842 >> 172.21.15.19:0/3142173842 conn(0x5610d5a6d0f0 msgr2=0x5610d5a6d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- 172.21.15.19:0/3142173842 shutdown_connections 2024-09-12T03:30:02.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- 172.21.15.19:0/3142173842 wait complete. 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 -- start start 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.647+0000 7f1804a7b740 1 Processor -- start 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5610d5a7e6d0 con 0x5610d5a70b60 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5610d5990f10 con 0x5610d5a7bd40 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5610d5a7e850 con 0x5610d5a71480 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 10 client.0 _ll_register_callbacks cb 0x5610d5959a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60820/0 (socket says 172.21.15.19:60820) 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 -- 172.21.15.19:0/22332685 learned_addr learned my addr 172.21.15.19:0/22332685 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:02.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1800c3f640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17f3fff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 -- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 msgr2=0x5610d5a790c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a790c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 -- 172.21.15.19:0/22332685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 msgr2=0x5610d5a79600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a79600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 -- 172.21.15.19:0/22332685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d57476a0 con 0x5610d5a70b60 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17f3fff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17fbfff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f17ec012740 tx=0x7f17ec012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ea7261464d5920 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec004030 con 0x5610d5a70b60 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17ec0041d0 con 0x5610d5a70b60 2024-09-12T03:30:02.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.648+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e40bb910 con 0x5610d5a70b60 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f17ec0043d0 con 0x5610d5a70b60 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 10 client.17574 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5610d5a8a210 con 0x5610d5a70b60 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e40bb910 con 0x5610d5a70b60 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f17ec003c10 con 0x5610d5a70b60 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 10 client.17574 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 10 client.17574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 20 client.17574 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5a8dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 4 client.17574 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610d59e37f0 con 0x5610d5a8b770 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.649+0000 7f17f3fff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5a8dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f17f3fff640 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5a8dbf0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=109191df47b9dbc1 server_cookie=9a0c4974512642bc in_seq=0 out_seq=0 2024-09-12T03:30:02.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f17db7fe640 10 client.17574 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f17db7fe640 1 -- 172.21.15.19:0/22332685 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610d59e37f0 con 0x5610d5a8b770 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f17db7fe640 10 client.17574 handle_command_reply: tid=0 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 client.17574 shutdown 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 msgr2=0x5610d5a78b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f17ec012740 tx=0x7f17ec012c00 comp rx=0 tx=0).stop 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 shutdown_connections 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5a8dbf0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a79600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a790c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f17db7fe640 0 client.17574 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.650+0000 7f1804a7b740 1 --2- 172.21.15.19:0/22332685 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a78b80 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.651+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 >> 172.21.15.19:0/22332685 conn(0x5610d5a6d0f0 msgr2=0x5610d5a72e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.651+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 shutdown_connections 2024-09-12T03:30:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.651+0000 7f1804a7b740 1 -- 172.21.15.19:0/22332685 wait complete. 2024-09-12T03:30:02.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.651+0000 7f1804a7b740 20 client.17574 trim_cache size 0 max 16384 2024-09-12T03:30:02.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.668+0000 7f1804a7b740 1 Processor -- start 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.668+0000 7f1804a7b740 1 -- start start 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a70b60 0x5610d5a934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a6ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ad630 con 0x5610d5a7bd40 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1af3b0 con 0x5610d5a70b60 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1804a7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1add00 con 0x5610d5a71480 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a6ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:47316/0 (socket says 172.21.15.19:47316) 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f1800c3f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a6ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60822/0 (socket says 172.21.15.19:60822) 2024-09-12T03:30:02.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 -- 172.21.15.19:0/190808149 learned_addr learned my addr 172.21.15.19:0/190808149 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17fbfff640 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a70b60 0x5610d5a934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 -- 172.21.15.19:0/190808149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a70b60 msgr2=0x5610d5a934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a70b60 0x5610d5a934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 -- 172.21.15.19:0/190808149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 msgr2=0x5610d5a6ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a6ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 -- 172.21.15.19:0/190808149 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610d5a6cf20 con 0x5610d5a71480 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.669+0000 7f17f3fff640 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b7d70 tx=0x7f17e40bb550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6d37206cc7bf7df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f17f1ffb640 1 -- 172.21.15.19:0/190808149 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40bb910 con 0x5610d5a71480 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f17f1ffb640 1 -- 172.21.15.19:0/190808149 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4030700 con 0x5610d5a71480 2024-09-12T03:30:02.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f17f1ffb640 1 -- 172.21.15.19:0/190808149 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e40a6110 con 0x5610d5a71480 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 -- 172.21.15.19:0/190808149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 msgr2=0x5610d5a93a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17e40b7d70 tx=0x7f17e40bb550 comp rx=0 tx=0).stop 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 -- 172.21.15.19:0/190808149 shutdown_connections 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a7bd40 0x5610d5a6ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a93a00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 --2- 172.21.15.19:0/190808149 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a70b60 0x5610d5a934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 -- 172.21.15.19:0/190808149 >> 172.21.15.19:0/190808149 conn(0x5610d5a6d0f0 msgr2=0x5610d5a77c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 -- 172.21.15.19:0/190808149 shutdown_connections 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.670+0000 7f1804a7b740 1 -- 172.21.15.19:0/190808149 wait complete. 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 -- start start 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 Processor -- start 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a7f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a7f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1aec20 con 0x5610d5a70b60 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ae490 con 0x5610d5a7bd40 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc1ab550 con 0x5610d5a71480 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1804a7b740 10 client.0 _ll_register_callbacks cb 0x7f17ec009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f17fbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60824/0 (socket says 172.21.15.19:60824) 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f17fbfff640 1 -- 172.21.15.19:0/744166835 learned_addr learned my addr 172.21.15.19:0/744166835 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f17f3fff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a7f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.671+0000 7f1800c3f640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a7f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 -- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 msgr2=0x5610d5a7f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a7f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 -- 172.21.15.19:0/744166835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 msgr2=0x5610d5a7f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a7f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 -- 172.21.15.19:0/744166835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610d5a7ff80 con 0x5610d5a70b60 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17fbfff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0104b0 tx=0x7f17ec004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8bd6b07b4a2677 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00fb30 con 0x5610d5a70b60 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17ec00fcd0 con 0x5610d5a70b60 2024-09-12T03:30:02.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e4030700 con 0x5610d5a70b60 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f17ec010650 con 0x5610d5a70b60 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f1804a7b740 10 client.17580 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.672+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5610d5afa7c0 con 0x5610d5a70b60 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17e4030700 con 0x5610d5a70b60 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f17ec0039c0 con 0x5610d5a70b60 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 10 client.17580 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 20 client.17580 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5afd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 4 client.17580 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5610d59ddd00 con 0x5610d5a8b770 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f17f3fff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5afd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:02.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f17db7fe640 10 client.17580 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.673+0000 7f17f3fff640 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5afd030 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73600b94ffb256a9 server_cookie=ef52b68c8159859 in_seq=0 out_seq=0 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f17db7fe640 1 -- 172.21.15.19:0/744166835 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5610d59ddd00 con 0x5610d5a8b770 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f17db7fe640 10 client.17580 handle_command_reply: tid=0 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f1804a7b740 1 client.17580 shutdown 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 msgr2=0x5610d5a84020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f17ec0104b0 tx=0x7f17ec004230 comp rx=0 tx=0).stop 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 shutdown_connections 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.674+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5610d5a8b770 0x5610d5afd030 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5610d5a7bd40 0x5610d5a7f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5610d5a71480 0x5610d5a7f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 --2- 172.21.15.19:0/744166835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5610d5a70b60 0x5610d5a84020 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 >> 172.21.15.19:0/744166835 conn(0x5610d5a6d0f0 msgr2=0x5610d5a6ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f17db7fe640 0 client.17580 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 shutdown_connections 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 1 -- 172.21.15.19:0/744166835 wait complete. 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f1804a7b740 20 client.17580 trim_cache size 0 max 16384 2024-09-12T03:30:02.680 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "08d931e1-6aa8-458a-9fb1-bf0626396470", 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 msgr2=0x7f17e80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 0x7f17e80a5040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f17f4009330 tx=0x7f17f4004ab0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 msgr2=0x7f17fc198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 0x7f17fc198a50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f17e4035040 tx=0x7f17e4033b60 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 shutdown_connections 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17e80a2b80 0x7f17e80a5040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17fc106f10 0x7f17fc19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17fc102790 0x7f17fc198a50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 --2- 172.21.15.19:0/3696597870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17fc101de0 0x7f17fc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 >> 172.21.15.19:0/3696597870 conn(0x7f17fc0fd8c0 msgr2=0x7f17fc0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 shutdown_connections 2024-09-12T03:30:02.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:02.675+0000 7f18026a2640 1 -- 172.21.15.19:0/3696597870 wait complete. 2024-09-12T03:30:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:03 smithi121 ceph-mon[34963]: pgmap v667: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-09-12T03:30:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:03 smithi019 ceph-mon[30953]: pgmap v667: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-09-12T03:30:03.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:30:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:03 smithi135 ceph-mon[35474]: pgmap v667: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 443 op/s 2024-09-12T03:30:03.995 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:04.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:04 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:30:04.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:04 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:04.405 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:04 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 -- 172.21.15.19:0/1398720291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370101de0 msgr2=0x7f43701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 --2- 172.21.15.19:0/1398720291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370101de0 0x7f43701021c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f436002a410 comp rx=0 tx=0).stop 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 -- 172.21.15.19:0/1398720291 shutdown_connections 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 --2- 172.21.15.19:0/1398720291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370106f10 0x7f437010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 --2- 172.21.15.19:0/1398720291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370102790 0x7f43701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 --2- 172.21.15.19:0/1398720291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370101de0 0x7f43701021c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 -- 172.21.15.19:0/1398720291 >> 172.21.15.19:0/1398720291 conn(0x7f43700fd8c0 msgr2=0x7f43700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 -- 172.21.15.19:0/1398720291 shutdown_connections 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 -- 172.21.15.19:0/1398720291 wait complete. 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.488+0000 7f43760ca640 1 Processor -- start 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 -- start start 2024-09-12T03:30:04.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370101de0 0x7f43700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 0x7f4370074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f4370111f70 con 0x7f4370106f10 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4370111df0 con 0x7f4370101de0 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f43760ca640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43701120f0 con 0x7f4370102790 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60834/0 (socket says 172.21.15.19:60834) 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436ffff640 1 -- 172.21.15.19:0/4026466703 learned_addr learned my addr 172.21.15.19:0/4026466703 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 0x7f4370074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436f7fe640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370101de0 0x7f43700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 -- 172.21.15.19:0/4026466703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370101de0 msgr2=0x7f43700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370101de0 0x7f43700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 -- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 msgr2=0x7f4370074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436effd640 1 -- 172.21.15.19:0/4026466703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43701a6990 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.489+0000 7f436ffff640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f436effd640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 0x7f4370074a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f436400d6d0 tx=0x7f436400db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f436cff9640 1 -- 172.21.15.19:0/4026466703 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364017070 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701a6c80 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f436cff9640 1 -- 172.21.15.19:0/4026466703 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4364009820 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f436cff9640 1 -- 172.21.15.19:0/4026466703 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364004cd0 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.490+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43701a71c0 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.492+0000 7f436cff9640 1 -- 172.21.15.19:0/4026466703 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f43640099d0 con 0x7f4370102790 2024-09-12T03:30:04.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.492+0000 7f43784a3740 1 Processor -- start 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.492+0000 7f43784a3740 1 -- start start 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf74a0 0x55bc0ecf7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed04a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ec5d2e0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ec04ce0 con 0x55bc0ecf74a0 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f43784a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ec592a0 con 0x55bc0ed02680 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60850/0 (socket says 172.21.15.19:60850) 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 -- 172.21.15.19:0/4213648809 learned_addr learned my addr 172.21.15.19:0/4213648809 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:04.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436ffff640 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf74a0 0x55bc0ecf7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436effd640 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed04a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 -- 172.21.15.19:0/4213648809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 msgr2=0x55bc0ed04a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed04a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 -- 172.21.15.19:0/4213648809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf74a0 msgr2=0x55bc0ecf7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf74a0 0x55bc0ecf7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 -- 172.21.15.19:0/4213648809 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0ebe9440 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436f7fe640 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f4360005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ed285d9037ebef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f436cff9640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 0x7f43580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.493+0000 7f4355ffb640 1 -- 172.21.15.19:0/4213648809 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360040070 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f4355ffb640 1 -- 172.21.15.19:0/4213648809 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360031830 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f436ffff640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 0x7f43580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.535 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f4355ffb640 1 -- 172.21.15.19:0/4213648809 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360035040 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f436cff9640 1 -- 172.21.15.19:0/4026466703 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f43640ceb20 con 0x7f4370102790 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 -- 172.21.15.19:0/4213648809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 msgr2=0x55bc0ed02140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f436ffff640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 0x7f43580a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43680049d0 tx=0x7f4368009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f4360005e00 comp rx=0 tx=0).stop 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 -- 172.21.15.19:0/4213648809 shutdown_connections 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed04a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed02140 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 --2- 172.21.15.19:0/4213648809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf74a0 0x55bc0ecf7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 -- 172.21.15.19:0/4213648809 >> 172.21.15.19:0/4213648809 conn(0x55bc0ecf3a30 msgr2=0x55bc0ecf3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 -- 172.21.15.19:0/4213648809 shutdown_connections 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.494+0000 7f43784a3740 1 -- 172.21.15.19:0/4213648809 wait complete. 2024-09-12T03:30:04.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 -- start start 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 Processor -- start 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ecff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ed05250 con 0x55bc0ecf74a0 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ec18050 con 0x55bc0ed02680 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55bc0ed053d0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f43784a3740 10 client.0 _ll_register_callbacks cb 0x55bc0ec58160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f436effd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ecff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60866/0 (socket says 172.21.15.19:60866) 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f436f7fe640 1 -- 172.21.15.19:0/1351850589 learned_addr learned my addr 172.21.15.19:0/1351850589 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.495+0000 7f436ffff640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 -- 172.21.15.19:0/1351850589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 msgr2=0x55bc0ecff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 -- 172.21.15.19:0/1351850589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 msgr2=0x55bc0ed01fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 -- 172.21.15.19:0/1351850589 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc0e9ce3c0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436ffff640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436f7fe640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f436effd640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ecff260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43640cc680 tx=0x7f43640d21f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436e44856af12935 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364020070 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4360031830 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4364097ab0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f436409af80 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f43784a3740 10 client.26944 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc0ed10cb0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4360031830 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f436409a9b0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f43784a3740 10 client.26944 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.496+0000 7f43784a3740 10 client.26944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f43784a3740 20 client.26944 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:04.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed12050 0x55bc0ed14490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f43784a3740 4 client.26944 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc0ec6a850 con 0x55bc0ed12050 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f436f7fe640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed12050 0x55bc0ed14490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f434effd640 10 client.26944 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.497+0000 7f436f7fe640 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed12050 0x55bc0ed14490 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d6c371d617c66dc server_cookie=ab485b2f8998b60 in_seq=0 out_seq=0 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.520+0000 7f434effd640 1 -- 172.21.15.19:0/1351850589 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc0ec6a850 con 0x55bc0ed12050 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.520+0000 7f434effd640 10 client.26944 handle_command_reply: tid=0 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.520+0000 7f43784a3740 1 client.26944 shutdown 2024-09-12T03:30:04.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.520+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 msgr2=0x55bc0ecff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.520+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ecff260 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f43640cc680 tx=0x7f43640d21f0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 shutdown_connections 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed12050 0x55bc0ed14490 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ecff260 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 --2- 172.21.15.19:0/1351850589 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed01fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 >> 172.21.15.19:0/1351850589 conn(0x55bc0ecf3a30 msgr2=0x55bc0ecf9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f434effd640 0 client.26944 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 shutdown_connections 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 1 -- 172.21.15.19:0/1351850589 wait complete. 2024-09-12T03:30:04.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.521+0000 7f43784a3740 20 client.26944 trim_cache size 0 max 16384 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 Processor -- start 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 -- start start 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed19d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecfc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43701af810 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43701b1590 con 0x55bc0ed02680 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f43784a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43701afee0 con 0x55bc0ecf74a0 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60882/0 (socket says 172.21.15.19:60882) 2024-09-12T03:30:04.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 -- 172.21.15.19:0/2753967255 learned_addr learned my addr 172.21.15.19:0/2753967255 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436f7fe640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed19d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436ffff640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecfc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 -- 172.21.15.19:0/2753967255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf74a0 msgr2=0x55bc0ed19d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed19d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 -- 172.21.15.19:0/2753967255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 msgr2=0x55bc0ecfc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecfc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.538+0000 7f436effd640 1 -- 172.21.15.19:0/2753967255 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc0ecf3860 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.539+0000 7f436ffff640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecfc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.539+0000 7f436effd640 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f436400d6a0 tx=0x7f4364099950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5a7b847f5de721 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.539+0000 7f43557fa640 1 -- 172.21.15.19:0/2753967255 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364020070 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.539+0000 7f43557fa640 1 -- 172.21.15.19:0/2753967255 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43640d4060 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.539+0000 7f43557fa640 1 -- 172.21.15.19:0/2753967255 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436409ada0 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- 172.21.15.19:0/2753967255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 msgr2=0x55bc0ed1a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f436400d6a0 tx=0x7f4364099950 comp rx=0 tx=0).stop 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- 172.21.15.19:0/2753967255 shutdown_connections 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ed02680 0x55bc0ecfc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed1a2a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 --2- 172.21.15.19:0/2753967255 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed19d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- 172.21.15.19:0/2753967255 >> 172.21.15.19:0/2753967255 conn(0x55bc0ecf3a30 msgr2=0x55bc0ecfdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- 172.21.15.19:0/2753967255 shutdown_connections 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- 172.21.15.19:0/2753967255 wait complete. 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 -- start start 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.540+0000 7f43784a3740 1 Processor -- start 2024-09-12T03:30:04.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed08110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed08650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f43701b0e00 con 0x55bc0ecf74a0 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43701b0670 con 0x55bc0ecf7dc0 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f43701ad730 con 0x55bc0ed02680 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f43784a3740 10 client.0 _ll_register_callbacks cb 0x7f43640057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:04.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:60886/0 (socket says 172.21.15.19:60886) 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 -- 172.21.15.19:0/2016833055 learned_addr learned my addr 172.21.15.19:0/2016833055 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436ffff640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed08650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436effd640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed08110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 -- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 msgr2=0x55bc0ed08650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed08650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 -- 172.21.15.19:0/2016833055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf7dc0 msgr2=0x55bc0ed08110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed08110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 -- 172.21.15.19:0/2016833055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc0ed08e30 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f436f7fe640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f4360005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed7288ec3f6bde0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f434effd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.541+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4360040040 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43640d4060 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f4360004040 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4360035040 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 10 client.17598 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc0ed7ccd0 con 0x55bc0ecf74a0 2024-09-12T03:30:04.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43640d4060 con 0x55bc0ecf74a0 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f4360004c10 con 0x55bc0ecf74a0 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 10 client.17598 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 20 client.17598 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed7d010 0x55bc0ed7f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 4 client.17598 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bc0ecffb30 con 0x55bc0ed7d010 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.542+0000 7f436effd640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed7d010 0x55bc0ed7f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.543+0000 7f436effd640 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed7d010 0x55bc0ed7f400 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69ef5b3e1b18eab server_cookie=c64ec51c21a5a9ff in_seq=0 out_seq=0 2024-09-12T03:30:04.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.543+0000 7f434effd640 10 client.17598 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:04.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f434effd640 1 -- 172.21.15.19:0/2016833055 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55bc0ecffb30 con 0x55bc0ed7d010 2024-09-12T03:30:04.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f434effd640 10 client.17598 handle_command_reply: tid=0 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 client.17598 shutdown 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 msgr2=0x55bc0ed0ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f4360009900 tx=0x7f4360005360 comp rx=0 tx=0).stop 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 shutdown_connections 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55bc0ed02680 0x55bc0ed08650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55bc0ed7d010 0x55bc0ed7f400 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55bc0ecf7dc0 0x55bc0ed08110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 --2- 172.21.15.19:0/2016833055 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55bc0ecf74a0 0x55bc0ed0ad70 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 >> 172.21.15.19:0/2016833055 conn(0x55bc0ecf3a30 msgr2=0x55bc0ed00ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.567+0000 7f434effd640 0 client.17598 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 shutdown_connections 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43784a3740 1 -- 172.21.15.19:0/2016833055 wait complete. 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43784a3740 20 client.17598 trim_cache size 0 max 16384 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:04.570 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "scrub active (101 inodes in the stack)", 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "08d931e1-6aa8-458a-9fb1-bf0626396470": { 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "08d931e1-6aa8-458a-9fb1-bf0626396470", 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:04.571 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 msgr2=0x7f43580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 0x7f43580a5040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f43680049d0 tx=0x7f4368009500 comp rx=0 tx=0).stop 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 msgr2=0x7f4370074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 0x7f4370074a30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f436400d6d0 tx=0x7f436400db90 comp rx=0 tx=0).stop 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 shutdown_connections 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f43580a2b80 0x7f43580a5040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f4370106f10 0x7f4370074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f4370102790 0x7f4370074a30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 --2- 172.21.15.19:0/4026466703 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4370101de0 0x7f43700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:04.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 >> 172.21.15.19:0/4026466703 conn(0x7f43700fd8c0 msgr2=0x7f43700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:04.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.568+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 shutdown_connections 2024-09-12T03:30:04.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:04.569+0000 7f43760ca640 1 -- 172.21.15.19:0/4026466703 wait complete. 2024-09-12T03:30:04.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:04 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:30:04.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:04 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:04.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:04 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:04 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:30:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:04 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:04 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe584ff9640 1 -- 172.21.15.19:0/886672295 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5623fff1fbe0 con 0x562400039150 2024-09-12T03:30:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe584ff9640 10 client.17526 handle_command_reply: tid=0 2024-09-12T03:30:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 client.17526 shutdown 2024-09-12T03:30:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 msgr2=0x5623fffc2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:05.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe59c03a040 tx=0x7fe59c0126a0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 shutdown_connections 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5623fffbea30 0x5623fffc66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x562400039150 0x56240003b570 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5623fffb4170 0x5623fffc26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 --2- 172.21.15.19:0/886672295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5623fffb3850 0x5623fffc2160 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe584ff9640 0 client.17526 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.283+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 >> 172.21.15.19:0/886672295 conn(0x5623fffafde0 msgr2=0x5623fffba850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 shutdown_connections 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a9706740 1 -- 172.21.15.19:0/886672295 wait complete. 2024-09-12T03:30:05.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a9706740 20 client.17526 trim_cache size 0 max 16384 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 58, 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 38, 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "75861731": { 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 38, 2024-09-12T03:30:05.286 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.6, 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:05.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:05.288 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 msgr2=0x7fe5880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 0x7fe5880a5040 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe590005770 tx=0x7fe590005b50 comp rx=0 tx=0).stop 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 msgr2=0x7fe5a010ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010ef50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe598009fd0 tx=0x7fe59800ef10 comp rx=0 tx=0).stop 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 shutdown_connections 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe5a0102ef0 0x7fe5a010ef50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe5880a2b80 0x7fe5880a5040 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe5a0102570 0x7fe5a010ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 --2- 172.21.15.19:0/641461307 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe5a0100e20 0x7fe5a010e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 >> 172.21.15.19:0/641461307 conn(0x7fe5a00fc7d0 msgr2=0x7fe5a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.284+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 shutdown_connections 2024-09-12T03:30:05.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:05.285+0000 7fe5a732d640 1 -- 172.21.15.19:0/641461307 wait complete. 2024-09-12T03:30:05.387 INFO:tasks.cephfs.filesystem:scrub status for tag:08d931e1-6aa8-458a-9fb1-bf0626396470 - {'path': '/', 'tag': '08d931e1-6aa8-458a-9fb1-bf0626396470', 'options': 'recursive,force'} 2024-09-12T03:30:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:05 smithi121 ceph-mon[34963]: pgmap v668: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:30:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:05 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:05.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:05 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:05 smithi019 ceph-mon[30953]: pgmap v668: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:30:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:05 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:05 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:05 smithi135 ceph-mon[35474]: pgmap v668: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:30:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:05 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:05 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:06.030 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '75861731', quiesce took 12.3 seconds. Will release after: 10.25 2024-09-12T03:30:07.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:07 smithi121 ceph-mon[34963]: pgmap v669: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 930 KiB/s wr, 313 op/s 2024-09-12T03:30:07.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:07 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:07.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:07 smithi019 ceph-mon[30953]: pgmap v669: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 930 KiB/s wr, 313 op/s 2024-09-12T03:30:07.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:07 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:07.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:07 smithi135 ceph-mon[35474]: pgmap v669: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 885 KiB/s rd, 930 KiB/s wr, 313 op/s 2024-09-12T03:30:07.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:07 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.654 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:08 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.654 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:08 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:08 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:08 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:08 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:08.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:08 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:09.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:09 smithi121 ceph-mon[34963]: pgmap v670: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 568 KiB/s wr, 283 op/s 2024-09-12T03:30:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:09 smithi019 ceph-mon[30953]: pgmap v670: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 568 KiB/s wr, 283 op/s 2024-09-12T03:30:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:09 smithi135 ceph-mon[35474]: pgmap v670: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 563 KiB/s rd, 568 KiB/s wr, 283 op/s 2024-09-12T03:30:11.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:11 smithi121 ceph-mon[34963]: pgmap v671: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 367 KiB/s wr, 203 op/s 2024-09-12T03:30:11.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:11 smithi019 ceph-mon[30953]: pgmap v671: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 367 KiB/s wr, 203 op/s 2024-09-12T03:30:11.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:11 smithi135 ceph-mon[35474]: pgmap v671: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 529 KiB/s rd, 367 KiB/s wr, 203 op/s 2024-09-12T03:30:13.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:13 smithi121 ceph-mon[34963]: pgmap v672: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 245 KiB/s wr, 179 op/s 2024-09-12T03:30:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:13 smithi019 ceph-mon[30953]: pgmap v672: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 245 KiB/s wr, 179 op/s 2024-09-12T03:30:13.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:13 smithi135 ceph-mon[35474]: pgmap v672: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 245 KiB/s wr, 179 op/s 2024-09-12T03:30:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:15 smithi121 ceph-mon[34963]: pgmap v673: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 110 KiB/s wr, 112 op/s 2024-09-12T03:30:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:15 smithi019 ceph-mon[30953]: pgmap v673: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 110 KiB/s wr, 112 op/s 2024-09-12T03:30:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:15 smithi135 ceph-mon[35474]: pgmap v673: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 110 KiB/s wr, 112 op/s 2024-09-12T03:30:16.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:30:16.588 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:17.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2863754136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 msgr2=0x7fc8881021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2863754136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881021c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc878009900 tx=0x7fc87802a410 comp rx=0 tx=0).stop 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2863754136 shutdown_connections 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2863754136 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888106f10 0x7fc88810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2863754136 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888102790 0x7fc8881067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2863754136 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881021c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2863754136 >> 172.21.15.19:0/2863754136 conn(0x7fc8880fd8c0 msgr2=0x7fc8880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2863754136 shutdown_connections 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.086+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2863754136 wait complete. 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 Processor -- start 2024-09-12T03:30:17.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 -- start start 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888102790 0x7fc8881989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 0x7fc88819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc888111f90 con 0x7fc888101de0 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc888111e10 con 0x7fc888102790 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc88d9a6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc888112110 con 0x7fc888106f10 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc886ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37456/0 (socket says 172.21.15.19:37456) 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc886ffd640 1 -- 172.21.15.19:0/2870586852 learned_addr learned my addr 172.21.15.19:0/2870586852 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.087+0000 7fc8867fc640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888102790 0x7fc8881989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:17.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc8877fe640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 0x7fc88819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 -- 172.21.15.19:0/2870586852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 msgr2=0x7fc88819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 0x7fc88819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 -- 172.21.15.19:0/2870586852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888102790 msgr2=0x7fc8881989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888102790 0x7fc8881989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 -- 172.21.15.19:0/2870586852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88819d460 con 0x7fc888101de0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc8877fe640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 0x7fc88819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc886ffd640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc878005ab0 tx=0x7fc8780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87803f030 con 0x7fc888101de0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc878003ce0 con 0x7fc888101de0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88819d6f0 con 0x7fc888101de0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc878035070 con 0x7fc888101de0 2024-09-12T03:30:17.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.088+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8881a5030 con 0x7fc888101de0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.089+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8881032c0 con 0x7fc888101de0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.090+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc878042030 con 0x7fc888101de0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.091+0000 7fc88c9a4640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 0x7fc8700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.091+0000 7fc8867fc640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 0x7fc8700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.091+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc8780efa00 con 0x7fc888101de0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.091+0000 7fc8867fc640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 0x7fc8700a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8881087d0 tx=0x7fc87c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:17.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.095+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8780bc910 con 0x7fc888101de0 2024-09-12T03:30:17.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.312+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88810a0f0 con 0x7fc888101de0 2024-09-12T03:30:17.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.313+0000 7fc88c9a4640 1 -- 172.21.15.19:0/2870586852 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7fc8780bc210 con 0x7fc888101de0 2024-09-12T03:30:17.313 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:30:17.313 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:30:17.314 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:30:17.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 msgr2=0x7fc8700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:17.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 0x7fc8700a5040 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc8881087d0 tx=0x7fc87c009110 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 msgr2=0x7fc8881984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc878005ab0 tx=0x7fc8780042f0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 shutdown_connections 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc888106f10 0x7fc88819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc888102790 0x7fc8881989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc8700a2b80 0x7fc8700a5040 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 --2- 172.21.15.19:0/2870586852 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc888101de0 0x7fc8881984b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 >> 172.21.15.19:0/2870586852 conn(0x7fc8880fd8c0 msgr2=0x7fc8880feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 shutdown_connections 2024-09-12T03:30:17.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:17.315+0000 7fc88d9a6640 1 -- 172.21.15.19:0/2870586852 wait complete. 2024-09-12T03:30:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:17 smithi121 ceph-mon[34963]: pgmap v674: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 KiB/s wr, 83 op/s 2024-09-12T03:30:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:17 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2870586852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:17 smithi019 ceph-mon[30953]: pgmap v674: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 KiB/s wr, 83 op/s 2024-09-12T03:30:17.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:17 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2870586852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:17.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:17 smithi135 ceph-mon[35474]: pgmap v674: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 11 KiB/s wr, 83 op/s 2024-09-12T03:30:17.797 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:17 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2870586852' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:18.857 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 75861731 --release --await' 2024-09-12T03:30:18.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 75861731 --release --await 2024-09-12T03:30:19.123 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:19.540 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:19 smithi019 ceph-mon[30953]: pgmap v675: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 11 KiB/s wr, 80 op/s 2024-09-12T03:30:19.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:30:19.541 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:19 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:30:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:19 smithi121 ceph-mon[34963]: pgmap v675: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 11 KiB/s wr, 80 op/s 2024-09-12T03:30:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:30:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:19 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:30:19.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 -- 172.21.15.19:0/669982977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8101280 msgr2=0x7f2be810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.658 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/669982977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8101280 0x7f2be810df40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009900 tx=0x7f2bd802a410 comp rx=0 tx=0).stop 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 -- 172.21.15.19:0/669982977 shutdown_connections 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/669982977 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be810e480 0x7f2be8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/669982977 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be8101280 0x7f2be810df40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/669982977 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be81008d0 0x7f2be8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 -- 172.21.15.19:0/669982977 >> 172.21.15.19:0/669982977 conn(0x7f2be80fc7a0 msgr2=0x7f2be80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 -- 172.21.15.19:0/669982977 shutdown_connections 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.630+0000 7f2bef7ed640 1 -- 172.21.15.19:0/669982977 wait complete. 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 Processor -- start 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 -- start start 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 0x7f2be819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be8101280 0x7f2be819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.659 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8114190 con 0x7f2be810e480 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8114010 con 0x7f2be81008d0 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bef7ed640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2be8114310 con 0x7f2be8101280 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bedd8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bedd8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37474/0 (socket says 172.21.15.19:37474) 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bedd8a640 1 -- 172.21.15.19:0/1481896634 learned_addr learned my addr 172.21.15.19:0/1481896634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2becd88640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be8101280 0x7f2be819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.631+0000 7f2bed589640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 0x7f2be819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.660 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 -- 172.21.15.19:0/1481896634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be8101280 msgr2=0x7f2be819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be8101280 0x7f2be819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 -- 172.21.15.19:0/1481896634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 msgr2=0x7f2be819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 0x7f2be819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 -- 172.21.15.19:0/1481896634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be819f620 con 0x7f2be810e480 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bed589640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 0x7f2be819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bedd8a640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2be000cab0 tx=0x7f2be000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.661 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bde7fc640 1 -- 172.21.15.19:0/1481896634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0007e10 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bde7fc640 1 -- 172.21.15.19:0/1481896634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be000f930 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bde7fc640 1 -- 172.21.15.19:0/1481896634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0018b80 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be819f910 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.632+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be81a71f0 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.635+0000 7f2bde7fc640 1 -- 172.21.15.19:0/1481896634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2be000fae0 con 0x7f2be810e480 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.635+0000 7f2bf1bc6740 1 Processor -- start 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.635+0000 7f2bf1bc6740 1 -- start start 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c61495e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c614969240 0x55c61496b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.662 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c6148c3620 con 0x55c61495e980 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c61486ae50 con 0x55c61495e060 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c6148bf5e0 con 0x55c614969240 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37482/0 (socket says 172.21.15.19:37482) 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2becd88640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c61495e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 -- 172.21.15.19:0/1687267722 learned_addr learned my addr 172.21.15.19:0/1687267722 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bde7fc640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 0x7f2bd40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2becd88640 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c614969240 0x55c61496b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 -- 172.21.15.19:0/1687267722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c614969240 msgr2=0x55c61496b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c614969240 0x55c61496b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 -- 172.21.15.19:0/1687267722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 msgr2=0x55c61495e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c61495e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.663 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 -- 172.21.15.19:0/1687267722 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61485bd40 con 0x55c61495e980 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bde7fc640 1 -- 172.21.15.19:0/1481896634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2be00cf390 con 0x7f2be810e480 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bedd8a640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 0x7f2bd40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.636+0000 7f2bed589640 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2be4008c60 tx=0x7f2be400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8904c65b0ce43f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bd37fe640 1 -- 172.21.15.19:0/1687267722 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4014070 con 0x55c61495e980 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bd37fe640 1 -- 172.21.15.19:0/1687267722 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be40047a0 con 0x55c61495e980 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bd37fe640 1 -- 172.21.15.19:0/1687267722 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4010760 con 0x55c61495e980 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1687267722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 msgr2=0x55c614968d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bedd8a640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 0x7f2bd40a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2be0098000 tx=0x7f2be00185a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f2be4008c60 tx=0x7f2be400e720 comp rx=0 tx=0).stop 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1687267722 shutdown_connections 2024-09-12T03:30:19.664 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c614969240 0x55c61496b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c61495e980 0x55c614968d00 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1687267722 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c61495e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.637+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1687267722 >> 172.21.15.19:0/1687267722 conn(0x55c61495a5f0 msgr2=0x55c61495a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.638+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1687267722 shutdown_connections 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.638+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1687267722 wait complete. 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.638+0000 7f2bf1bc6740 1 -- start start 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.638+0000 7f2bf1bc6740 1 Processor -- start 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c614966160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.665 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e980 0x55c6149666a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55c61496be10 con 0x55c614969240 2024-09-12T03:30:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55c61484bf10 con 0x55c61495e060 2024-09-12T03:30:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55c61496bf90 con 0x55c61495e980 2024-09-12T03:30:19.666 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 10 client.0 _ll_register_callbacks cb 0x55c61486fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37492/0 (socket says 172.21.15.19:37492) 2024-09-12T03:30:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 -- 172.21.15.19:0/1907439140 learned_addr learned my addr 172.21.15.19:0/1907439140 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c614966160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.667 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bed589640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e980 0x55c6149666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 -- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e980 msgr2=0x55c6149666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e980 0x55c6149666a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 -- 172.21.15.19:0/1907439140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 msgr2=0x55c614966160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c614966160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 -- 172.21.15.19:0/1907439140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6145d2580 con 0x55c614969240 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2becd88640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009900 tx=0x7f2bd802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78210d0a233be801 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8035070 con 0x55c614969240 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2bd8004260 con 0x55c614969240 2024-09-12T03:30:19.668 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.639+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be40047a0 con 0x55c614969240 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8053050 con 0x55c614969240 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 10 client.17616 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c61469c960 con 0x55c614969240 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2be40047a0 con 0x55c614969240 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f2bd8004940 con 0x55c614969240 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 10 client.17616 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 10 client.17616 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 20 client.17616 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c614978db0 0x55c61497b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 4 client.17616 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6148d0b90 con 0x55c614978db0 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.640+0000 7f2bed589640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c614978db0 0x55c61497b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.641+0000 7f2bd0ff9640 10 client.17616 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.641+0000 7f2bed589640 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c614978db0 0x55c61497b1f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ded4a0e3399bb7 server_cookie=363dd381d11b6484 in_seq=0 out_seq=0 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.641+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/1907439140 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c6148d0b90 con 0x55c614978db0 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.641+0000 7f2bd0ff9640 10 client.17616 handle_command_reply: tid=0 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 client.17616 shutdown 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 msgr2=0x55c614966be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009900 tx=0x7f2bd802a410 comp rx=0 tx=0).stop 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 shutdown_connections 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614966be0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e980 0x55c6149666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c614978db0 0x55c61497b1f0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1907439140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e060 0x55c614966160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 >> 172.21.15.19:0/1907439140 conn(0x55c61495a5f0 msgr2=0x55c614959750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:19.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bd0ff9640 0 client.17616 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:19.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 shutdown_connections 2024-09-12T03:30:19.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1907439140 wait complete. 2024-09-12T03:30:19.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.642+0000 7f2bf1bc6740 20 client.17616 trim_cache size 0 max 16384 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 Processor -- start 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 -- start start 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c614961650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614960b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81af800 con 0x55c614969240 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81b1580 con 0x55c61495e980 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81afed0 con 0x55c61495e060 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37494/0 (socket says 172.21.15.19:37494) 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2becd88640 1 -- 172.21.15.19:0/1460935546 learned_addr learned my addr 172.21.15.19:0/1460935546 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2bed589640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614960b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.659+0000 7f2becd88640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c614961650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 -- 172.21.15.19:0/1460935546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 msgr2=0x55c614961650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c614961650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 -- 172.21.15.19:0/1460935546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 msgr2=0x55c614960b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614960b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 -- 172.21.15.19:0/1460935546 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61495f530 con 0x55c614969240 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2bed589640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614960b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2becd88640 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001eb0 tx=0x7f2bd8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538aede0bde44255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2bd2ffd640 1 -- 172.21.15.19:0/1460935546 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8035070 con 0x55c614969240 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2bd2ffd640 1 -- 172.21.15.19:0/1460935546 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8004260 con 0x55c614969240 2024-09-12T03:30:19.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.660+0000 7f2bd2ffd640 1 -- 172.21.15.19:0/1460935546 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8003b40 con 0x55c614969240 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1460935546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 msgr2=0x55c614959640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8001eb0 tx=0x7f2bd8005d90 comp rx=0 tx=0).stop 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1460935546 shutdown_connections 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c614959640 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614960b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/1460935546 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c614961650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1460935546 >> 172.21.15.19:0/1460935546 conn(0x55c61495a5f0 msgr2=0x55c614958f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1460935546 shutdown_connections 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/1460935546 wait complete. 2024-09-12T03:30:19.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 -- start start 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.661+0000 7f2bf1bc6740 1 Processor -- start 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c6149711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614971720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81b0df0 con 0x55c614969240 2024-09-12T03:30:19.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81b0660 con 0x55c61495e980 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2be81ad720 con 0x55c61495e060 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 10 client.0 _ll_register_callbacks cb 0x7f2bd8031a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37508/0 (socket says 172.21.15.19:37508) 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 -- 172.21.15.19:0/3023388230 learned_addr learned my addr 172.21.15.19:0/3023388230 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c6149711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bed589640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614971720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 -- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 msgr2=0x55c6149711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c6149711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 -- 172.21.15.19:0/3023388230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 msgr2=0x55c614971720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614971720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 -- 172.21.15.19:0/3023388230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c614971d60 con 0x55c614969240 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2becd88640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031f60 tx=0x7f2bd8031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1740b30d545403e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bd0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:19.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8035020 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f2bd80038e0 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.662+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bd8004260 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2bd8050040 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 10 client.17622 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c6149e4010 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bd8004260 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f2bd80055e0 con 0x55c614969240 2024-09-12T03:30:19.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 10 client.17622 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 10 client.17622 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 20 client.17622 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c6149e46e0 0x55c6149e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 4 client.17622 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "75861731", "release": true, "await": true}] 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "75861731", "release": true, "await": true}) -- 0x55c6148cb0a0 con 0x55c6149e46e0 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.663+0000 7f2bed589640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c6149e46e0 0x55c6149e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.664+0000 7f2bed589640 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c6149e46e0 0x55c6149e6b00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=170576de73689115 server_cookie=4025824ebc871e96 in_seq=0 out_seq=0 2024-09-12T03:30:19.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:19.664+0000 7f2bd0ff9640 10 client.17622 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:19 smithi135 ceph-mon[35474]: pgmap v675: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 11 KiB/s wr, 80 op/s 2024-09-12T03:30:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:30:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:19 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:30:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bd0ff9640 1 -- 172.21.15.19:0/3023388230 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c6148cb0a0 con 0x55c6149e46e0 2024-09-12T03:30:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bd0ff9640 10 client.17622 handle_command_reply: tid=0 2024-09-12T03:30:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 client.17622 shutdown 2024-09-12T03:30:20.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 msgr2=0x55c61496e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8031f60 tx=0x7f2bd8031e60 comp rx=0 tx=0).stop 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 shutdown_connections 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55c614969240 0x55c61496e1b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55c6149e46e0 0x55c6149e6b00 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55c61495e980 0x55c614971720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 --2- 172.21.15.19:0/3023388230 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55c61495e060 0x55c6149711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bd0ff9640 0 client.17622 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.551+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 >> 172.21.15.19:0/3023388230 conn(0x55c61495a5f0 msgr2=0x55c614966940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.552+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 shutdown_connections 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.552+0000 7f2bf1bc6740 1 -- 172.21.15.19:0/3023388230 wait complete. 2024-09-12T03:30:20.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.552+0000 7f2bf1bc6740 20 client.17622 trim_cache size 0 max 16384 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 58, 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 40, 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "75861731": { 2024-09-12T03:30:20.554 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 40, 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.6, 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:30:20.555 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:20.556 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:20.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 msgr2=0x7f2bd40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 0x7f2bd40a5040 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2be0098000 tx=0x7f2be00185a0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 msgr2=0x7f2be819ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f2be000cab0 tx=0x7f2be000cf70 comp rx=0 tx=0).stop 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 shutdown_connections 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f2bd40a2b80 0x7f2bd40a5040 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2be810e480 0x7f2be819ef40 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2be8101280 0x7f2be819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 --2- 172.21.15.19:0/1481896634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2be81008d0 0x7f2be819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:20.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 >> 172.21.15.19:0/1481896634 conn(0x7f2be80fc7a0 msgr2=0x7f2be80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:20.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 shutdown_connections 2024-09-12T03:30:20.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:20.553+0000 7f2bef7ed640 1 -- 172.21.15.19:0/1481896634 wait complete. 2024-09-12T03:30:20.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:20 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:30:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:20 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:30:20.695 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:20 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:30:21.313 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '75861731', total seconds elapsed: 27.6 2024-09-12T03:30:21.313 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.55 2024-09-12T03:30:21.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:30:21.586 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:21.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:21 smithi121 ceph-mon[34963]: pgmap v676: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 10 KiB/s wr, 54 op/s 2024-09-12T03:30:21.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:21 smithi019 ceph-mon[30953]: pgmap v676: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 10 KiB/s wr, 54 op/s 2024-09-12T03:30:21.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:21 smithi135 ceph-mon[35474]: pgmap v676: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 92 KiB/s rd, 10 KiB/s wr, 54 op/s 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- 172.21.15.19:0/1338174463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70102790 msgr2=0x7f3c701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 --2- 172.21.15.19:0/1338174463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70102790 0x7f3c701067e0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009900 tx=0x7f3c5c02a410 comp rx=0 tx=0).stop 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- 172.21.15.19:0/1338174463 shutdown_connections 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 --2- 172.21.15.19:0/1338174463 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70106f10 0x7f3c7010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 --2- 172.21.15.19:0/1338174463 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70102790 0x7f3c701067e0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 --2- 172.21.15.19:0/1338174463 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- 172.21.15.19:0/1338174463 >> 172.21.15.19:0/1338174463 conn(0x7f3c700fd8c0 msgr2=0x7f3c700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- 172.21.15.19:0/1338174463 shutdown_connections 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- 172.21.15.19:0/1338174463 wait complete. 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 Processor -- start 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.197+0000 7f3c76e84640 1 -- start start 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70106f10 0x7f3c7019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70111f70 con 0x7f3c70106f10 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c70111df0 con 0x7f3c70102790 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c76e84640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c701120f0 con 0x7f3c70101de0 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c74c20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:22.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c75421640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70106f10 0x7f3c7019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c74c20640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45798/0 (socket says 172.21.15.19:45798) 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39252/0 (socket says 172.21.15.19:39252) 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c74c20640 1 -- 172.21.15.19:0/2102479510 learned_addr learned my addr 172.21.15.19:0/2102479510 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 -- 172.21.15.19:0/2102479510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 msgr2=0x7f3c70198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 -- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70106f10 msgr2=0x7f3c7019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70106f10 0x7f3c7019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 -- 172.21.15.19:0/2102479510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c7019d440 con 0x7f3c70102790 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c74c20640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.198+0000 7f3c6ffff640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c005940 tx=0x7f3c5c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:22.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.199+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c03f030 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.199+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c7019d6d0 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.199+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c5c0044c0 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.199+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c035070 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.199+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c701a5010 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.200+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c701021c0 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.201+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c5c042030 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.202+0000 7f3c6dffb640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 0x7f3c600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.202+0000 7f3c74c20640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 0x7f3c600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.202+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3c5c0ef5e0 con 0x7f3c70102790 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.202+0000 7f3c74c20640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 0x7f3c600a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3c64011550 tx=0x7f3c64005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:22.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.205+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c5c0bc4f0 con 0x7f3c70102790 2024-09-12T03:30:22.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.426+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c701997f0 con 0x7f3c70102790 2024-09-12T03:30:22.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.427+0000 7f3c6dffb640 1 -- 172.21.15.19:0/2102479510 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7f3c5c0bbdf0 con 0x7f3c70102790 2024-09-12T03:30:22.427 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:30:22.427 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:30:22.428 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:30:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 msgr2=0x7f3c600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 0x7f3c600a5040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3c64011550 tx=0x7f3c64005f90 comp rx=0 tx=0).stop 2024-09-12T03:30:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 msgr2=0x7f3c701989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c005940 tx=0x7f3c5c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 shutdown_connections 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c600a2b80 0x7f3c600a5040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c70106f10 0x7f3c7019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c70102790 0x7f3c701989d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 --2- 172.21.15.19:0/2102479510 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c70101de0 0x7f3c70198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 >> 172.21.15.19:0/2102479510 conn(0x7f3c700fd8c0 msgr2=0x7f3c700fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 shutdown_connections 2024-09-12T03:30:22.430 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:22.429+0000 7f3c76e84640 1 -- 172.21.15.19:0/2102479510 wait complete. 2024-09-12T03:30:23.134 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0' 2024-09-12T03:30:23.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 142.55 --await-for 45.0 2024-09-12T03:30:23.398 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:23 smithi019 ceph-mon[30953]: pgmap v677: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 KiB/s wr, 36 op/s 2024-09-12T03:30:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:23 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2102479510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:23.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:23 smithi121 ceph-mon[34963]: pgmap v677: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 KiB/s wr, 36 op/s 2024-09-12T03:30:23.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:23 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2102479510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:23.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:23 smithi135 ceph-mon[35474]: pgmap v677: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 7.7 KiB/s wr, 36 op/s 2024-09-12T03:30:23.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:23 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2102479510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.941+0000 7f783bf92640 1 -- 172.21.15.19:0/1002178673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834106a80 msgr2=0x7f7834106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.941+0000 7f783bf92640 1 --2- 172.21.15.19:0/1002178673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834106a80 0x7f7834106e60 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f7824009cd0 tx=0x7f782402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 -- 172.21.15.19:0/1002178673 shutdown_connections 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 --2- 172.21.15.19:0/1002178673 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834101420 0x7f783410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 --2- 172.21.15.19:0/1002178673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834100a80 0x7f7834100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 --2- 172.21.15.19:0/1002178673 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834106a80 0x7f7834106e60 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 -- 172.21.15.19:0/1002178673 >> 172.21.15.19:0/1002178673 conn(0x7f78340fc7a0 msgr2=0x7f78340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 -- 172.21.15.19:0/1002178673 shutdown_connections 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 -- 172.21.15.19:0/1002178673 wait complete. 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 Processor -- start 2024-09-12T03:30:23.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.942+0000 7f783bf92640 1 -- start start 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834101420 0x7f78341989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7834110e80 con 0x7f7834100a80 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7834110d00 con 0x7f7834101420 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783bf92640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7834111000 con 0x7f7834106a80 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.945 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42196/0 (socket says 172.21.15.19:42196) 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 -- 172.21.15.19:0/2885718775 learned_addr learned my addr 172.21.15.19:0/2885718775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45822/0 (socket says 172.21.15.19:45822) 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783952d640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834101420 0x7f78341989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 -- 172.21.15.19:0/2885718775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 msgr2=0x7f783419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 -- 172.21.15.19:0/2885718775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834101420 msgr2=0x7f78341989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834101420 0x7f78341989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 -- 172.21.15.19:0/2885718775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783419d410 con 0x7f7834100a80 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f783a52f640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.943+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7824005e30 tx=0x7f7824031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.944+0000 7f7832ffd640 1 -- 172.21.15.19:0/2885718775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824051070 con 0x7f7834100a80 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.944+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f783419d6a0 con 0x7f7834100a80 2024-09-12T03:30:23.946 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.944+0000 7f7832ffd640 1 -- 172.21.15.19:0/2885718775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7824003e50 con 0x7f7834100a80 2024-09-12T03:30:23.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.944+0000 7f7832ffd640 1 -- 172.21.15.19:0/2885718775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7824030800 con 0x7f7834100a80 2024-09-12T03:30:23.947 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.944+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78340fccc0 con 0x7f7834100a80 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 Processor -- start 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 -- start start 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f7832ffd640 1 -- 172.21.15.19:0/2885718775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f78240041b0 con 0x7f7834100a80 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bdf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4be1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616e4b39160 con 0x5616e4bdfa30 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5616e4ae0720 con 0x5616e4bd5170 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783e36b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616e4b35120 con 0x5616e4bd4850 2024-09-12T03:30:23.949 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783952d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4be1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45824/0 (socket says 172.21.15.19:45824) 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.947+0000 7f783a52f640 1 -- 172.21.15.19:0/534279021 learned_addr learned my addr 172.21.15.19:0/534279021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f7839d2e640 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bdf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 -- 172.21.15.19:0/534279021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 msgr2=0x5616e4bd4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 -- 172.21.15.19:0/534279021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 msgr2=0x5616e4bdf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bdf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 -- 172.21.15.19:0/534279021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e48aaba0 con 0x5616e4bdfa30 2024-09-12T03:30:23.950 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783a52f640 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f7832ffd640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 0x7f781c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783952d640 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4be1e20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7828005d00 tx=0x7f78280107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2ed101667f8ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f783a52f640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 0x7f781c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f7832ffd640 1 -- 172.21.15.19:0/2885718775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f782402e020 con 0x7f7834100a80 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f781bfff640 1 -- 172.21.15.19:0/534279021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782801c480 con 0x5616e4bdfa30 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.948+0000 7f781bfff640 1 -- 172.21.15.19:0/534279021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782801cb00 con 0x5616e4bdfa30 2024-09-12T03:30:23.951 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f781bfff640 1 -- 172.21.15.19:0/534279021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828011bf0 con 0x5616e4bdfa30 2024-09-12T03:30:23.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 -- 172.21.15.19:0/534279021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 msgr2=0x5616e4be1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783a52f640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 0x7f781c0a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f782c009920 tx=0x7f782c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4be1e20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f7828005d00 tx=0x7f78280107a0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 -- 172.21.15.19:0/534279021 shutdown_connections 2024-09-12T03:30:23.955 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4be1e20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bdf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 --2- 172.21.15.19:0/534279021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bd4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 -- 172.21.15.19:0/534279021 >> 172.21.15.19:0/534279021 conn(0x5616e4bd0de0 msgr2=0x5616e4bd11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 -- 172.21.15.19:0/534279021 shutdown_connections 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.949+0000 7f783e36b740 1 -- 172.21.15.19:0/534279021 wait complete. 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 -- start start 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 Processor -- start 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4bdc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.956 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4bdd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5616e4be2600 con 0x5616e4bd5170 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5616e4ac1900 con 0x5616e4bd4850 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5616e4be2780 con 0x5616e4bdfa30 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783e36b740 10 client.0 _ll_register_callbacks cb 0x5616e4abf780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783952d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783952d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42208/0 (socket says 172.21.15.19:42208) 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.950+0000 7f783952d640 1 -- 172.21.15.19:0/2204875821 learned_addr learned my addr 172.21.15.19:0/2204875821 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:23.957 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783a52f640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4bdd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4bdc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 -- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 msgr2=0x5616e4bdd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4bdd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 -- 172.21.15.19:0/2204875821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 msgr2=0x5616e4bdc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4bdc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 -- 172.21.15.19:0/2204875821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e4b01c50 con 0x5616e4bd5170 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4bdc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783952d640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f782800bf50 tx=0x7f782800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afee7923e2f4531 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.958 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f78197fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782801c690 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f782801c830 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782801cb00 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7828004020 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783e36b740 10 client.17640 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.951+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5616e4bee280 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782801cb00 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f7828003710 con 0x5616e4bd5170 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 10 client.17640 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 10 client.17640 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 20 client.17640 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:23.959 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4bef740 0x5616e4bf1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 4 client.17640 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616e4b46890 con 0x5616e4bef740 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.952+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4bef740 0x5616e4bf1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.953+0000 7f7839d2e640 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4bef740 0x5616e4bf1b30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=112104dcd908d21b server_cookie=9b5d3a339dbdd9f1 in_seq=0 out_seq=0 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.953+0000 7f78197fa640 10 client.17640 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f78197fa640 1 -- 172.21.15.19:0/2204875821 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616e4b46890 con 0x5616e4bef740 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f78197fa640 10 client.17640 handle_command_reply: tid=0 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 client.17640 shutdown 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 msgr2=0x5616e4bdce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f782800bf50 tx=0x7f782800bf80 comp rx=0 tx=0).stop 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 shutdown_connections 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4bdd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.960 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4bdce20 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4bef740 0x5616e4bf1b30 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 --2- 172.21.15.19:0/2204875821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4bdc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f78197fa640 0 client.17640 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 >> 172.21.15.19:0/2204875821 conn(0x5616e4bd0de0 msgr2=0x5616e4bd6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 shutdown_connections 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 1 -- 172.21.15.19:0/2204875821 wait complete. 2024-09-12T03:30:23.961 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.954+0000 7f783e36b740 20 client.17640 trim_cache size 0 max 16384 2024-09-12T03:30:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 Processor -- start 2024-09-12T03:30:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 -- start start 2024-09-12T03:30:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bf7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bf7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ab410 con 0x5616e4bdfa30 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ad190 con 0x5616e4bd5170 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783e36b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78341abae0 con 0x5616e4bd4850 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783a52f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783a52f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42214/0 (socket says 172.21.15.19:42214) 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783a52f640 1 -- 172.21.15.19:0/1321846556 learned_addr learned my addr 172.21.15.19:0/1321846556 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f7839d2e640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bf7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.974+0000 7f783952d640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bf7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 -- 172.21.15.19:0/1321846556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 msgr2=0x5616e4bf7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bf7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 -- 172.21.15.19:0/1321846556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 msgr2=0x5616e4bf7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bf7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 -- 172.21.15.19:0/1321846556 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616e4bd0c10 con 0x5616e4bdfa30 2024-09-12T03:30:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783952d640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bf7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:23.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f783a52f640 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f782c004390 tx=0x7f782c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f1c736a6d5c692 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f781b7fe640 1 -- 172.21.15.19:0/1321846556 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c004480 con 0x5616e4bdfa30 2024-09-12T03:30:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f781b7fe640 1 -- 172.21.15.19:0/1321846556 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782c005500 con 0x5616e4bdfa30 2024-09-12T03:30:23.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.975+0000 7f781b7fe640 1 -- 172.21.15.19:0/1321846556 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782c0035f0 con 0x5616e4bdfa30 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- 172.21.15.19:0/1321846556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 msgr2=0x5616e4bd9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f782c004390 tx=0x7f782c015000 comp rx=0 tx=0).stop 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- 172.21.15.19:0/1321846556 shutdown_connections 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bdfa30 0x5616e4bd9b40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd5170 0x5616e4bf7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 --2- 172.21.15.19:0/1321846556 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bd4850 0x5616e4bf7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.984 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- 172.21.15.19:0/1321846556 >> 172.21.15.19:0/1321846556 conn(0x5616e4bd0de0 msgr2=0x5616e4bde960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- 172.21.15.19:0/1321846556 shutdown_connections 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- 172.21.15.19:0/1321846556 wait complete. 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 -- start start 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.976+0000 7f783e36b740 1 Processor -- start 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4be8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4be48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f78341aca00 con 0x5616e4bd5170 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f78341ac270 con 0x5616e4bd4850 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f78341a9330 con 0x5616e4bdfa30 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783e36b740 10 client.0 _ll_register_callbacks cb 0x7f78240fb520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:23.985 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4be48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42230/0 (socket says 172.21.15.19:42230) 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 -- 172.21.15.19:0/607140474 learned_addr learned my addr 172.21.15.19:0/607140474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783a52f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4be48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45862/0 (socket says 172.21.15.19:45862) 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f7839d2e640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4be8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 -- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 msgr2=0x5616e4be48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4be48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 -- 172.21.15.19:0/607140474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 msgr2=0x5616e4be8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4be8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f783952d640 1 -- 172.21.15.19:0/607140474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616e4bd1a70 con 0x5616e4bd5170 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.977+0000 7f7839d2e640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4be8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783952d640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7828006010 tx=0x7f7828010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21a25ee3cfa7443 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:23.986 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828035040 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f782c005500 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f782800b7e0 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7828003d20 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 10 client.17646 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5616e4c5ae80 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f782c005500 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f7828003710 con 0x5616e4bd5170 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 10 client.17646 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 10 client.17646 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 20 client.17646 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:23.987 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.978+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4c5b6a0 0x5616e4c5dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.979+0000 7f783e36b740 4 client.17646 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}] 2024-09-12T03:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.979+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.55, "await_for": 45.0}) -- 0x5616e4b40be0 con 0x5616e4c5b6a0 2024-09-12T03:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.979+0000 7f7839d2e640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4c5b6a0 0x5616e4c5dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.979+0000 7f78197fa640 10 client.17646 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:23.988 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:23.979+0000 7f7839d2e640 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4c5b6a0 0x5616e4c5dac0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d9911766230165c server_cookie=7834ffc85e374d01 in_seq=0 out_seq=0 2024-09-12T03:30:25.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:25 smithi121 ceph-mon[34963]: pgmap v678: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 176 KiB/s wr, 24 op/s 2024-09-12T03:30:25.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:25 smithi019 ceph-mon[30953]: pgmap v678: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 176 KiB/s wr, 24 op/s 2024-09-12T03:30:25.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:25 smithi135 ceph-mon[35474]: pgmap v678: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 176 KiB/s wr, 24 op/s 2024-09-12T03:30:27.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:27 smithi121 ceph-mon[34963]: pgmap v679: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 362 KiB/s wr, 55 op/s 2024-09-12T03:30:27.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:27 smithi019 ceph-mon[30953]: pgmap v679: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 362 KiB/s wr, 55 op/s 2024-09-12T03:30:27.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:27 smithi135 ceph-mon[35474]: pgmap v679: 97 pgs: 97 active+clean; 1.8 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 362 KiB/s wr, 55 op/s 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f78197fa640 1 -- 172.21.15.19:0/607140474 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5616e4b40be0 con 0x5616e4c5b6a0 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f78197fa640 10 client.17646 handle_command_reply: tid=0 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 client.17646 shutdown 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 msgr2=0x5616e4be43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f7828006010 tx=0x7f7828010e10 comp rx=0 tx=0).stop 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 shutdown_connections 2024-09-12T03:30:28.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5616e4bdfa30 0x5616e4be48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5616e4bd5170 0x5616e4be43a0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5616e4c5b6a0 0x5616e4c5dac0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 --2- 172.21.15.19:0/607140474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5616e4bd4850 0x5616e4be8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f78197fa640 0 client.17646 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 >> 172.21.15.19:0/607140474 conn(0x5616e4bd0de0 msgr2=0x5616e4bdc880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.712+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 shutdown_connections 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783e36b740 1 -- 172.21.15.19:0/607140474 wait complete. 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783e36b740 20 client.17646 trim_cache size 0 max 16384 2024-09-12T03:30:28.714 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 58, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 42, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "650c73d6": { 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 42, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:30:28.715 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.6, 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:28.716 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 msgr2=0x7f781c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:28.717 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 0x7f781c0a5040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f782c009920 tx=0x7f782c005f90 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 msgr2=0x7f7834198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.713+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f7824005e30 tx=0x7f7824031af0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 shutdown_connections 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7834106a80 0x7f783419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7834101420 0x7f78341989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f781c0a2b80 0x7f781c0a5040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 --2- 172.21.15.19:0/2885718775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7834100a80 0x7f7834198460 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 >> 172.21.15.19:0/2885718775 conn(0x7f78340fc7a0 msgr2=0x7f783406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 shutdown_connections 2024-09-12T03:30:28.718 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:28.714+0000 7f783bf92640 1 -- 172.21.15.19:0/2885718775 wait complete. 2024-09-12T03:30:29.570 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '650c73d6', quiesce took 8.3 seconds. Will release after: 14.25 2024-09-12T03:30:29.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:29 smithi121 ceph-mon[34963]: pgmap v680: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 862 KiB/s wr, 113 op/s 2024-09-12T03:30:29.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:29 smithi019 ceph-mon[30953]: pgmap v680: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 862 KiB/s wr, 113 op/s 2024-09-12T03:30:29.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:29 smithi135 ceph-mon[35474]: pgmap v680: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 862 KiB/s wr, 113 op/s 2024-09-12T03:30:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:31 smithi121 ceph-mon[34963]: pgmap v681: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 868 KiB/s wr, 127 op/s 2024-09-12T03:30:31.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:31 smithi019 ceph-mon[30953]: pgmap v681: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 868 KiB/s wr, 127 op/s 2024-09-12T03:30:31.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:31 smithi135 ceph-mon[35474]: pgmap v681: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 868 KiB/s wr, 127 op/s 2024-09-12T03:30:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:33 smithi019 ceph-mon[30953]: pgmap v682: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 873 KiB/s wr, 135 op/s 2024-09-12T03:30:33.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:33 smithi135 ceph-mon[35474]: pgmap v682: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 873 KiB/s wr, 135 op/s 2024-09-12T03:30:33.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:33 smithi121 ceph-mon[34963]: pgmap v682: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 873 KiB/s wr, 135 op/s 2024-09-12T03:30:35.388 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:30:35.663 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:35.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:35 smithi019 ceph-mon[30953]: pgmap v683: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 878 KiB/s wr, 138 op/s 2024-09-12T03:30:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:35 smithi135 ceph-mon[35474]: pgmap v683: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 878 KiB/s wr, 138 op/s 2024-09-12T03:30:35.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:35 smithi121 ceph-mon[34963]: pgmap v683: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 879 KiB/s rd, 878 KiB/s wr, 138 op/s 2024-09-12T03:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- 172.21.15.19:0/1321371014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf74103760 msgr2=0x7fbf74107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 --2- 172.21.15.19:0/1321371014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf74103760 0x7fbf74107ba0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbf68009930 tx=0x7fbf6802a420 comp rx=0 tx=0).stop 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- 172.21.15.19:0/1321371014 shutdown_connections 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 --2- 172.21.15.19:0/1321371014 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf741082e0 0x7fbf7410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 --2- 172.21.15.19:0/1321371014 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf74103760 0x7fbf74107ba0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 --2- 172.21.15.19:0/1321371014 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 0x7fbf74103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- 172.21.15.19:0/1321371014 >> 172.21.15.19:0/1321371014 conn(0x7fbf740fc7d0 msgr2=0x7fbf740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- 172.21.15.19:0/1321371014 shutdown_connections 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- 172.21.15.19:0/1321371014 wait complete. 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 Processor -- start 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.158+0000 7fbf7c492640 1 -- start start 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 0x7fbf74100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.170 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf74103760 0x7fbf74100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112070 con 0x7fbf741082e0 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74111ef0 con 0x7fbf74102db0 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7c492640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741121f0 con 0x7fbf74103760 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41838/0 (socket says 172.21.15.19:41838) 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 -- 172.21.15.19:0/2660766232 learned_addr learned my addr 172.21.15.19:0/2660766232 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 0x7fbf74100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.171 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf74103760 0x7fbf74100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 -- 172.21.15.19:0/2660766232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf74103760 msgr2=0x7fbf74100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf74103760 0x7fbf74100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 -- 172.21.15.19:0/2660766232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 msgr2=0x7fbf74100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 0x7fbf74100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 -- 172.21.15.19:0/2660766232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf741a6a90 con 0x7fbf741082e0 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf7aa2f640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d750 tx=0x7fbf6c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.159+0000 7fbf737fe640 1 -- 172.21.15.19:0/2660766232 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c017070 con 0x7fbf741082e0 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.160+0000 7fbf737fe640 1 -- 172.21.15.19:0/2660766232 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf6c005500 con 0x7fbf741082e0 2024-09-12T03:30:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.160+0000 7fbf737fe640 1 -- 172.21.15.19:0/2660766232 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c004d90 con 0x7fbf741082e0 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.160+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf741a6d80 con 0x7fbf741082e0 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.160+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741a7290 con 0x7fbf741082e0 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf737fe640 1 -- 172.21.15.19:0/2660766232 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf6c004f30 con 0x7fbf741082e0 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf7e86b740 1 Processor -- start 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf7e86b740 1 -- start start 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f07093d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.162+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f0708a32e0 con 0x55f07093ddc0 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f07084ace0 con 0x55f070948680 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f07089f2a0 con 0x55f07093d4a0 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41848/0 (socket says 172.21.15.19:41848) 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3300652826 learned_addr learned my addr 172.21.15.19:0/3300652826 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f07093d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf737fe640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 0x7fbf5c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3300652826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 msgr2=0x55f07093d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f07093d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3300652826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 msgr2=0x55f07094aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3300652826 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f07082f440 con 0x55f07093ddc0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7aa2f640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 0x7fbf5c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf737fe640 1 -- 172.21.15.19:0/2660766232 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf6c0ce800 con 0x7fbf741082e0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbf640039f0 tx=0x7fbf64003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bdf4d58733c85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.163+0000 7fbf70ff9640 1 -- 172.21.15.19:0/3300652826 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64004b20 con 0x55f07093ddc0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf70ff9640 1 -- 172.21.15.19:0/3300652826 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf64004cc0 con 0x55f07093ddc0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf70ff9640 1 -- 172.21.15.19:0/3300652826 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64012740 con 0x55f07093ddc0 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3300652826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 msgr2=0x55f070948140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fbf640039f0 tx=0x7fbf64003eb0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7aa2f640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 0x7fbf5c0a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c09b0b0 tx=0x7fbf6c065da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3300652826 shutdown_connections 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070948140 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3300652826 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f07093d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3300652826 >> 172.21.15.19:0/3300652826 conn(0x55f070939a30 msgr2=0x55f070939e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3300652826 shutdown_connections 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.164+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3300652826 wait complete. 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 -- start start 2024-09-12T03:30:36.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 Processor -- start 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f07093ddc0 0x55f0709451e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f070948680 0x55f070945720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f07094b250 con 0x55f07093d4a0 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f07085e050 con 0x55f07093ddc0 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f07094b3d0 con 0x55f070948680 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7e86b740 10 client.0 _ll_register_callbacks cb 0x55f07089e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf79a2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf79a2d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41856/0 (socket says 172.21.15.19:41856) 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf79a2d640 1 -- 172.21.15.19:0/1523848200 learned_addr learned my addr 172.21.15.19:0/1523848200 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.165+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f07093ddc0 0x55f0709451e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f070948680 0x55f070945720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 -- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f070948680 msgr2=0x55f070945720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f070948680 0x55f070945720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 -- 172.21.15.19:0/1523848200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f07093ddc0 msgr2=0x55f0709451e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f07093ddc0 0x55f0709451e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 -- 172.21.15.19:0/1523848200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0706143c0 con 0x55f07093d4a0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbf740697d0 tx=0x7fbf68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5535f5933e7759f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68035040 con 0x55f07093d4a0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf68003c00 con 0x55f07093d4a0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf64004cc0 con 0x55f07093d4a0 2024-09-12T03:30:36.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbf68003e00 con 0x55f07093d4a0 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf7e86b740 10 client.17658 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f070956b70 con 0x55f07093d4a0 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.166+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf64004cc0 con 0x55f07093d4a0 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fbf6803d750 con 0x55f07093d4a0 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 10 client.17658 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 20 client.17658 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f070957fd0 0x55f07095a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 4 client.17658 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:36.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0708b0850 con 0x55f070957fd0 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f070957fd0 0x55f07095a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f070957fd0 0x55f07095a470 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363e89fd2eb156b8 server_cookie=fa9da9971f60e97 in_seq=0 out_seq=0 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.167+0000 7fbf59ffb640 10 client.17658 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf59ffb640 1 -- 172.21.15.19:0/1523848200 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0708b0850 con 0x55f070957fd0 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf59ffb640 10 client.17658 handle_command_reply: tid=0 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 client.17658 shutdown 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 msgr2=0x55f070947f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fbf740697d0 tx=0x7fbf68009260 comp rx=0 tx=0).stop 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 shutdown_connections 2024-09-12T03:30:36.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f070948680 0x55f070945720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f070957fd0 0x55f07095a470 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f07093ddc0 0x55f0709451e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1523848200 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093d4a0 0x55f070947f30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 >> 172.21.15.19:0/1523848200 conn(0x55f070939a30 msgr2=0x55f07093f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.168+0000 7fbf59ffb640 0 client.17658 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.169+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 shutdown_connections 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.169+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1523848200 wait complete. 2024-09-12T03:30:36.181 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.169+0000 7fbf7e86b740 20 client.17658 trim_cache size 0 max 16384 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 Processor -- start 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 -- start start 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709544b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07095fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.187 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741af900 con 0x55f07093ddc0 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b1680 con 0x55f070948680 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741affd0 con 0x55f07093d4a0 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41858/0 (socket says 172.21.15.19:41858) 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf79a2d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 -- 172.21.15.19:0/1821246907 learned_addr learned my addr 172.21.15.19:0/1821246907 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07095fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 -- 172.21.15.19:0/1821246907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 msgr2=0x55f0709544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 -- 172.21.15.19:0/1821246907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 msgr2=0x55f07095fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.186+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07095fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.188 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf7a22e640 1 -- 172.21.15.19:0/1821246907 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f070943730 con 0x55f07093ddc0 2024-09-12T03:30:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x55f07093a890 tx=0x7fbf64000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80361da540d24fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf5bfff640 1 -- 172.21.15.19:0/1821246907 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf640140d0 con 0x55f07093ddc0 2024-09-12T03:30:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf5bfff640 1 -- 172.21.15.19:0/1821246907 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf64014270 con 0x55f07093ddc0 2024-09-12T03:30:36.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf5bfff640 1 -- 172.21.15.19:0/1821246907 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64014b90 con 0x55f07093ddc0 2024-09-12T03:30:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1821246907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 msgr2=0x55f070938350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.187+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x55f07093a890 tx=0x7fbf64000f40 comp rx=0 tx=0).stop 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1821246907 shutdown_connections 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07095fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070938350 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/1821246907 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1821246907 >> 172.21.15.19:0/1821246907 conn(0x55f070939a30 msgr2=0x55f070947260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1821246907 shutdown_connections 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- 172.21.15.19:0/1821246907 wait complete. 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- start start 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 Processor -- start 2024-09-12T03:30:36.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709509c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b0ef0 con 0x55f07093ddc0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b0760 con 0x55f070948680 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741ad820 con 0x55f07093d4a0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7e86b740 10 client.0 _ll_register_callbacks cb 0x7fbf68005e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.188+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41868/0 (socket says 172.21.15.19:41868) 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3064052825 learned_addr learned my addr 172.21.15.19:0/3064052825 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 msgr2=0x55f0709509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3064052825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 msgr2=0x55f07094df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 -- 172.21.15.19:0/3064052825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f07094e790 con 0x55f07093ddc0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7a22e640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbf640096e0 tx=0x7fbf640140d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740626e3afaf6784 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64005250 con 0x55f07093ddc0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fbf640053f0 con 0x55f07093ddc0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf64014270 con 0x55f07093ddc0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.189+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbf640147f0 con 0x55f07093ddc0 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 10 client.17664 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:36.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55f0709c2990 con 0x55f07093ddc0 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf64014270 con 0x55f07093ddc0 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fbf6400f040 con 0x55f07093ddc0 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 10 client.17664 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 10 client.17664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 20 client.17664 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f0709c2f00 0x55f0709c5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 4 client.17664 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f070945ab0 con 0x55f0709c2f00 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.190+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f0709c2f00 0x55f0709c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf59ffb640 10 client.17664 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf79a2d640 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f0709c2f00 0x55f0709c5320 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30f3524ea99be7f server_cookie=75c5a2772d45de18 in_seq=0 out_seq=0 2024-09-12T03:30:36.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf59ffb640 1 -- 172.21.15.19:0/3064052825 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f070945ab0 con 0x55f0709c2f00 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf59ffb640 10 client.17664 handle_command_reply: tid=0 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 client.17664 shutdown 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 msgr2=0x55f070950f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fbf640096e0 tx=0x7fbf640140d0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 shutdown_connections 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f0709c2f00 0x55f0709c5320 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f070948680 0x55f07094df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f07093ddc0 0x55f070950f00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 --2- 172.21.15.19:0/3064052825 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f07093d4a0 0x55f0709509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 >> 172.21.15.19:0/3064052825 conn(0x55f070939a30 msgr2=0x55f070943ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf59ffb640 0 client.17664 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 shutdown_connections 2024-09-12T03:30:36.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 1 -- 172.21.15.19:0/3064052825 wait complete. 2024-09-12T03:30:36.198 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.191+0000 7fbf7e86b740 20 client.17664 trim_cache size 0 max 16384 2024-09-12T03:30:36.198 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:30:36.198 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:30:36.198 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 msgr2=0x7fbf5c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 0x7fbf5c0a50c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c09b0b0 tx=0x7fbf6c065da0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 msgr2=0x7fbf74101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00d750 tx=0x7fbf6c00dc10 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 shutdown_connections 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fbf5c0a2c00 0x7fbf5c0a50c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fbf741082e0 0x7fbf74101490 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fbf74103760 0x7fbf74100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 --2- 172.21.15.19:0/2660766232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbf74102db0 0x7fbf74100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 >> 172.21.15.19:0/2660766232 conn(0x7fbf740fc7d0 msgr2=0x7fbf74105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:36.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 shutdown_connections 2024-09-12T03:30:36.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:36.192+0000 7fbf7c492640 1 -- 172.21.15.19:0/2660766232 wait complete. 2024-09-12T03:30:36.901 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:30:36.902 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:30:37.161 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:37.606 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:37 smithi019 ceph-mon[30953]: pgmap v684: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 711 KiB/s wr, 125 op/s 2024-09-12T03:30:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 -- 172.21.15.19:0/1849696584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090106f10 msgr2=0x7f209010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 --2- 172.21.15.19:0/1849696584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090106f10 0x7f209010ea90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2088009ca0 tx=0x7f208802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 -- 172.21.15.19:0/1849696584 shutdown_connections 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 --2- 172.21.15.19:0/1849696584 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090106f10 0x7f209010ea90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 --2- 172.21.15.19:0/1849696584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090102790 0x7f20901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 --2- 172.21.15.19:0/1849696584 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090101de0 0x7f20901021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 -- 172.21.15.19:0/1849696584 >> 172.21.15.19:0/1849696584 conn(0x7f20900fd8c0 msgr2=0x7f20900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 -- 172.21.15.19:0/1849696584 shutdown_connections 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 -- 172.21.15.19:0/1849696584 wait complete. 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.690+0000 7f2094e96640 1 Processor -- start 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 -- start start 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111f70 con 0x7f2090101de0 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2090111df0 con 0x7f2090106f10 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2094e96640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20901120f0 con 0x7f2090102790 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41888/0 (socket says 172.21.15.19:41888) 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 -- 172.21.15.19:0/1584623529 learned_addr learned my addr 172.21.15.19:0/1584623529 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208ed9d640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f2087fff640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 -- 172.21.15.19:0/1584623529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102790 msgr2=0x7f2090074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 -- 172.21.15.19:0/1584623529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 msgr2=0x7f2090074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 -- 172.21.15.19:0/1584623529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20901a6890 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208ed9d640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.691+0000 7f208e59c640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f208000d770 tx=0x7f208000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.692+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080017070 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.692+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20901a6b80 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.692+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2080005500 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.692+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2080004e00 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.692+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20901a70c0 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.693+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20901021c0 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.694+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2080004fa0 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.694+0000 7f20877fe640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 0x7f207c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.695+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f20800cef10 con 0x7f2090101de0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.695+0000 7f2087fff640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 0x7f207c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:37.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.695+0000 7f2087fff640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 0x7f207c0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2078005d80 tx=0x7f2078005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:37.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.698+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f208009be80 con 0x7f2090101de0 2024-09-12T03:30:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:37 smithi135 ceph-mon[35474]: pgmap v684: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 711 KiB/s wr, 125 op/s 2024-09-12T03:30:37.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:37 smithi121 ceph-mon[34963]: pgmap v684: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 711 KiB/s wr, 125 op/s 2024-09-12T03:30:37.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.912+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f209002d7c0 con 0x7f2090101de0 2024-09-12T03:30:37.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.913+0000 7f20877fe640 1 -- 172.21.15.19:0/1584623529 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7f208009b780 con 0x7f2090101de0 2024-09-12T03:30:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:30:37.914 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:30:37.914 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:30:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 msgr2=0x7f207c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 0x7f207c0a5040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2078005d80 tx=0x7f2078005c90 comp rx=0 tx=0).stop 2024-09-12T03:30:37.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 msgr2=0x7f20900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f208000d770 tx=0x7f208000dc30 comp rx=0 tx=0).stop 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 shutdown_connections 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2090106f10 0x7f2090074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2090102790 0x7f2090074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f207c0a2b80 0x7f207c0a5040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 --2- 172.21.15.19:0/1584623529 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2090101de0 0x7f20900744f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.915+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 >> 172.21.15.19:0/1584623529 conn(0x7f20900fd8c0 msgr2=0x7f20900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:37.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.916+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 shutdown_connections 2024-09-12T03:30:37.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:37.916+0000 7f2094e96640 1 -- 172.21.15.19:0/1584623529 wait complete. 2024-09-12T03:30:38.573 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1584623529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:38.574 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:30:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1584623529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:38.833 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:38.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1584623529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:39.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- 172.21.15.19:0/835906017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88100a60 msgr2=0x7fda88100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- 172.21.15.19:0/835906017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88100a60 0x7fda88100ec0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fda78009930 tx=0x7fda7802a420 comp rx=0 tx=0).stop 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- 172.21.15.19:0/835906017 shutdown_connections 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- 172.21.15.19:0/835906017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88101400 0x7fda8810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- 172.21.15.19:0/835906017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88100a60 0x7fda88100ec0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- 172.21.15.19:0/835906017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 0x7fda88106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- 172.21.15.19:0/835906017 >> 172.21.15.19:0/835906017 conn(0x7fda880fc780 msgr2=0x7fda880feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- 172.21.15.19:0/835906017 shutdown_connections 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- 172.21.15.19:0/835906017 wait complete. 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 Processor -- start 2024-09-12T03:30:39.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- start start 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88100a60 0x7fda881984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 0x7fda8819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda88111f80 con 0x7fda88101400 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda88111e00 con 0x7fda88100a60 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.400+0000 7fda902e6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda88112100 con 0x7fda88106a60 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41906/0 (socket says 172.21.15.19:41906) 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 -- 172.21.15.19:0/3841507394 learned_addr learned my addr 172.21.15.19:0/3841507394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8e883640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 0x7fda8819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8e082640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88100a60 0x7fda881984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 -- 172.21.15.19:0/3841507394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 msgr2=0x7fda8819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 0x7fda8819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 -- 172.21.15.19:0/3841507394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88100a60 msgr2=0x7fda881984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88100a60 0x7fda881984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 -- 172.21.15.19:0/3841507394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda8819d4a0 con 0x7fda88101400 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda8d881640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fda78009230 tx=0x7fda78004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda7f7fe640 1 -- 172.21.15.19:0/3841507394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7803f030 con 0x7fda88101400 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda7f7fe640 1 -- 172.21.15.19:0/3841507394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda78003ad0 con 0x7fda88101400 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda8819d730 con 0x7fda88101400 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda7f7fe640 1 -- 172.21.15.19:0/3841507394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78035070 con 0x7fda88101400 2024-09-12T03:30:39.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.401+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda881a5070 con 0x7fda88101400 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.404+0000 7fda926bf740 1 Processor -- start 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 -- start start 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda7f7fe640 1 -- 172.21.15.19:0/3841507394 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda78042030 con 0x7fda88101400 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d305e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d306b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5647d2fc42f0 con 0x5647d305e5d0 2024-09-12T03:30:39.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5647d2f6bcf0 con 0x5647d305fb40 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda926bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647d2fc02b0 con 0x5647d3069370 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41910/0 (socket says 172.21.15.19:41910) 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 -- 172.21.15.19:0/2485149283 learned_addr learned my addr 172.21.15.19:0/2485149283 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e883640 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d305e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8d881640 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d306b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 -- 172.21.15.19:0/2485149283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 msgr2=0x5647d306b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d306b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 -- 172.21.15.19:0/2485149283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 msgr2=0x5647d305e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.422 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d305e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.405+0000 7fda8e082640 1 -- 172.21.15.19:0/2485149283 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d2f503a0 con 0x5647d305e5d0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda7f7fe640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda8e883640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda7f7fe640 1 -- 172.21.15.19:0/3841507394 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fda780ef470 con 0x7fda88101400 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda8e082640 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fda8400ab90 tx=0x7fda8400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b50c84d5bf1d65 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda7cdea640 1 -- 172.21.15.19:0/2485149283 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8401d070 con 0x5647d305e5d0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda7cdea640 1 -- 172.21.15.19:0/2485149283 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda84018b20 con 0x5647d305e5d0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.406+0000 7fda7cdea640 1 -- 172.21.15.19:0/2485149283 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84010b10 con 0x5647d305e5d0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- 172.21.15.19:0/2485149283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 msgr2=0x5647d305ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda8e883640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 0x7fda740a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda80005fd0 tx=0x7fda800096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.423 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fda8400ab90 tx=0x7fda8400f790 comp rx=0 tx=0).stop 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- 172.21.15.19:0/2485149283 shutdown_connections 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d306b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d305ea10 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 --2- 172.21.15.19:0/2485149283 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d305e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- 172.21.15.19:0/2485149283 >> 172.21.15.19:0/2485149283 conn(0x5647d3027310 msgr2=0x5647d305ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- 172.21.15.19:0/2485149283 shutdown_connections 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- 172.21.15.19:0/2485149283 wait complete. 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 -- start start 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.407+0000 7fda926bf740 1 Processor -- start 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d3066020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.424 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d3066610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5647d306c070 con 0x5647d305e5d0 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5647d2f7f060 con 0x5647d305fb40 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5647d306c1f0 con 0x5647d3069370 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda926bf740 10 client.0 _ll_register_callbacks cb 0x5647d2cd28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41920/0 (socket says 172.21.15.19:41920) 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 -- 172.21.15.19:0/370999637 learned_addr learned my addr 172.21.15.19:0/370999637 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8d881640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d3066020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e883640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d3066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 -- 172.21.15.19:0/370999637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 msgr2=0x5647d3066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.425 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d3066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 -- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 msgr2=0x5647d3066020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d3066020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 -- 172.21.15.19:0/370999637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d2d353c0 con 0x5647d305e5d0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e883640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d3066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda8e082640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fda84010470 tx=0x7fda84018670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3da3e474b6364 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.408+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8401d050 con 0x5647d305e5d0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fda84010a90 con 0x5647d305e5d0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda84018b20 con 0x5647d305e5d0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fda8400ae90 con 0x5647d305e5d0 2024-09-12T03:30:39.426 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 10 client.17682 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5647d3077b10 con 0x5647d305e5d0 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda84018b20 con 0x5647d305e5d0 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fda840047e0 con 0x5647d305e5d0 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 10 client.17682 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 10 client.17682 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 20 client.17682 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d3078e60 0x5647d307b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 4 client.17682 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.409+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647d2fd1860 con 0x5647d3078e60 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.410+0000 7fda8d881640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d3078e60 0x5647d307b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.427 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.410+0000 7fda8d881640 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d3078e60 0x5647d307b250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58b6efd7419dfc64 server_cookie=6d575757ccf2ffd7 in_seq=0 out_seq=0 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.410+0000 7fda71ffb640 10 client.17682 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda71ffb640 1 -- 172.21.15.19:0/370999637 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647d2fd1860 con 0x5647d3078e60 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda71ffb640 10 client.17682 handle_command_reply: tid=0 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 client.17682 shutdown 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 msgr2=0x5647d3068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fda84010470 tx=0x7fda84018670 comp rx=0 tx=0).stop 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 shutdown_connections 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d3078e60 0x5647d307b250 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d3066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d3066020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda71ffb640 0 client.17682 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:30:39.428 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 --2- 172.21.15.19:0/370999637 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d3068d70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 >> 172.21.15.19:0/370999637 conn(0x5647d3027310 msgr2=0x5647d3060930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 shutdown_connections 2024-09-12T03:30:39.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 1 -- 172.21.15.19:0/370999637 wait complete. 2024-09-12T03:30:39.429 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.411+0000 7fda926bf740 20 client.17682 trim_cache size 0 max 16384 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 Processor -- start 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 -- start start 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305e5d0 0x5647d30594e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 0x5647d3061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ad670 con 0x5647d3069370 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda881af3f0 con 0x5647d305e5d0 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda926bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda881add40 con 0x5647d305fb40 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda8e883640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.434 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda8e883640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41930/0 (socket says 172.21.15.19:41930) 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda8e883640 1 -- 172.21.15.19:0/1012750635 learned_addr learned my addr 172.21.15.19:0/1012750635 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda8e082640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305e5d0 0x5647d30594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.425+0000 7fda8d881640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 0x5647d3061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 -- 172.21.15.19:0/1012750635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 msgr2=0x5647d3061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 0x5647d3061120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 -- 172.21.15.19:0/1012750635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305e5d0 msgr2=0x5647d30594e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305e5d0 0x5647d30594e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 -- 172.21.15.19:0/1012750635 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647d305a390 con 0x5647d3069370 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8d881640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 0x5647d3061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:39.435 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda8e883640 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fda80005c40 tx=0x7fda80016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e45bc93cfd04ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda73fff640 1 -- 172.21.15.19:0/1012750635 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8002d070 con 0x5647d3069370 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda73fff640 1 -- 172.21.15.19:0/1012750635 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda800288f0 con 0x5647d3069370 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.426+0000 7fda73fff640 1 -- 172.21.15.19:0/1012750635 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda8001c8d0 con 0x5647d3069370 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- 172.21.15.19:0/1012750635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 msgr2=0x5647d3061660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fda80005c40 tx=0x7fda80016000 comp rx=0 tx=0).stop 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- 172.21.15.19:0/1012750635 shutdown_connections 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d3069370 0x5647d3061660 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d305fb40 0x5647d3061120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 --2- 172.21.15.19:0/1012750635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305e5d0 0x5647d30594e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- 172.21.15.19:0/1012750635 >> 172.21.15.19:0/1012750635 conn(0x5647d3027310 msgr2=0x5647d30680a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.436 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- 172.21.15.19:0/1012750635 shutdown_connections 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- 172.21.15.19:0/1012750635 wait complete. 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 -- start start 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.427+0000 7fda926bf740 1 Processor -- start 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d306da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d30e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fda881aec60 con 0x5647d305e5d0 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ae4d0 con 0x5647d305fb40 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fda881ab590 con 0x5647d3069370 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 10 client.0 _ll_register_callbacks cb 0x7fda8401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:39.437 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41946/0 (socket says 172.21.15.19:41946) 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 -- 172.21.15.19:0/1182707196 learned_addr learned my addr 172.21.15.19:0/1182707196 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8d881640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d306da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e883640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d30e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 -- 172.21.15.19:0/1182707196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 msgr2=0x5647d30e28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d30e28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 -- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 msgr2=0x5647d306da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d306da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 -- 172.21.15.19:0/1182707196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647d30690e0 con 0x5647d305e5d0 2024-09-12T03:30:39.438 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8d881640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d306da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda8e082640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fda8400a5f0 tx=0x7fda840186e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90706fc0c9b2707e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda71ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda84035080 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fda84018c70 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.428+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda800288f0 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fda8400ae90 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 10 client.17688 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5647d3066980 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda800288f0 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fda84004ef0 con 0x5647d305e5d0 2024-09-12T03:30:39.439 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 10 client.17688 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 10 client.17688 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 20 client.17688 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d30e4330 0x5647d30e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 4 client.17688 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647d2fcbd70 con 0x5647d30e4330 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.429+0000 7fda8d881640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d30e4330 0x5647d30e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda8d881640 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d30e4330 0x5647d30e6750 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a3ba72933c890ca0 server_cookie=4d0f3ee093966409 in_seq=0 out_seq=0 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda71ffb640 10 client.17688 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda71ffb640 1 -- 172.21.15.19:0/1182707196 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647d2fcbd70 con 0x5647d30e4330 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda71ffb640 10 client.17688 handle_command_reply: tid=0 2024-09-12T03:30:39.440 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 client.17688 shutdown 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 msgr2=0x5647d306d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fda8400a5f0 tx=0x7fda840186e0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 shutdown_connections 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5647d30e4330 0x5647d30e6750 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5647d3069370 0x5647d30e28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5647d305fb40 0x5647d306da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda71ffb640 0 client.17688 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 --2- 172.21.15.19:0/1182707196 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5647d305e5d0 0x5647d306d500 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 >> 172.21.15.19:0/1182707196 conn(0x5647d3027310 msgr2=0x5647d30647c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.441 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 shutdown_connections 2024-09-12T03:30:39.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.430+0000 7fda926bf740 1 -- 172.21.15.19:0/1182707196 wait complete. 2024-09-12T03:30:39.442 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda926bf740 20 client.17688 trim_cache size 0 max 16384 2024-09-12T03:30:39.442 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 msgr2=0x7fda740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 0x7fda740a5040 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fda80005fd0 tx=0x7fda800096a0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 msgr2=0x7fda88198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:39.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fda78009230 tx=0x7fda78004270 comp rx=0 tx=0).stop 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 shutdown_connections 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fda88106a60 0x7fda8819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fda740a2b80 0x7fda740a5040 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fda88101400 0x7fda88198a30 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 --2- 172.21.15.19:0/3841507394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fda88100a60 0x7fda881984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 >> 172.21.15.19:0/3841507394 conn(0x7fda880fc780 msgr2=0x7fda880fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 shutdown_connections 2024-09-12T03:30:39.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:39.431+0000 7fda902e6640 1 -- 172.21.15.19:0/3841507394 wait complete. 2024-09-12T03:30:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:39 smithi019 ceph-mon[30953]: pgmap v685: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 517 KiB/s wr, 93 op/s 2024-09-12T03:30:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:39 smithi135 ceph-mon[35474]: pgmap v685: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 517 KiB/s wr, 93 op/s 2024-09-12T03:30:39.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:39 smithi121 ceph-mon[34963]: pgmap v685: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 517 KiB/s wr, 93 op/s 2024-09-12T03:30:40.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:30:40.474 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:40.595 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:40 smithi019 ceph-mon[30953]: pgmap v686: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 35 op/s 2024-09-12T03:30:40.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:40 smithi135 ceph-mon[35474]: pgmap v686: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 35 op/s 2024-09-12T03:30:40.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:40 smithi121 ceph-mon[34963]: pgmap v686: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 35 op/s 2024-09-12T03:30:41.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 -- 172.21.15.19:0/3380546363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c101420 msgr2=0x7f129c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/3380546363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c101420 0x7f129c10d950 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1294009cd0 tx=0x7f129402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 -- 172.21.15.19:0/3380546363 shutdown_connections 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/3380546363 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c101420 0x7f129c10d950 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/3380546363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c100a80 0x7f129c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/3380546363 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c106a80 0x7f129c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.986+0000 7f12a3c7e640 1 -- 172.21.15.19:0/3380546363 >> 172.21.15.19:0/3380546363 conn(0x7f129c0fc7a0 msgr2=0x7f129c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- 172.21.15.19:0/3380546363 shutdown_connections 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- 172.21.15.19:0/3380546363 wait complete. 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 Processor -- start 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- start start 2024-09-12T03:30:41.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c100a80 0x7f129c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c101420 0x7f129c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f129c110e80 con 0x7f129c106a80 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f129c110d00 con 0x7f129c101420 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.987+0000 7f12a3c7e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f129c111000 con 0x7f129c100a80 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51100/0 (socket says 172.21.15.19:51100) 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 -- 172.21.15.19:0/4063596310 learned_addr learned my addr 172.21.15.19:0/4063596310 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a1219640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c101420 0x7f129c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c100a80 0x7f129c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 -- 172.21.15.19:0/4063596310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c100a80 msgr2=0x7f129c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c100a80 0x7f129c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 -- 172.21.15.19:0/4063596310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c101420 msgr2=0x7f129c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c101420 0x7f129c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 -- 172.21.15.19:0/4063596310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c19d380 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a221b640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1294005cc0 tx=0x7f1294003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f128effd640 1 -- 172.21.15.19:0/4063596310 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294004040 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c19d610 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f128effd640 1 -- 172.21.15.19:0/4063596310 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12940046c0 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f128effd640 1 -- 172.21.15.19:0/4063596310 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1294046480 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.988+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1a4f50 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f128effd640 1 -- 172.21.15.19:0/4063596310 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12940305a0 con 0x7f129c106a80 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 Processor -- start 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 -- start start 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1eb40 0x55fd29b1d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 0x55fd29b2a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd29a832f0 con 0x55fd29b1d5d0 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd29a2acf0 con 0x55fd29b28370 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a6057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd29a7f2b0 con 0x55fd29b1eb40 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a1a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a1a1a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51104/0 (socket says 172.21.15.19:51104) 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.991+0000 7f12a1a1a640 1 -- 172.21.15.19:0/2859215560 learned_addr learned my addr 172.21.15.19:0/2859215560 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1219640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1eb40 0x55fd29b1d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f128effd640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 0x7f12880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1219640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 0x55fd29b2a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 -- 172.21.15.19:0/2859215560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1eb40 msgr2=0x55fd29b1d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1eb40 0x55fd29b1d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 -- 172.21.15.19:0/2859215560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 msgr2=0x55fd29b2a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 0x55fd29b2a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 -- 172.21.15.19:0/2859215560 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd29a0f3a0 con 0x55fd29b1d5d0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1219640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 0x55fd29b2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a221b640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 0x7f12880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f129800ab90 tx=0x7f129800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5fe1e54e5392e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f128effd640 1 -- 172.21.15.19:0/4063596310 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f129402e030 con 0x7f129c106a80 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f1287fff640 1 -- 172.21.15.19:0/2859215560 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129801d070 con 0x55fd29b1d5d0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f1287fff640 1 -- 172.21.15.19:0/2859215560 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298018b20 con 0x55fd29b1d5d0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.992+0000 7f1287fff640 1 -- 172.21.15.19:0/2859215560 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1298010b10 con 0x55fd29b1d5d0 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- 172.21.15.19:0/2859215560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 msgr2=0x55fd29b1da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a221b640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 0x7f12880a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12940bcb30 tx=0x7f129403d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f129800ab90 tx=0x7f129800f790 comp rx=0 tx=0).stop 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- 172.21.15.19:0/2859215560 shutdown_connections 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b28370 0x55fd29b2a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b1da10 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 --2- 172.21.15.19:0/2859215560 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1eb40 0x55fd29b1d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- 172.21.15.19:0/2859215560 >> 172.21.15.19:0/2859215560 conn(0x55fd29ae6310 msgr2=0x55fd29b19c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- 172.21.15.19:0/2859215560 shutdown_connections 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- 172.21.15.19:0/2859215560 wait complete. 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.993+0000 7f12a6057740 1 -- start start 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 Processor -- start 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55fd29b2b070 con 0x55fd29b28370 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55fd29a3e060 con 0x55fd29b1eb40 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55fd29b2b1f0 con 0x55fd29b1d5d0 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a6057740 10 client.0 _ll_register_callbacks cb 0x55fd297918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51118/0 (socket says 172.21.15.19:51118) 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 -- 172.21.15.19:0/273146641 learned_addr learned my addr 172.21.15.19:0/273146641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 -- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 msgr2=0x55fd29b27d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 -- 172.21.15.19:0/273146641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 msgr2=0x55fd29b25030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b25030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.994+0000 7f12a1219640 1 -- 172.21.15.19:0/273146641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd297f43c0 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a1219640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f129000ba20 tx=0x7f129000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a19fc1357823b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290012650 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12900127f0 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1298018b20 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f12900129f0 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 10 client.17700 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fd29b36b60 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1298018b20 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f12900113e0 con 0x55fd29b28370 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 10 client.17700 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 10 client.17700 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:30:41.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 20 client.17700 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29b38070 0x55fd29b3a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 4 client.17700 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.995+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd29a90860 con 0x55fd29b38070 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.996+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29b38070 0x55fd29b3a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.996+0000 7f12857fa640 10 client.17700 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.996+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29b38070 0x55fd29b3a4b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d080e2d9b95a45bb server_cookie=c908cc219d0449e in_seq=0 out_seq=0 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12857fa640 1 -- 172.21.15.19:0/273146641 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd29a90860 con 0x55fd29b38070 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12857fa640 10 client.17700 handle_command_reply: tid=0 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 client.17700 shutdown 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 msgr2=0x55fd29b25640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f129000ba20 tx=0x7f129000bee0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 shutdown_connections 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29b38070 0x55fd29b3a4b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b25640 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b25030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 --2- 172.21.15.19:0/273146641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b27d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12857fa640 0 client.17700 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 >> 172.21.15.19:0/273146641 conn(0x55fd29ae6310 msgr2=0x55fd29b1f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 shutdown_connections 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 1 -- 172.21.15.19:0/273146641 wait complete. 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:40.997+0000 7f12a6057740 20 client.17700 trim_cache size 0 max 16384 2024-09-12T03:30:41.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 Processor -- start 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 -- start start 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b26fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1ad4a0 con 0x55fd29b28370 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1af220 con 0x55fd29b1eb40 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a6057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1adb70 con 0x55fd29b1d5d0 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51128/0 (socket says 172.21.15.19:51128) 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 -- 172.21.15.19:0/2460295015 learned_addr learned my addr 172.21.15.19:0/2460295015 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b26fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 -- 172.21.15.19:0/2460295015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 msgr2=0x55fd29b26fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b26fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 -- 172.21.15.19:0/2460295015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 msgr2=0x55fd29b20120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b20120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 -- 172.21.15.19:0/2460295015 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd29b3ab30 con 0x55fd29b28370 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b20120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.011+0000 7f12a1219640 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1290004570 tx=0x7f1290000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850b0db51d37e955 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12877fe640 1 -- 172.21.15.19:0/2460295015 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290011a70 con 0x55fd29b28370 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12877fe640 1 -- 172.21.15.19:0/2460295015 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290018030 con 0x55fd29b28370 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12877fe640 1 -- 172.21.15.19:0/2460295015 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290028450 con 0x55fd29b28370 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 -- 172.21.15.19:0/2460295015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 msgr2=0x55fd29b20660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1290004570 tx=0x7f1290000bc0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 -- 172.21.15.19:0/2460295015 shutdown_connections 2024-09-12T03:30:41.031 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b28370 0x55fd29b20660 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b20120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 --2- 172.21.15.19:0/2460295015 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b1d5d0 0x55fd29b26fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 -- 172.21.15.19:0/2460295015 >> 172.21.15.19:0/2460295015 conn(0x55fd29ae6310 msgr2=0x55fd29b26890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 -- 172.21.15.19:0/2460295015 shutdown_connections 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.012+0000 7f12a6057740 1 -- 172.21.15.19:0/2460295015 wait complete. 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 -- start start 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 Processor -- start 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b28370 0x55fd29b21f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1aea90 con 0x55fd29b1d5d0 2024-09-12T03:30:41.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1ae300 con 0x55fd29b1eb40 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1ab3c0 con 0x55fd29b28370 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a6057740 10 client.0 _ll_register_callbacks cb 0x7f1290004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a1219640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51142/0 (socket says 172.21.15.19:51142) 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a1219640 1 -- 172.21.15.19:0/4182988670 learned_addr learned my addr 172.21.15.19:0/4182988670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.013+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b28370 0x55fd29b21f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 -- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b28370 msgr2=0x55fd29b21f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b28370 0x55fd29b21f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 -- 172.21.15.19:0/4182988670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 msgr2=0x55fd29b219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 -- 172.21.15.19:0/4182988670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd29b22470 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a1219640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1290011920 tx=0x7f1290001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e27bc48ecd71ede server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12857fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290040060 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1290011c70 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1290018030 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1290012450 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a6057740 10 client.17706 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fd29ba29b0 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.014+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290018030 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f12900046c0 con 0x55fd29b1d5d0 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 10 client.17706 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:41.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 10 client.17706 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 20 client.17706 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29ba2f60 0x55fd29ba5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 4 client.17706 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fd29a8ad70 con 0x55fd29ba2f60 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29ba2f60 0x55fd29ba5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12a1a1a640 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29ba2f60 0x55fd29ba5350 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a56f1a892cb47d7c server_cookie=dda1f00a6ea8365d in_seq=0 out_seq=0 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12857fa640 10 client.17706 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.015+0000 7f12857fa640 1 -- 172.21.15.19:0/4182988670 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fd29a8ad70 con 0x55fd29ba2f60 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12857fa640 10 client.17706 handle_command_reply: tid=0 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 client.17706 shutdown 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 msgr2=0x55fd29b308a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1290011920 tx=0x7f1290001e50 comp rx=0 tx=0).stop 2024-09-12T03:30:41.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 shutdown_connections 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55fd29ba2f60 0x55fd29ba5350 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55fd29b28370 0x55fd29b21f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55fd29b1eb40 0x55fd29b219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12857fa640 0 client.17706 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 --2- 172.21.15.19:0/4182988670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55fd29b1d5d0 0x55fd29b308a0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 >> 172.21.15.19:0/4182988670 conn(0x55fd29ae6310 msgr2=0x55fd29b33300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 shutdown_connections 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 1 -- 172.21.15.19:0/4182988670 wait complete. 2024-09-12T03:30:41.036 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a6057740 20 client.17706 trim_cache size 0 max 16384 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 msgr2=0x7f12880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 0x7f12880a5040 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f12940bcb30 tx=0x7f129403d040 comp rx=0 tx=0).stop 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 msgr2=0x7f129c19cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.016+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1294005cc0 tx=0x7f1294003970 comp rx=0 tx=0).stop 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 shutdown_connections 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12880a2b80 0x7f12880a5040 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f129c106a80 0x7f129c19cca0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.037 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f129c101420 0x7f129c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 --2- 172.21.15.19:0/4063596310 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f129c100a80 0x7f129c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 >> 172.21.15.19:0/4063596310 conn(0x7f129c0fc7a0 msgr2=0x7f129c06a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 shutdown_connections 2024-09-12T03:30:41.038 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:41.017+0000 7f12a3c7e640 1 -- 172.21.15.19:0/4063596310 wait complete. 2024-09-12T03:30:41.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:30:41.977 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 -- 172.21.15.19:0/100055737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 msgr2=0x7f86e41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 --2- 172.21.15.19:0/100055737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e41021c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009900 tx=0x7f86e002a410 comp rx=0 tx=0).stop 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 -- 172.21.15.19:0/100055737 shutdown_connections 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 --2- 172.21.15.19:0/100055737 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 0x7f86e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 --2- 172.21.15.19:0/100055737 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 0x7f86e41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 --2- 172.21.15.19:0/100055737 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e41021c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.503+0000 7f86ebac9640 1 -- 172.21.15.19:0/100055737 >> 172.21.15.19:0/100055737 conn(0x7f86e40fd8c0 msgr2=0x7f86e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.504+0000 7f86ebac9640 1 -- 172.21.15.19:0/100055737 shutdown_connections 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.504+0000 7f86ebac9640 1 -- 172.21.15.19:0/100055737 wait complete. 2024-09-12T03:30:42.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.504+0000 7f86ebac9640 1 Processor -- start 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.504+0000 7f86ebac9640 1 -- start start 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 0x7f86e4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 0x7f86e419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111f70 con 0x7f86e4101de0 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86e4111df0 con 0x7f86e4102790 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ebac9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41120f0 con 0x7f86e4106f10 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51154/0 (socket says 172.21.15.19:51154) 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 -- 172.21.15.19:0/1749924290 learned_addr learned my addr 172.21.15.19:0/1749924290 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:42.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86ea066640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 0x7f86e419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9064640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 0x7f86e4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 -- 172.21.15.19:0/1749924290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 msgr2=0x7f86e419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 0x7f86e419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 -- 172.21.15.19:0/1749924290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 msgr2=0x7f86e4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 0x7f86e4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 -- 172.21.15.19:0/1749924290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e419d3e0 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.505+0000 7f86e9865640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86e0006010 tx=0x7f86e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.506+0000 7f86daffd640 1 -- 172.21.15.19:0/1749924290 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e003f030 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.506+0000 7f86daffd640 1 -- 172.21.15.19:0/1749924290 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86e0003ce0 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.506+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e419d670 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.506+0000 7f86daffd640 1 -- 172.21.15.19:0/1749924290 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0035070 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.506+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e41a4fb0 con 0x7f86e4101de0 2024-09-12T03:30:42.523 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.508+0000 7f86daffd640 1 -- 172.21.15.19:0/1749924290 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86e0042030 con 0x7f86e4101de0 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.508+0000 7f86edea2740 1 Processor -- start 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 -- start start 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5613677082f0 con 0x5613677a25d0 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5613676afcf0 con 0x5613677ad370 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86edea2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613677042b0 con 0x5613677a3b40 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86daffd640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 0x7f86d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51168/0 (socket says 172.21.15.19:51168) 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9865640 1 -- 172.21.15.19:0/1412173416 learned_addr learned my addr 172.21.15.19:0/1412173416 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9064640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86ea066640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86ea066640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 0x7f86d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86daffd640 1 -- 172.21.15.19:0/1749924290 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86e00ef6a0 con 0x7f86e4101de0 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9064640 1 -- 172.21.15.19:0/1412173416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 msgr2=0x5613677a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.509+0000 7f86e9064640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86e9064640 1 -- 172.21.15.19:0/1412173416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 msgr2=0x5613677a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86e9064640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86e9064640 1 -- 172.21.15.19:0/1412173416 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613676943a0 con 0x5613677ad370 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86e9865640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86ea066640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 0x7f86d00a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009fd0 tx=0x7f86dc009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86e9064640 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677af760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0a1cd791b251a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86cffff640 1 -- 172.21.15.19:0/1412173416 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4004020 con 0x5613677ad370 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86cffff640 1 -- 172.21.15.19:0/1412173416 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d40041c0 con 0x5613677ad370 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.510+0000 7f86cffff640 1 -- 172.21.15.19:0/1412173416 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d40186a0 con 0x5613677ad370 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- 172.21.15.19:0/1412173416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 msgr2=0x5613677af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677af760 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f86d40058c0 tx=0x7f86d400f3c0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- 172.21.15.19:0/1412173416 shutdown_connections 2024-09-12T03:30:42.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677af760 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 --2- 172.21.15.19:0/1412173416 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- 172.21.15.19:0/1412173416 >> 172.21.15.19:0/1412173416 conn(0x56136776b310 msgr2=0x56136779ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- 172.21.15.19:0/1412173416 shutdown_connections 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- 172.21.15.19:0/1412173416 wait complete. 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.511+0000 7f86edea2740 1 -- start start 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 Processor -- start 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5613677b0070 con 0x5613677ad370 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5613676c3060 con 0x5613677a25d0 2024-09-12T03:30:42.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5613677b01f0 con 0x5613677a3b40 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86edea2740 10 client.0 _ll_register_callbacks cb 0x5613674168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51174/0 (socket says 172.21.15.19:51174) 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 -- 172.21.15.19:0/363078287 learned_addr learned my addr 172.21.15.19:0/363078287 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86e9865640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86e9064640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 -- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 msgr2=0x5613677a9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.528 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 -- 172.21.15.19:0/363078287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 msgr2=0x5613677acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86ea066640 1 -- 172.21.15.19:0/363078287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613674793c0 con 0x5613677ad370 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86e9064640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.512+0000 7f86e9865640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86ea066640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00ab90 tx=0x7f86dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9b20b044938fa8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc01ebb0 con 0x5613677ad370 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86dc01ed50 con 0x5613677ad370 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d40041c0 con 0x5613677ad370 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f86dc004b20 con 0x5613677ad370 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86edea2740 10 client.17718 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:42.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5613677bba80 con 0x5613677ad370 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d40041c0 con 0x5613677ad370 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f86dc00f040 con 0x5613677ad370 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86edea2740 10 client.17718 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.513+0000 7f86edea2740 10 client.17718 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86edea2740 20 client.17718 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5613677bd000 0x5613677bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86edea2740 4 client.17718 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561367715860 con 0x5613677bd000 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86e9865640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5613677bd000 0x5613677bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86e9865640 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5613677bd000 0x5613677bf4a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8e1028d755bdfd2 server_cookie=42e9bc04f2c7f208 in_seq=0 out_seq=0 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.514+0000 7f86cd7fa640 10 client.17718 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.515+0000 7f86cd7fa640 1 -- 172.21.15.19:0/363078287 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561367715860 con 0x5613677bd000 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.515+0000 7f86cd7fa640 10 client.17718 handle_command_reply: tid=0 2024-09-12T03:30:42.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.515+0000 7f86edea2740 1 client.17718 shutdown 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 msgr2=0x5613677aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00ab90 tx=0x7f86dc004a40 comp rx=0 tx=0).stop 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 shutdown_connections 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5613677bd000 0x5613677bf4a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677ad370 0x5613677aa4f0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 --2- 172.21.15.19:0/363078287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86cd7fa640 0 client.17718 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:30:42.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 >> 172.21.15.19:0/363078287 conn(0x56136776b310 msgr2=0x5613677a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 shutdown_connections 2024-09-12T03:30:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 1 -- 172.21.15.19:0/363078287 wait complete. 2024-09-12T03:30:42.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.516+0000 7f86edea2740 20 client.17718 trim_cache size 0 max 16384 2024-09-12T03:30:42.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.529+0000 7f86edea2740 1 Processor -- start 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.529+0000 7f86edea2740 1 -- start start 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 0x5613677a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41ad630 con 0x5613677a3b40 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41af3b0 con 0x5613677a25d0 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86edea2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41add00 con 0x5613677ad370 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51188/0 (socket says 172.21.15.19:51188) 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 -- 172.21.15.19:0/2738149944 learned_addr learned my addr 172.21.15.19:0/2738149944 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9865640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86ea066640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 0x5613677a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 -- 172.21.15.19:0/2738149944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 msgr2=0x5613677a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 0x5613677a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 -- 172.21.15.19:0/2738149944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 msgr2=0x5613677b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 -- 172.21.15.19:0/2738149944 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56136779f960 con 0x5613677a3b40 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86ea066640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 0x5613677a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86e9064640 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60ecb39f248d8d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.530+0000 7f86cf7fe640 1 -- 172.21.15.19:0/2738149944 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d400fd70 con 0x5613677a3b40 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86cf7fe640 1 -- 172.21.15.19:0/2738149944 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4005520 con 0x5613677a3b40 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86cf7fe640 1 -- 172.21.15.19:0/2738149944 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d4018440 con 0x5613677a3b40 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 -- 172.21.15.19:0/2738149944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 msgr2=0x5613677a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4005dc0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.538 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 -- 172.21.15.19:0/2738149944 shutdown_connections 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677ad370 0x5613677a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a3b40 0x5613677a5120 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 --2- 172.21.15.19:0/2738149944 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677a25d0 0x5613677b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 -- 172.21.15.19:0/2738149944 >> 172.21.15.19:0/2738149944 conn(0x56136776b310 msgr2=0x5613677a9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 -- 172.21.15.19:0/2738149944 shutdown_connections 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.531+0000 7f86edea2740 1 -- 172.21.15.19:0/2738149944 wait complete. 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 -- start start 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 Processor -- start 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677a7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41aec20 con 0x5613677a25d0 2024-09-12T03:30:42.539 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41ae490 con 0x5613677ad370 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86e41ab550 con 0x5613677a3b40 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86edea2740 10 client.0 _ll_register_callbacks cb 0x7f86dc012760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86e9865640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51196/0 (socket says 172.21.15.19:51196) 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86e9865640 1 -- 172.21.15.19:0/163117884 learned_addr learned my addr 172.21.15.19:0/163117884 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86ea066640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677a7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.532+0000 7f86e9064640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 -- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 msgr2=0x5613677a6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 -- 172.21.15.19:0/163117884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 msgr2=0x5613677a7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677a7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.540 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 -- 172.21.15.19:0/163117884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613677a7a30 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86e9865640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b8480 tx=0x7f86e00bbe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4630e9d2d1cb9cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86cd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0049040 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86e00ed260 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86d4005520 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f86e00303a0 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86edea2740 10 client.17724 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561367827d20 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.533+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86d4005520 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f86e00bc280 con 0x5613677a25d0 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 10 client.17724 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 10 client.17724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:30:42.541 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 20 client.17724 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x561367828490 0x56136782a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 4 client.17724 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56136770fd70 con 0x561367828490 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86e9064640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x561367828490 0x56136782a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86e9064640 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x561367828490 0x56136782a8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aafb2239ef1e4f57 server_cookie=4da9910bb83e2615 in_seq=0 out_seq=0 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.534+0000 7f86cd7fa640 10 client.17724 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86cd7fa640 1 -- 172.21.15.19:0/163117884 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56136770fd70 con 0x561367828490 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86cd7fa640 10 client.17724 handle_command_reply: tid=0 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 client.17724 shutdown 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 msgr2=0x5613677b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b8480 tx=0x7f86e00bbe70 comp rx=0 tx=0).stop 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 shutdown_connections 2024-09-12T03:30:42.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x561367828490 0x56136782a8b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5613677ad370 0x5613677a7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5613677a3b40 0x5613677a6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 --2- 172.21.15.19:0/163117884 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5613677a25d0 0x5613677b5c20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86cd7fa640 0 client.17724 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 >> 172.21.15.19:0/163117884 conn(0x56136776b310 msgr2=0x5613677ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 shutdown_connections 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 1 -- 172.21.15.19:0/163117884 wait complete. 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.535+0000 7f86edea2740 20 client.17724 trim_cache size 0 max 16384 2024-09-12T03:30:42.543 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 msgr2=0x7f86d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 0x7f86d00a5040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009fd0 tx=0x7f86dc009600 comp rx=0 tx=0).stop 2024-09-12T03:30:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 msgr2=0x7f86e4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f86e0006010 tx=0x7f86e00042f0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.544 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 shutdown_connections 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86e4106f10 0x7f86e419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86e4102790 0x7f86e4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f86d00a2b80 0x7f86d00a5040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 --2- 172.21.15.19:0/1749924290 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f86e4101de0 0x7f86e4198430 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 >> 172.21.15.19:0/1749924290 conn(0x7f86e40fd8c0 msgr2=0x7f86e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 shutdown_connections 2024-09-12T03:30:42.545 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:42.536+0000 7f86ebac9640 1 -- 172.21.15.19:0/1749924290 wait complete. 2024-09-12T03:30:42.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:42 smithi121 ceph-mon[34963]: pgmap v687: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-09-12T03:30:42.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:42 smithi019 ceph-mon[30953]: pgmap v687: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-09-12T03:30:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:42 smithi135 ceph-mon[35474]: pgmap v687: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 10 KiB/s wr, 21 op/s 2024-09-12T03:30:43.330 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:30:43.596 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:43.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:30:43.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:43.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:43.883 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:43 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:43.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:43.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:43.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:43 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:44.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:30:44.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:44.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:43 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 -- 172.21.15.19:0/2914166547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103660 msgr2=0x7f26b410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/2914166547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103660 0x7f26b410bb50 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009900 tx=0x7f26a402a480 comp rx=0 tx=0).stop 2024-09-12T03:30:44.108 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 -- 172.21.15.19:0/2914166547 shutdown_connections 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/2914166547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b410c170 0x7f26b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/2914166547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b4103660 0x7f26b410bb50 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/2914166547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4102d40 0x7f26b4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 -- 172.21.15.19:0/2914166547 >> 172.21.15.19:0/2914166547 conn(0x7f26b40fc7a0 msgr2=0x7f26b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 -- 172.21.15.19:0/2914166547 shutdown_connections 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.098+0000 7f26bc4fd640 1 -- 172.21.15.19:0/2914166547 wait complete. 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 Processor -- start 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 -- start start 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b4102d40 0x7f26b4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4103660 0x7f26b4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.109 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4111f70 con 0x7f26b410c170 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26b4111df0 con 0x7f26b4102d40 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26bc4fd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41120f0 con 0x7f26b4103660 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26baa9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26baa9a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51204/0 (socket says 172.21.15.19:51204) 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26baa9a640 1 -- 172.21.15.19:0/305566543 learned_addr learned my addr 172.21.15.19:0/305566543 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26ba299640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b4102d40 0x7f26b4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.099+0000 7f26b9a98640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4103660 0x7f26b4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 -- 172.21.15.19:0/305566543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4103660 msgr2=0x7f26b4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4103660 0x7f26b4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 -- 172.21.15.19:0/305566543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b4102d40 msgr2=0x7f26b4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b4102d40 0x7f26b4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 -- 172.21.15.19:0/305566543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b419d3c0 con 0x7f26b410c170 2024-09-12T03:30:44.110 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26baa9a640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00cce0 tx=0x7f26ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26ab7fe640 1 -- 172.21.15.19:0/305566543 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00f450 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26ab7fe640 1 -- 172.21.15.19:0/305566543 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26ac00fad0 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b419d6b0 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26ab7fe640 1 -- 172.21.15.19:0/305566543 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26ac018c10 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.100+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b41a4f90 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.102+0000 7f26ab7fe640 1 -- 172.21.15.19:0/305566543 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26ac00f5f0 con 0x7f26b410c170 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 Processor -- start 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 -- start start 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a48a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a55760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559cf69ae2f0 con 0x559cf6a49b40 2024-09-12T03:30:44.111 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559cf6955cf0 con 0x559cf6a53370 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26be8d6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cf69aa2b0 con 0x559cf6a485d0 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a48090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51210/0 (socket says 172.21.15.19:51210) 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.103+0000 7f26b9a98640 1 -- 172.21.15.19:0/578626148 learned_addr learned my addr 172.21.15.19:0/578626148 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a48a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ab7fe640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 0x7f26a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26b9a98640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a55760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26baa9a640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 0x7f26a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 -- 172.21.15.19:0/578626148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 msgr2=0x559cf6a55760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a55760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 -- 172.21.15.19:0/578626148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 msgr2=0x559cf6a48090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a48090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.112 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 -- 172.21.15.19:0/578626148 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf693a3a0 con 0x559cf6a485d0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26b9a98640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ab7fe640 1 -- 172.21.15.19:0/305566543 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f26ac0cf600 con 0x7f26b410c170 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26ba299640 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a48a10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009c20 tx=0x7f26b0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a94cab272d57e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26baa9a640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 0x7f26a00a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0982d0 tx=0x7f26ac096820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.104+0000 7f26a8ff9640 1 -- 172.21.15.19:0/578626148 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004050 con 0x559cf6a485d0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26a8ff9640 1 -- 172.21.15.19:0/578626148 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0005500 con 0x559cf6a485d0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26a8ff9640 1 -- 172.21.15.19:0/578626148 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004dd0 con 0x559cf6a485d0 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 -- 172.21.15.19:0/578626148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 msgr2=0x559cf6a48a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a48a10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009c20 tx=0x7f26b0011810 comp rx=0 tx=0).stop 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 -- 172.21.15.19:0/578626148 shutdown_connections 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a55760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.113 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a48a10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 --2- 172.21.15.19:0/578626148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 -- 172.21.15.19:0/578626148 >> 172.21.15.19:0/578626148 conn(0x559cf6a11310 msgr2=0x559cf6a44c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 -- 172.21.15.19:0/578626148 shutdown_connections 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.105+0000 7f26be8d6740 1 -- 172.21.15.19:0/578626148 wait complete. 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 -- start start 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 Processor -- start 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a52d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a50610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559cf6a56070 con 0x559cf6a49b40 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559cf6969060 con 0x559cf6a53370 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559cf6a561f0 con 0x559cf6a485d0 2024-09-12T03:30:44.114 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26be8d6740 10 client.0 _ll_register_callbacks cb 0x559cf66bc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26ba299640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26ba299640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51226/0 (socket says 172.21.15.19:51226) 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26ba299640 1 -- 172.21.15.19:0/3652383775 learned_addr learned my addr 172.21.15.19:0/3652383775 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26b9a98640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.106+0000 7f26b9a98640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a50610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 -- 172.21.15.19:0/3652383775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 msgr2=0x559cf6a52d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a52d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.115 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 -- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 msgr2=0x559cf6a50610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a50610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 -- 172.21.15.19:0/3652383775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf671f3c0 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26b9a98640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26ba299640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2782da8361a6ebf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f269dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004d60 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f26b0004f00 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26b0005500 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f26b0005120 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26be8d6740 10 client.17736 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.107+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559cf6a61a80 con 0x559cf6a49b40 2024-09-12T03:30:44.116 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26b0005500 con 0x559cf6a49b40 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f26b00235f0 con 0x559cf6a49b40 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 10 client.17736 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 10 client.17736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 20 client.17736 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6a62dd0 0x559cf6a651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 4 client.17736 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cf69bb860 con 0x559cf6a62dd0 2024-09-12T03:30:44.117 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26b9a98640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6a62dd0 0x559cf6a651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f269dffb640 10 client.17736 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:30:44.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.108+0000 7f26b9a98640 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6a62dd0 0x559cf6a651c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=58fa283d3ae87c25 server_cookie=154a19ef92b16538 in_seq=0 out_seq=0 2024-09-12T03:30:44.118 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f269dffb640 1 -- 172.21.15.19:0/3652383775 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cf69bb860 con 0x559cf6a62dd0 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f269dffb640 10 client.17736 handle_command_reply: tid=0 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 client.17736 shutdown 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 msgr2=0x559cf6a50020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f26b0000c00 tx=0x7f26b00048f0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 shutdown_connections 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6a62dd0 0x559cf6a651c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a50610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a49b40 0x559cf6a50020 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 --2- 172.21.15.19:0/3652383775 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a485d0 0x559cf6a52d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 >> 172.21.15.19:0/3652383775 conn(0x559cf6a11310 msgr2=0x559cf6a4a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.109+0000 7f269dffb640 0 client.17736 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:30:44.119 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.110+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 shutdown_connections 2024-09-12T03:30:44.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.110+0000 7f26be8d6740 1 -- 172.21.15.19:0/3652383775 wait complete. 2024-09-12T03:30:44.120 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.110+0000 7f26be8d6740 20 client.17736 trim_cache size 0 max 16384 2024-09-12T03:30:44.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 Processor -- start 2024-09-12T03:30:44.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 -- start start 2024-09-12T03:30:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ad610 con 0x559cf6a485d0 2024-09-12T03:30:44.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41af390 con 0x559cf6a53370 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26be8d6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41adce0 con 0x559cf6a49b40 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51228/0 (socket says 172.21.15.19:51228) 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 -- 172.21.15.19:0/821569276 learned_addr learned my addr 172.21.15.19:0/821569276 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26ba299640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a44700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 -- 172.21.15.19:0/821569276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 msgr2=0x559cf6a44700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a44700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 -- 172.21.15.19:0/821569276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 msgr2=0x559cf6a44c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.123+0000 7f26b9a98640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a44c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f26b9a98640 1 -- 172.21.15.19:0/821569276 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cf6a45dd0 con 0x559cf6a485d0 2024-09-12T03:30:44.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f26ba299640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:44.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f26b9a98640 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009230 tx=0x7f26a402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0138bcec862e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f269ffff640 1 -- 172.21.15.19:0/821569276 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4035040 con 0x559cf6a485d0 2024-09-12T03:30:44.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f269ffff640 1 -- 172.21.15.19:0/821569276 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a4005510 con 0x559cf6a485d0 2024-09-12T03:30:44.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.124+0000 7f269ffff640 1 -- 172.21.15.19:0/821569276 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4004df0 con 0x559cf6a485d0 2024-09-12T03:30:44.129 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- 172.21.15.19:0/821569276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 msgr2=0x559cf6a5f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009230 tx=0x7f26a402a480 comp rx=0 tx=0).stop 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- 172.21.15.19:0/821569276 shutdown_connections 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a44c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 --2- 172.21.15.19:0/821569276 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5f310 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- 172.21.15.19:0/821569276 >> 172.21.15.19:0/821569276 conn(0x559cf6a11310 msgr2=0x559cf6a64de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- 172.21.15.19:0/821569276 shutdown_connections 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- 172.21.15.19:0/821569276 wait complete. 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 -- start start 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.125+0000 7f26be8d6740 1 Processor -- start 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a5bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a4c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41aec00 con 0x559cf6a485d0 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ae470 con 0x559cf6a53370 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f26b41ab530 con 0x559cf6a49b40 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26be8d6740 10 client.0 _ll_register_callbacks cb 0x7f26b0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51238/0 (socket says 172.21.15.19:51238) 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 -- 172.21.15.19:0/350985480 learned_addr learned my addr 172.21.15.19:0/350985480 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26ba299640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a5bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a4c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 -- 172.21.15.19:0/350985480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 msgr2=0x559cf6a5bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a5bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 -- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 msgr2=0x559cf6a4c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a4c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 -- 172.21.15.19:0/350985480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cf6a4d100 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f26b9a98640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f26a40092c0 tx=0x7f26a40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f54fc52abafdb32 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.126+0000 7f269dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4035040 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f26a4004f80 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26a4005510 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f26a4005180 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 10 client.17742 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x559cf6acd470 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f26a4005510 con 0x559cf6a485d0 2024-09-12T03:30:44.132 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f26a4063030 con 0x559cf6a485d0 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 10 client.17742 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 10 client.17742 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 20 client.17742 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.127+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6acdbf0 0x559cf6ad0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26be8d6740 4 client.17742 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559cf69b5d70 con 0x559cf6acdbf0 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26ba299640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6acdbf0 0x559cf6ad0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26ba299640 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6acdbf0 0x559cf6ad0010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5429475173a23a6 server_cookie=2fe35c93b018f18e in_seq=0 out_seq=0 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f269dffb640 10 client.17742 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:30:44.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f269dffb640 1 -- 172.21.15.19:0/350985480 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559cf69b5d70 con 0x559cf6acdbf0 2024-09-12T03:30:44.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f269dffb640 10 client.17742 handle_command_reply: tid=0 2024-09-12T03:30:44.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26be8d6740 1 client.17742 shutdown 2024-09-12T03:30:44.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 msgr2=0x559cf6a5b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.128+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f26a40092c0 tx=0x7f26a40049d0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 shutdown_connections 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x559cf6acdbf0 0x559cf6ad0010 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f269dffb640 0 client.17742 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559cf6a53370 0x559cf6a4c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559cf6a49b40 0x559cf6a5bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 --2- 172.21.15.19:0/350985480 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559cf6a485d0 0x559cf6a5b710 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 >> 172.21.15.19:0/350985480 conn(0x559cf6a11310 msgr2=0x559cf6a44700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 shutdown_connections 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 1 -- 172.21.15.19:0/350985480 wait complete. 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26be8d6740 20 client.17742 trim_cache size 0 max 16384 2024-09-12T03:30:44.135 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 msgr2=0x7f26a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 0x7f26a00a5040 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f26ac0982d0 tx=0x7f26ac096820 comp rx=0 tx=0).stop 2024-09-12T03:30:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 msgr2=0x7f26b419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:44.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.129+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00cce0 tx=0x7f26ac00a620 comp rx=0 tx=0).stop 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 shutdown_connections 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f26a00a2b80 0x7f26a00a5040 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f26b410c170 0x7f26b419cce0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f26b4103660 0x7f26b4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 --2- 172.21.15.19:0/305566543 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f26b4102d40 0x7f26b4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 >> 172.21.15.19:0/305566543 conn(0x7f26b40fc7a0 msgr2=0x7f26b40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 shutdown_connections 2024-09-12T03:30:44.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:44.130+0000 7f26bc4fd640 1 -- 172.21.15.19:0/305566543 wait complete. 2024-09-12T03:30:44.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:44 smithi121 ceph-mon[34963]: pgmap v688: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-09-12T03:30:44.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:44.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:44.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:44.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:44 smithi019 ceph-mon[30953]: pgmap v688: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-09-12T03:30:44.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:44.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:44.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:44.954 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:44.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:30:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:44 smithi135 ceph-mon[35474]: pgmap v688: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 15 op/s 2024-09-12T03:30:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:30:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:30:45.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:30:45.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 -- 172.21.15.19:0/1218989407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198106a80 msgr2=0x7f2198106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:45.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 --2- 172.21.15.19:0/1218989407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198106a80 0x7f2198106e60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f218c009930 tx=0x7f218c02a420 comp rx=0 tx=0).stop 2024-09-12T03:30:45.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 -- 172.21.15.19:0/1218989407 shutdown_connections 2024-09-12T03:30:45.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 --2- 172.21.15.19:0/1218989407 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198101420 0x7f219810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 --2- 172.21.15.19:0/1218989407 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 0x7f2198100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 --2- 172.21.15.19:0/1218989407 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198106a80 0x7f2198106e60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 -- 172.21.15.19:0/1218989407 >> 172.21.15.19:0/1218989407 conn(0x7f21980fc7a0 msgr2=0x7f21980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 -- 172.21.15.19:0/1218989407 shutdown_connections 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.449+0000 7f219eb73640 1 -- 172.21.15.19:0/1218989407 wait complete. 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 Processor -- start 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 -- start start 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 0x7f2198198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198106a80 0x7f219819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2198110e80 con 0x7f2198101420 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2198110d00 con 0x7f2198100a80 2024-09-12T03:30:45.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f219eb73640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f2198111000 con 0x7f2198106a80 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.450+0000 7f2197fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51254/0 (socket says 172.21.15.19:51254) 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 -- 172.21.15.19:0/2255057508 learned_addr learned my addr 172.21.15.19:0/2255057508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f219c90f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 0x7f2198198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f219d110640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198106a80 0x7f219819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 -- 172.21.15.19:0/2255057508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198106a80 msgr2=0x7f219819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198106a80 0x7f219819cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 -- 172.21.15.19:0/2255057508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 msgr2=0x7f2198198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 0x7f2198198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 -- 172.21.15.19:0/2255057508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219819d410 con 0x7f2198101420 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2197fff640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f218400c890 tx=0x7f218400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:45.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184004220 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219819d700 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21840043c0 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2184004a00 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.451+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21981a4fe0 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.452+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2198100ee0 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.453+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21840079b0 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.454+0000 7f2195ffb640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 0x7f21880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.454+0000 7f219c90f640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 0x7f21880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.454+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f21840cec80 con 0x7f2198101420 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.455+0000 7f219c90f640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 0x7f21880a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f218c009900 tx=0x7f218c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:45.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.458+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2184014030 con 0x7f2198101420 2024-09-12T03:30:45.508 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:45.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.687+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f219802d7c0 con 0x7f2198101420 2024-09-12T03:30:45.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.688+0000 7f2195ffb640 1 -- 172.21.15.19:0/2255057508 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7f218409b5a0 con 0x7f2198101420 2024-09-12T03:30:45.688 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:30:45.688 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:30:45.689 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 msgr2=0x7f21880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 0x7f21880a5040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f218c009900 tx=0x7f218c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 msgr2=0x7f21981989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f218400c890 tx=0x7f218400cd50 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 shutdown_connections 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2198106a80 0x7f219819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f21880a2b80 0x7f21880a5040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2198101420 0x7f21981989a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 --2- 172.21.15.19:0/2255057508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2198100a80 0x7f2198198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:45.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.690+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 >> 172.21.15.19:0/2255057508 conn(0x7f21980fc7a0 msgr2=0x7f219806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:45.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.691+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 shutdown_connections 2024-09-12T03:30:45.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:45.691+0000 7f219eb73640 1 -- 172.21.15.19:0/2255057508 wait complete. 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 -- 172.21.15.19:0/2056147294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 msgr2=0x7fed6c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 --2- 172.21.15.19:0/2056147294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c1021d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009900 tx=0x7fed5c02a410 comp rx=0 tx=0).stop 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 -- 172.21.15.19:0/2056147294 shutdown_connections 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 --2- 172.21.15.19:0/2056147294 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 0x7fed6c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 --2- 172.21.15.19:0/2056147294 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 0x7fed6c1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 --2- 172.21.15.19:0/2056147294 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c1021d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 -- 172.21.15.19:0/2056147294 >> 172.21.15.19:0/2056147294 conn(0x7fed6c0fd8f0 msgr2=0x7fed6c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 -- 172.21.15.19:0/2056147294 shutdown_connections 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.182+0000 7fed7100d640 1 -- 172.21.15.19:0/2056147294 wait complete. 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 Processor -- start 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 -- start start 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 0x7fed6c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 0x7fed6c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c111fa0 con 0x7fed6c101df0 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c111e20 con 0x7fed6c1027a0 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed7100d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c112120 con 0x7fed6c106f20 2024-09-12T03:30:46.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51270/0 (socket says 172.21.15.19:51270) 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed6ad9d640 1 -- 172.21.15.19:0/492280300 learned_addr learned my addr 172.21.15.19:0/492280300 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.183+0000 7fed6a59c640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 0x7fed6c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6b59e640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 0x7fed6c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 -- 172.21.15.19:0/492280300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 msgr2=0x7fed6c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 0x7fed6c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 -- 172.21.15.19:0/492280300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 msgr2=0x7fed6c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 0x7fed6c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 -- 172.21.15.19:0/492280300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c19d430 con 0x7fed6c101df0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005ab0 tx=0x7fed5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed53fff640 1 -- 172.21.15.19:0/492280300 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c03f030 con 0x7fed6c101df0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed53fff640 1 -- 172.21.15.19:0/492280300 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed5c003ce0 con 0x7fed6c101df0 2024-09-12T03:30:46.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c19d6c0 con 0x7fed6c101df0 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed53fff640 1 -- 172.21.15.19:0/492280300 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c035070 con 0x7fed6c101df0 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.184+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1a5000 con 0x7fed6c101df0 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed53fff640 1 -- 172.21.15.19:0/492280300 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed5c042030 con 0x7fed6c101df0 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 Processor -- start 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 -- start start 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 0x563ff3e02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e0d370 0x563ff3e0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ff3d682f0 con 0x563ff3e03b40 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ff3d0fcf0 con 0x563ff3e025d0 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed733e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ff3d642b0 con 0x563ff3e0d370 2024-09-12T03:30:46.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed53fff640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 0x7fed540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed6b59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed6b59e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51286/0 (socket says 172.21.15.19:51286) 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.187+0000 7fed6b59e640 1 -- 172.21.15.19:0/1047617670 learned_addr learned my addr 172.21.15.19:0/1047617670 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e0d370 0x563ff3e0f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 0x7fed540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed53fff640 1 -- 172.21.15.19:0/492280300 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed5c0ef900 con 0x7fed6c101df0 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 0x563ff3e02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 -- 172.21.15.19:0/1047617670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e0d370 msgr2=0x563ff3e0f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e0d370 0x563ff3e0f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 -- 172.21.15.19:0/1047617670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 msgr2=0x563ff3e02a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 0x563ff3e02a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 -- 172.21.15.19:0/1047617670 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff3cf43a0 con 0x563ff3e03b40 2024-09-12T03:30:46.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 0x563ff3e02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:46.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cb20 tx=0x7fed640072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e85bb2e41776f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed517fa640 1 -- 172.21.15.19:0/1047617670 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed64015070 con 0x563ff3e03b40 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed517fa640 1 -- 172.21.15.19:0/1047617670 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64016040 con 0x563ff3e03b40 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed517fa640 1 -- 172.21.15.19:0/1047617670 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6401e020 con 0x563ff3e03b40 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.188+0000 7fed6b59e640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 0x7fed540a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed6401f000 tx=0x7fed6401b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- 172.21.15.19:0/1047617670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 msgr2=0x563ff3e02090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fed6400cb20 tx=0x7fed640072b0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- 172.21.15.19:0/1047617670 shutdown_connections 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e0d370 0x563ff3e0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e025d0 0x563ff3e02a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 --2- 172.21.15.19:0/1047617670 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e02090 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- 172.21.15.19:0/1047617670 >> 172.21.15.19:0/1047617670 conn(0x563ff3dcb310 msgr2=0x563ff3dfec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- 172.21.15.19:0/1047617670 shutdown_connections 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- 172.21.15.19:0/1047617670 wait complete. 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 -- start start 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.189+0000 7fed733e6740 1 Processor -- start 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e03b40 0x563ff3e0a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e0a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563ff3e10070 con 0x563ff3e025d0 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563ff3d23060 con 0x563ff3e0d370 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563ff3e101f0 con 0x563ff3e03b40 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed733e6740 10 client.0 _ll_register_callbacks cb 0x563ff3a768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51292/0 (socket says 172.21.15.19:51292) 2024-09-12T03:30:46.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 -- 172.21.15.19:0/1216225970 learned_addr learned my addr 172.21.15.19:0/1216225970 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e0a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e03b40 0x563ff3e0a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 -- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e03b40 msgr2=0x563ff3e0a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e03b40 0x563ff3e0a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 -- 172.21.15.19:0/1216225970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 msgr2=0x563ff3e0a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e0a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 -- 172.21.15.19:0/1216225970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff3ad93c0 con 0x563ff3e025d0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0ed410 tx=0x7fed5c0b84d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6841181d7255dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.190+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c049040 con 0x563ff3e025d0 2024-09-12T03:30:46.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed5c0b87e0 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed64016040 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fed5c0d5140 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 10 client.17760 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563ff3e19aa0 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed64016040 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fed5c0dc0c0 con 0x563ff3e025d0 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 10 client.17760 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 10 client.17760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 20 client.17760 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e1af50 0x563ff3e1b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 4 client.17760 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ff3d75860 con 0x563ff3e1af50 2024-09-12T03:30:46.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.191+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e1af50 0x563ff3e1b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.192+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e1af50 0x563ff3e1b3a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d60cf05a7ed1ef server_cookie=dbe3ead6b1115931 in_seq=0 out_seq=0 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.192+0000 7fed4eff5640 10 client.17760 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.192+0000 7fed4eff5640 1 -- 172.21.15.19:0/1216225970 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ff3d75860 con 0x563ff3e1af50 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.192+0000 7fed4eff5640 10 client.17760 handle_command_reply: tid=0 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 client.17760 shutdown 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 msgr2=0x563ff3e0cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0ed410 tx=0x7fed5c0b84d0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 shutdown_connections 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e1af50 0x563ff3e1b3a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e0a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e03b40 0x563ff3e0a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 --2- 172.21.15.19:0/1216225970 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e025d0 0x563ff3e0cd80 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 >> 172.21.15.19:0/1216225970 conn(0x563ff3dcb310 msgr2=0x563ff3e04930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed4eff5640 0 client.17760 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 shutdown_connections 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 1 -- 172.21.15.19:0/1216225970 wait complete. 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.193+0000 7fed733e6740 20 client.17760 trim_cache size 0 max 16384 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 Processor -- start 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 -- start start 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e08f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e2d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1ab400 con 0x563ff3e03b40 2024-09-12T03:30:46.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1ad180 con 0x563ff3e0d370 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed733e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1abad0 con 0x563ff3e025d0 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51304/0 (socket says 172.21.15.19:51304) 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed6a59c640 1 -- 172.21.15.19:0/1230655374 learned_addr learned my addr 172.21.15.19:0/1230655374 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e08f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.206+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e2d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 -- 172.21.15.19:0/1230655374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 msgr2=0x563ff3e08f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e08f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 -- 172.21.15.19:0/1230655374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 msgr2=0x563ff3e2d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e2d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 -- 172.21.15.19:0/1230655374 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ff3e022d0 con 0x563ff3e03b40 2024-09-12T03:30:46.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6b59e640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed6a59c640 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x563ff3e03070 tx=0x7fed6000abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a80e2971d3669ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed50ff9640 1 -- 172.21.15.19:0/1230655374 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6001d470 con 0x563ff3e03b40 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed50ff9640 1 -- 172.21.15.19:0/1230655374 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6001daf0 con 0x563ff3e03b40 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed50ff9640 1 -- 172.21.15.19:0/1230655374 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6000fc80 con 0x563ff3e03b40 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 -- 172.21.15.19:0/1230655374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 msgr2=0x563ff3dfdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x563ff3e03070 tx=0x7fed6000abe0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 -- 172.21.15.19:0/1230655374 shutdown_connections 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e2d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3dfdbf0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 --2- 172.21.15.19:0/1230655374 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e08f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.207+0000 7fed733e6740 1 -- 172.21.15.19:0/1230655374 >> 172.21.15.19:0/1230655374 conn(0x563ff3dcb310 msgr2=0x563ff3dfd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- 172.21.15.19:0/1230655374 shutdown_connections 2024-09-12T03:30:46.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- 172.21.15.19:0/1230655374 wait complete. 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- start start 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 Processor -- start 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e16a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1ac9f0 con 0x563ff3e03b40 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1ac260 con 0x563ff3e0d370 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c1a9320 con 0x563ff3e025d0 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed733e6740 10 client.0 _ll_register_callbacks cb 0x7fed5c005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed6a59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51320/0 (socket says 172.21.15.19:51320) 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed6a59c640 1 -- 172.21.15.19:0/199891779 learned_addr learned my addr 172.21.15.19:0/199891779 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:46.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.208+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6b59e640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e16a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 -- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 msgr2=0x563ff3e123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e123d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 -- 172.21.15.19:0/199891779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 msgr2=0x563ff3e16a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e16a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 -- 172.21.15.19:0/199891779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ff3e170c0 con 0x563ff3e03b40 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6b59e640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e16a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed6a59c640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x563ff3dfd380 tx=0x7fed6000af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959843ae28a7ff49 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed6001dca0 con 0x563ff3e03b40 2024-09-12T03:30:46.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed6000fa00 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed6001daf0 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fed6000fc00 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed733e6740 10 client.17766 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563ff3e8fef0 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.209+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed6001daf0 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fed6002f070 con 0x563ff3e03b40 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 10 client.17766 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 20 client.17766 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e906e0 0x563ff3e92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:46.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 4 client.17766 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ff3d6fd70 con 0x563ff3e906e0 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e906e0 0x563ff3e92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed6ad9d640 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e906e0 0x563ff3e92b00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73476a5ede49acf server_cookie=4f685d685bf8d1c2 in_seq=0 out_seq=0 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed4eff5640 10 client.17766 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed4eff5640 1 -- 172.21.15.19:0/199891779 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ff3d6fd70 con 0x563ff3e906e0 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed4eff5640 10 client.17766 handle_command_reply: tid=0 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.210+0000 7fed733e6740 1 client.17766 shutdown 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 msgr2=0x563ff3e12910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x563ff3dfd380 tx=0x7fed6000af60 comp rx=0 tx=0).stop 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 shutdown_connections 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x563ff3e906e0 0x563ff3e92b00 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563ff3e0d370 0x563ff3e16a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.293 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563ff3e03b40 0x563ff3e12910 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 --2- 172.21.15.19:0/199891779 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563ff3e025d0 0x563ff3e123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed4eff5640 0 client.17766 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:46.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 >> 172.21.15.19:0/199891779 conn(0x563ff3dcb310 msgr2=0x563ff3dfdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.294 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 shutdown_connections 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 1 -- 172.21.15.19:0/199891779 wait complete. 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed733e6740 20 client.17766 trim_cache size 0 max 16384 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 msgr2=0x7fed540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 0x7fed540a5040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fed6401f000 tx=0x7fed6401b000 comp rx=0 tx=0).stop 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 msgr2=0x7fed6c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005ab0 tx=0x7fed5c0042f0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 shutdown_connections 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed6c106f20 0x7fed6c19cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed6c1027a0 0x7fed6c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed540a2b80 0x7fed540a5040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 --2- 172.21.15.19:0/492280300 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed6c101df0 0x7fed6c198480 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.211+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 >> 172.21.15.19:0/492280300 conn(0x7fed6c0fd8f0 msgr2=0x7fed6c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.212+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 shutdown_connections 2024-09-12T03:30:46.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:46.212+0000 7fed7100d640 1 -- 172.21.15.19:0/492280300 wait complete. 2024-09-12T03:30:46.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:46 smithi121 ceph-mon[34963]: pgmap v689: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:30:46.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:46 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2255057508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:46.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:46 smithi019 ceph-mon[30953]: pgmap v689: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:30:46.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:46 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2255057508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:46 smithi135 ceph-mon[35474]: pgmap v689: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:30:47.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:46 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2255057508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:30:47.119 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 650c73d6 --release --await' 2024-09-12T03:30:47.120 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 650c73d6 --release --await 2024-09-12T03:30:47.813 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:48.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 -- 172.21.15.19:0/3874751738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 msgr2=0x7f10e410e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 --2- 172.21.15.19:0/3874751738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e410e790 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f10dc009ca0 tx=0x7f10dc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 -- 172.21.15.19:0/3874751738 shutdown_connections 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 --2- 172.21.15.19:0/3874751738 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e410e790 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 --2- 172.21.15.19:0/3874751738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 0x7f10e410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 --2- 172.21.15.19:0/3874751738 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 0x7f10e41031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 -- 172.21.15.19:0/3874751738 >> 172.21.15.19:0/3874751738 conn(0x7f10e40fc7d0 msgr2=0x7f10e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 -- 172.21.15.19:0/3874751738 shutdown_connections 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 -- 172.21.15.19:0/3874751738 wait complete. 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.320+0000 7f10e9d95640 1 Processor -- start 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 -- start start 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 0x7f10e4100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 0x7f10e4100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41121d0 con 0x7f10e410c2f0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4112050 con 0x7f10e41037a0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e9d95640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10e4112350 con 0x7f10e4102df0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51344/0 (socket says 172.21.15.19:51344) 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 -- 172.21.15.19:0/799749533 learned_addr learned my addr 172.21.15.19:0/799749533 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e37fe640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 0x7f10e4100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 0x7f10e4100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 -- 172.21.15.19:0/799749533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 msgr2=0x7f10e4100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 0x7f10e4100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 -- 172.21.15.19:0/799749533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 msgr2=0x7f10e4100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 0x7f10e4100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.321+0000 7f10e3fff640 1 -- 172.21.15.19:0/799749533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e41a6b50 con 0x7f10e410c2f0 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e3fff640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005cc0 tx=0x7f10dc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e0ff9640 1 -- 172.21.15.19:0/799749533 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc004240 con 0x7f10e410c2f0 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e0ff9640 1 -- 172.21.15.19:0/799749533 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10dc003e10 con 0x7f10e410c2f0 2024-09-12T03:30:48.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e41a6de0 con 0x7f10e410c2f0 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e0ff9640 1 -- 172.21.15.19:0/799749533 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10dc030e10 con 0x7f10e410c2f0 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.322+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e41a72f0 con 0x7f10e410c2f0 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.324+0000 7f10e0ff9640 1 -- 172.21.15.19:0/799749533 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10dc0043e0 con 0x7f10e410c2f0 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 Processor -- start 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 -- start start 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff7981440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff798c240 0x55dff798e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dff78e6620 con 0x55dff7981980 2024-09-12T03:30:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dff788de50 con 0x55dff798c240 2024-09-12T03:30:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10ec16e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dff78e25e0 con 0x55dff7981060 2024-09-12T03:30:48.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51356/0 (socket says 172.21.15.19:51356) 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 -- 172.21.15.19:0/3692540313 learned_addr learned my addr 172.21.15.19:0/3692540313 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff798c240 0x55dff798e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e0ff9640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 0x7f10cc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff7981440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 -- 172.21.15.19:0/3692540313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 msgr2=0x55dff7981440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff7981440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.325+0000 7f10e37fe640 1 -- 172.21.15.19:0/3692540313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff798c240 msgr2=0x55dff798e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e37fe640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff798c240 0x55dff798e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e37fe640 1 -- 172.21.15.19:0/3692540313 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff787ed40 con 0x55dff7981980 2024-09-12T03:30:48.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff7981440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:48.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e3fff640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 0x7f10cc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e0ff9640 1 -- 172.21.15.19:0/799749533 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f10dc0f0580 con 0x7f10e410c2f0 2024-09-12T03:30:48.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10e37fe640 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009a70 tx=0x7f10d4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14e78d6c8876427 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10c9ffb640 1 -- 172.21.15.19:0/3692540313 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4017070 con 0x55dff7981980 2024-09-12T03:30:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10c9ffb640 1 -- 172.21.15.19:0/3692540313 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4013470 con 0x55dff7981980 2024-09-12T03:30:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.326+0000 7f10c9ffb640 1 -- 172.21.15.19:0/3692540313 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4012630 con 0x55dff7981980 2024-09-12T03:30:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- 172.21.15.19:0/3692540313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 msgr2=0x55dff798bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10e3fff640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 0x7f10cc0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006010 tx=0x7f10dc087960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009a70 tx=0x7f10d4009f30 comp rx=0 tx=0).stop 2024-09-12T03:30:48.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- 172.21.15.19:0/3692540313 shutdown_connections 2024-09-12T03:30:48.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff798c240 0x55dff798e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff798bd00 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 --2- 172.21.15.19:0/3692540313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff7981440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- 172.21.15.19:0/3692540313 >> 172.21.15.19:0/3692540313 conn(0x55dff797d5f0 msgr2=0x55dff797d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:48.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- 172.21.15.19:0/3692540313 shutdown_connections 2024-09-12T03:30:48.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- 172.21.15.19:0/3692540313 wait complete. 2024-09-12T03:30:48.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 -- start start 2024-09-12T03:30:48.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.327+0000 7f10ec16e740 1 Processor -- start 2024-09-12T03:30:48.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff79890c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7989600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55dff798ee10 con 0x55dff798c240 2024-09-12T03:30:48.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55dff786ef10 con 0x55dff7981980 2024-09-12T03:30:48.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55dff798ef90 con 0x55dff7981060 2024-09-12T03:30:48.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 10 client.0 _ll_register_callbacks cb 0x55dff7892db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:48.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51360/0 (socket says 172.21.15.19:51360) 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 -- 172.21.15.19:0/1566883600 learned_addr learned my addr 172.21.15.19:0/1566883600 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff79890c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e37fe640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7989600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 -- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 msgr2=0x55dff79890c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff79890c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 -- 172.21.15.19:0/1566883600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 msgr2=0x55dff7989600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7989600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 -- 172.21.15.19:0/1566883600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff75f5580 con 0x55dff798c240 2024-09-12T03:30:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f10d8017d90 tx=0x7f10d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5262860c3bbf8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8017e80 con 0x55dff798c240 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f10d8010cf0 con 0x55dff798c240 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.328+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d4013470 con 0x55dff798c240 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f10d8015480 con 0x55dff798c240 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 10 client.17778 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:48.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dff76bf960 con 0x55dff798c240 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4013470 con 0x55dff798c240 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f10d80184c0 con 0x55dff798c240 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 10 client.17778 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 10 client.17778 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 20 client.17778 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff799bd60 0x55dff799e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 4 client.17778 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff78f3b90 con 0x55dff799bd60 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.329+0000 7f10e37fe640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff799bd60 0x55dff799e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.330+0000 7f10e37fe640 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff799bd60 0x55dff799e170 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36a94985ed8f8e1a server_cookie=e15c75c074ff02cc in_seq=0 out_seq=0 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.330+0000 7f10c2ffd640 10 client.17778 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:48.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.330+0000 7f10c2ffd640 1 -- 172.21.15.19:0/1566883600 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dff78f3b90 con 0x55dff799bd60 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.330+0000 7f10c2ffd640 10 client.17778 handle_command_reply: tid=0 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.330+0000 7f10ec16e740 1 client.17778 shutdown 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 msgr2=0x55dff7989b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f10d8017d90 tx=0x7f10d80046c0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 shutdown_connections 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff798c240 0x55dff7989b40 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff799bd60 0x55dff799e170 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7989600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 --2- 172.21.15.19:0/1566883600 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff7981060 0x55dff79890c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10c2ffd640 0 client.17778 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 >> 172.21.15.19:0/1566883600 conn(0x55dff797d5f0 msgr2=0x55dff79832d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 shutdown_connections 2024-09-12T03:30:48.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 1 -- 172.21.15.19:0/1566883600 wait complete. 2024-09-12T03:30:48.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.331+0000 7f10ec16e740 20 client.17778 trim_cache size 0 max 16384 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 Processor -- start 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 -- start start 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981060 0x55dff7983ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff79a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41af860 con 0x55dff7981980 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b15e0 con 0x55dff7981060 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10ec16e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41aff30 con 0x55dff798c240 2024-09-12T03:30:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.348+0000 7f10e37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51362/0 (socket says 172.21.15.19:51362) 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 -- 172.21.15.19:0/546745485 learned_addr learned my addr 172.21.15.19:0/546745485 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff79a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 -- 172.21.15.19:0/546745485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 msgr2=0x55dff79a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff79a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 -- 172.21.15.19:0/546745485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981060 msgr2=0x55dff7983ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981060 0x55dff7983ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 -- 172.21.15.19:0/546745485 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff7988070 con 0x55dff7981980 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff79a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10e37fe640 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f10d4013f00 tx=0x7f10d4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46b3f75887c3eb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10c97fa640 1 -- 172.21.15.19:0/546745485 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d402c070 con 0x55dff7981980 2024-09-12T03:30:48.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10c97fa640 1 -- 172.21.15.19:0/546745485 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d400edc0 con 0x55dff7981980 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.349+0000 7f10c97fa640 1 -- 172.21.15.19:0/546745485 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4017040 con 0x55dff7981980 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- 172.21.15.19:0/546745485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 msgr2=0x55dff79a3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f10d4013f00 tx=0x7f10d4008040 comp rx=0 tx=0).stop 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- 172.21.15.19:0/546745485 shutdown_connections 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff79a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981980 0x55dff79a3a80 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 --2- 172.21.15.19:0/546745485 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981060 0x55dff7983ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- 172.21.15.19:0/546745485 >> 172.21.15.19:0/546745485 conn(0x55dff797d5f0 msgr2=0x55dff798b140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- 172.21.15.19:0/546745485 shutdown_connections 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- 172.21.15.19:0/546745485 wait complete. 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 -- start start 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.350+0000 7f10ec16e740 1 Processor -- start 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7990bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff7a05a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b0e50 con 0x55dff7981060 2024-09-12T03:30:48.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41b06c0 con 0x55dff7981980 2024-09-12T03:30:48.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10e41ad7a0 con 0x55dff798c240 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10ec16e740 10 client.0 _ll_register_callbacks cb 0x7f10d80179d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51376/0 (socket says 172.21.15.19:51376) 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7990bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 -- 172.21.15.19:0/2330639824 learned_addr learned my addr 172.21.15.19:0/2330639824 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff7a05a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 -- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 msgr2=0x55dff7a05a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff7a05a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 -- 172.21.15.19:0/2330639824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 msgr2=0x55dff7990bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7990bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 -- 172.21.15.19:0/2330639824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff7a05f90 con 0x55dff7981060 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e37fe640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7990bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10e2ffd640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f10d8017ae0 tx=0x7f10d800a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b769be21a0dee382 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:48.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.351+0000 7f10c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d8029040 con 0x55dff7981060 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f10d8018450 con 0x55dff7981060 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10d400edc0 con 0x55dff7981060 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f10d800aae0 con 0x55dff7981060 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 10 client.17784 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55dff7a071e0 con 0x55dff7981060 2024-09-12T03:30:48.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d400edc0 con 0x55dff7981060 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f10d8015b90 con 0x55dff7981060 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 10 client.17784 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 10 client.17784 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 20 client.17784 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff7a079a0 0x55dff7a09dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:48.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 4 client.17784 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "650c73d6", "release": true, "await": true}] 2024-09-12T03:30:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "650c73d6", "release": true, "await": true}) -- 0x55dff78ee0a0 con 0x55dff7a079a0 2024-09-12T03:30:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.352+0000 7f10e37fe640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff7a079a0 0x55dff7a09dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.353+0000 7f10e37fe640 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff7a079a0 0x55dff7a09dc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76312d32d459eb39 server_cookie=10f7c1412f8272d8 in_seq=0 out_seq=0 2024-09-12T03:30:48.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:48.353+0000 7f10c2ffd640 10 client.17784 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:48.723 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:30:48.723 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:30:48.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:48 smithi121 ceph-mon[34963]: pgmap v690: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:30:48.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:48 smithi019 ceph-mon[30953]: pgmap v690: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:30:48.983 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:49.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:48 smithi135 ceph-mon[35474]: pgmap v690: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:30:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10c2ffd640 1 -- 172.21.15.19:0/2330639824 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55dff78ee0a0 con 0x55dff7a079a0 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10c2ffd640 10 client.17784 handle_command_reply: tid=0 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 client.17784 shutdown 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 msgr2=0x55dff7990680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f10d8017ae0 tx=0x7f10d800a1b0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 shutdown_connections 2024-09-12T03:30:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55dff798c240 0x55dff7a05a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55dff7a079a0 0x55dff7a09dc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10c2ffd640 0 client.17784 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55dff7981980 0x55dff7990bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 --2- 172.21.15.19:0/2330639824 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55dff7981060 0x55dff7990680 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 >> 172.21.15.19:0/2330639824 conn(0x55dff797d5f0 msgr2=0x55dff7989080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 shutdown_connections 2024-09-12T03:30:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 1 -- 172.21.15.19:0/2330639824 wait complete. 2024-09-12T03:30:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.269+0000 7f10ec16e740 20 client.17784 trim_cache size 0 max 16384 2024-09-12T03:30:49.273 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:49.274 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 58, 2024-09-12T03:30:49.274 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:30:49.274 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 44, 2024-09-12T03:30:49.274 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:30:49.274 INFO:teuthology.orchestra.run.smithi019.stdout: "650c73d6": { 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 44, 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:49.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 142.6, 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:30:49.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:30:49.277 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:30:49.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:49.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:49.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:49.277 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:49.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:49.279 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 msgr2=0x7f10cc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 0x7f10cc0a5040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006010 tx=0x7f10dc087960 comp rx=0 tx=0).stop 2024-09-12T03:30:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 msgr2=0x7f10e41014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f10dc005cc0 tx=0x7f10dc003c20 comp rx=0 tx=0).stop 2024-09-12T03:30:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 shutdown_connections 2024-09-12T03:30:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10cc0a2b80 0x7f10cc0a5040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10e410c2f0 0x7f10e41014d0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10e41037a0 0x7f10e4100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 --2- 172.21.15.19:0/799749533 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10e4102df0 0x7f10e4100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 >> 172.21.15.19:0/799749533 conn(0x7f10e40fc7d0 msgr2=0x7f10e40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 shutdown_connections 2024-09-12T03:30:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.270+0000 7f10e9d95640 1 -- 172.21.15.19:0/799749533 wait complete. 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 -- 172.21.15.19:0/2107371218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc101420 msgr2=0x7fcccc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/2107371218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc101420 0x7fcccc10d950 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009cd0 tx=0x7fccc402a8f0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 -- 172.21.15.19:0/2107371218 shutdown_connections 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/2107371218 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc101420 0x7fcccc10d950 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/2107371218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc100a80 0x7fcccc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/2107371218 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc106a80 0x7fcccc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 -- 172.21.15.19:0/2107371218 >> 172.21.15.19:0/2107371218 conn(0x7fcccc0fc7a0 msgr2=0x7fcccc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 -- 172.21.15.19:0/2107371218 shutdown_connections 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.491+0000 7fccd3cc7640 1 -- 172.21.15.19:0/2107371218 wait complete. 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 Processor -- start 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 -- start start 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc101420 0x7fcccc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 0x7fcccc19cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc110e80 con 0x7fcccc100a80 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc110d00 con 0x7fcccc106a80 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd3cc7640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc111000 con 0x7fcccc101420 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.492+0000 7fccd1a63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.512 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51386/0 (socket says 172.21.15.19:51386) 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 -- 172.21.15.19:0/4068651810 learned_addr learned my addr 172.21.15.19:0/4068651810 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1262640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc101420 0x7fcccc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd2264640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 0x7fcccc19cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 -- 172.21.15.19:0/4068651810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc101420 msgr2=0x7fcccc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc101420 0x7fcccc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 -- 172.21.15.19:0/4068651810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 msgr2=0x7fcccc19cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 0x7fcccc19cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 -- 172.21.15.19:0/4068651810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcccc19d350 con 0x7fcccc100a80 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd2264640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 0x7fcccc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd1a63640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fccc800e9f0 tx=0x7fccc800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.513 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccc2ffd640 1 -- 172.21.15.19:0/4068651810 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc800ccc0 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccc2ffd640 1 -- 172.21.15.19:0/4068651810 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc800ce60 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcccc19d640 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccc2ffd640 1 -- 172.21.15.19:0/4068651810 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8010690 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.493+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcccc1a4f20 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccc2ffd640 1 -- 172.21.15.19:0/4068651810 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fccc80046c0 con 0x7fcccc100a80 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 Processor -- start 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 -- start start 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 0x5617fad50800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad50d40 0x5617fad53130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617facab280 con 0x5617fad46480 2024-09-12T03:30:49.514 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617fac52b10 con 0x5617fad50d40 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd60a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5617faca7240 con 0x5617fad45b60 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd1a63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 0x5617fad50800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd2264640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd2264640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:38456/0 (socket says 172.21.15.19:38456) 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd1a63640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 0x5617fad50800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51396/0 (socket says 172.21.15.19:51396) 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd2264640 1 -- 172.21.15.19:0/3151294895 learned_addr learned my addr 172.21.15.19:0/3151294895 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.496+0000 7fccd1262640 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad50d40 0x5617fad53130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 -- 172.21.15.19:0/3151294895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad50d40 msgr2=0x5617fad53130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad50d40 0x5617fad53130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 -- 172.21.15.19:0/3151294895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 msgr2=0x5617fad50800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 0x5617fad50800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 -- 172.21.15.19:0/3151294895 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fac5f1f0 con 0x5617fad45b60 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccc2ffd640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 0x7fccb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd2264640 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009400 tx=0x7fccc4009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03b2c6078d39b7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccd1a63640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 0x7fccb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccc2ffd640 1 -- 172.21.15.19:0/4068651810 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fccc80cf3e0 con 0x7fcccc100a80 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccb7fff640 1 -- 172.21.15.19:0/3151294895 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4030620 con 0x5617fad45b60 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccb7fff640 1 -- 172.21.15.19:0/3151294895 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc4036070 con 0x5617fad45b60 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.497+0000 7fccb7fff640 1 -- 172.21.15.19:0/3151294895 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc40043c0 con 0x5617fad45b60 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- 172.21.15.19:0/3151294895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 msgr2=0x5617fad45f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009400 tx=0x7fccc4009a20 comp rx=0 tx=0).stop 2024-09-12T03:30:49.516 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd1a63640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 0x7fccb80a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fccc80cd260 tx=0x7fccc80d2ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- 172.21.15.19:0/3151294895 shutdown_connections 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad50d40 0x5617fad53130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad46480 0x5617fad50800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3151294895 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad45f40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- 172.21.15.19:0/3151294895 >> 172.21.15.19:0/3151294895 conn(0x5617fad420f0 msgr2=0x5617fad424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- 172.21.15.19:0/3151294895 shutdown_connections 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- 172.21.15.19:0/3151294895 wait complete. 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.498+0000 7fccd60a0740 1 -- start start 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 Processor -- start 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.517 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5617fad53910 con 0x5617fad50d40 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5617fac65f10 con 0x5617fad45b60 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5617fad53a90 con 0x5617fad46480 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd60a0740 10 client.0 _ll_register_callbacks cb 0x5617fac2ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51408/0 (socket says 172.21.15.19:51408) 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd2264640 1 -- 172.21.15.19:0/287673844 learned_addr learned my addr 172.21.15.19:0/287673844 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd1262640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.499+0000 7fccd2264640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 -- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 msgr2=0x5617fad4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad4e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.518 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 -- 172.21.15.19:0/287673844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 msgr2=0x5617fad4dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 -- 172.21.15.19:0/287673844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617faa1c6a0 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd1262640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd2264640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009bf0 tx=0x7fccc4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40236ab8aa14c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4049040 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fccc4004140 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc4036070 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fccc4004340 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd60a0740 10 client.17796 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5617fad5f4e0 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc4036070 con 0x5617fad50d40 2024-09-12T03:30:49.519 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fccc4031560 con 0x5617fad50d40 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.500+0000 7fccd60a0740 10 client.17796 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd60a0740 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd60a0740 20 client.17796 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fad60830 0x5617fad62c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd60a0740 4 client.17796 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617facb87f0 con 0x5617fad60830 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd1262640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fad60830 0x5617fad62c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccb57fa640 10 client.17796 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.501+0000 7fccd1262640 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fad60830 0x5617fad62c20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bf74bf2e15a0e0d server_cookie=1a7f7c95e59b2231 in_seq=0 out_seq=0 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccb57fa640 1 -- 172.21.15.19:0/287673844 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5617facb87f0 con 0x5617fad60830 2024-09-12T03:30:49.520 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccb57fa640 10 client.17796 handle_command_reply: tid=0 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 client.17796 shutdown 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 msgr2=0x5617fad4e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009bf0 tx=0x7fccc4003f60 comp rx=0 tx=0).stop 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 shutdown_connections 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad4e660 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fad60830 0x5617fad62c20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 --2- 172.21.15.19:0/287673844 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.521 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 >> 172.21.15.19:0/287673844 conn(0x5617fad420f0 msgr2=0x5617fad41270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccb57fa640 0 client.17796 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:49.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 shutdown_connections 2024-09-12T03:30:49.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.503+0000 7fccd60a0740 1 -- 172.21.15.19:0/287673844 wait complete. 2024-09-12T03:30:49.522 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.504+0000 7fccd60a0740 20 client.17796 trim_cache size 0 max 16384 2024-09-12T03:30:49.524 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 Processor -- start 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 -- start start 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1ad580 con 0x5617fad50d40 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1af300 con 0x5617fad45b60 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.522+0000 7fccd60a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1adc50 con 0x5617fad46480 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51422/0 (socket says 172.21.15.19:51422) 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 -- 172.21.15.19:0/3875514596 learned_addr learned my addr 172.21.15.19:0/3875514596 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:49.525 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd1262640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 -- 172.21.15.19:0/3875514596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 msgr2=0x5617fad68250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad68250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 -- 172.21.15.19:0/3875514596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 msgr2=0x5617fad4fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 -- 172.21.15.19:0/3875514596 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617fad504b0 con 0x5617fad50d40 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd1262640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccd2264640 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fccc4031040 tx=0x7fccc4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60260ed34610a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccb77fe640 1 -- 172.21.15.19:0/3875514596 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4003970 con 0x5617fad50d40 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccb77fe640 1 -- 172.21.15.19:0/3875514596 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc4036070 con 0x5617fad50d40 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.523+0000 7fccb77fe640 1 -- 172.21.15.19:0/3875514596 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4004da0 con 0x5617fad50d40 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- 172.21.15.19:0/3875514596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 msgr2=0x5617fad68790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.526 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fccc4031040 tx=0x7fccc4009a20 comp rx=0 tx=0).stop 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- 172.21.15.19:0/3875514596 shutdown_connections 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad68790 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad46480 0x5617fad68250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 --2- 172.21.15.19:0/3875514596 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad45b60 0x5617fad4fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- 172.21.15.19:0/3875514596 >> 172.21.15.19:0/3875514596 conn(0x5617fad420f0 msgr2=0x5617fad4f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- 172.21.15.19:0/3875514596 shutdown_connections 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- 172.21.15.19:0/3875514596 wait complete. 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 -- start start 2024-09-12T03:30:49.527 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.524+0000 7fccd60a0740 1 Processor -- start 2024-09-12T03:30:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad54b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad46480 0x5617fad550a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.529 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1aeb70 con 0x5617fad50d40 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1ae3e0 con 0x5617fad46480 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fcccc1ab4a0 con 0x5617fad45b60 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd60a0740 10 client.0 _ll_register_callbacks cb 0x7fccc80d2d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:51438/0 (socket says 172.21.15.19:51438) 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 -- 172.21.15.19:0/565126159 learned_addr learned my addr 172.21.15.19:0/565126159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad54b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd1262640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad46480 0x5617fad550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 -- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 msgr2=0x5617fad54b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad54b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.530 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 -- 172.21.15.19:0/565126159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad46480 msgr2=0x5617fad550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad46480 0x5617fad550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.525+0000 7fccd2264640 1 -- 172.21.15.19:0/565126159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617fadca380 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd2264640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005e30 tx=0x7fccc4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ae62ea8d0845c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc4003970 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fccc4003b10 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccc4036070 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fccc4004b40 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 10 client.17802 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5617fadcb660 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccc4036070 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7fccc4068070 con 0x5617fad50d40 2024-09-12T03:30:49.531 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 10 client.17802 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 20 client.17802 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.526+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fadcbc50 0x5617fadce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.527+0000 7fccd60a0740 4 client.17802 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.527+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5617facb2d00 con 0x5617fadcbc50 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.527+0000 7fccd1262640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fadcbc50 0x5617fadce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.527+0000 7fccd1262640 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fadcbc50 0x5617fadce070 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c00d16adb666ac23 server_cookie=b160dd0a571f07ac in_seq=0 out_seq=0 2024-09-12T03:30:49.532 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.527+0000 7fccb57fa640 10 client.17802 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:49.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.530+0000 7fccb57fa640 1 -- 172.21.15.19:0/565126159 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5617facb2d00 con 0x5617fadcbc50 2024-09-12T03:30:49.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.530+0000 7fccb57fa640 10 client.17802 handle_command_reply: tid=0 2024-09-12T03:30:49.533 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.530+0000 7fccd60a0740 1 client.17802 shutdown 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 msgr2=0x5617fad555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fccc4005e30 tx=0x7fccc4009800 comp rx=0 tx=0).stop 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 shutdown_connections 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5617fad50d40 0x5617fad555e0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5617fadcbc50 0x5617fadce070 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5617fad46480 0x5617fad550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 --2- 172.21.15.19:0/565126159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5617fad45b60 0x5617fad54b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccb57fa640 0 client.17802 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 >> 172.21.15.19:0/565126159 conn(0x5617fad420f0 msgr2=0x5617fad4e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 shutdown_connections 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 1 -- 172.21.15.19:0/565126159 wait complete. 2024-09-12T03:30:49.534 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd60a0740 20 client.17802 trim_cache size 0 max 16384 2024-09-12T03:30:49.535 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:49.535 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:30:49.535 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "3e614ca2-e9b3-432b-ad7a-277b20b62776", 2024-09-12T03:30:49.535 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:30:49.535 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 msgr2=0x7fccb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 0x7fccb80a50c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fccc80cd260 tx=0x7fccc80d2ec0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 msgr2=0x7fcccc1983a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fccc800e9f0 tx=0x7fccc800edf0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.531+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 shutdown_connections 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcccc106a80 0x7fcccc19cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fcccc101420 0x7fcccc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.536 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fccb80a2c00 0x7fccb80a50c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 --2- 172.21.15.19:0/4068651810 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fcccc100a80 0x7fcccc1983a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:49.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 >> 172.21.15.19:0/4068651810 conn(0x7fcccc0fc7a0 msgr2=0x7fcccc0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:49.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 shutdown_connections 2024-09-12T03:30:49.537 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:49.532+0000 7fccd3cc7640 1 -- 172.21.15.19:0/4068651810 wait complete. 2024-09-12T03:30:50.213 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '650c73d6', total seconds elapsed: 28.9 2024-09-12T03:30:50.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: pgmap v691: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-12T03:30:50.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:50 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:50.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: pgmap v691: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-12T03:30:50.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:50 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:30:51.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: pgmap v691: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-12T03:30:51.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:50 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:51.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:30:51.605 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 -- 172.21.15.19:0/2378218963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac102ef0 msgr2=0x7f9dac10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 --2- 172.21.15.19:0/2378218963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac102ef0 0x7f9dac10da40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009ca0 tx=0x7f9da402a950 comp rx=0 tx=0).stop 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 -- 172.21.15.19:0/2378218963 shutdown_connections 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 --2- 172.21.15.19:0/2378218963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac102ef0 0x7f9dac10da40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 --2- 172.21.15.19:0/2378218963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 0x7f9dac1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 --2- 172.21.15.19:0/2378218963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac100e20 0x7f9dac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 -- 172.21.15.19:0/2378218963 >> 172.21.15.19:0/2378218963 conn(0x7f9dac0fc7d0 msgr2=0x7f9dac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 -- 172.21.15.19:0/2378218963 shutdown_connections 2024-09-12T03:30:52.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.130+0000 7f9db403f640 1 -- 172.21.15.19:0/2378218963 wait complete. 2024-09-12T03:30:52.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.131+0000 7f9db403f640 1 Processor -- start 2024-09-12T03:30:52.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.131+0000 7f9db403f640 1 -- start start 2024-09-12T03:30:52.133 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 0x7f9dac199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac102ef0 0x7f9dac1a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac110fd0 con 0x7f9dac100e20 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac110e50 con 0x7f9dac102ef0 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db403f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac111150 con 0x7f9dac102570 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35740/0 (socket says 172.21.15.19:35740) 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1119132849 learned_addr learned my addr 172.21.15.19:0/1119132849 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db25dc640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac102ef0 0x7f9dac1a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db15da640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 0x7f9dac199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1119132849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 msgr2=0x7f9dac199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 0x7f9dac199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.134 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1119132849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac102ef0 msgr2=0x7f9dac1a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac102ef0 0x7f9dac1a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.132+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1119132849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac1a48c0 con 0x7f9dac100e20 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9f0 tx=0x7f9da800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9da2ffd640 1 -- 172.21.15.19:0/1119132849 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da800cd80 con 0x7f9dac100e20 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac1a4a90 con 0x7f9dac100e20 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9da2ffd640 1 -- 172.21.15.19:0/1119132849 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da8010910 con 0x7f9dac100e20 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9da2ffd640 1 -- 172.21.15.19:0/1119132849 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80188f0 con 0x7f9dac100e20 2024-09-12T03:30:52.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.133+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac1a4fd0 con 0x7f9dac100e20 2024-09-12T03:30:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.136+0000 7f9da2ffd640 1 -- 172.21.15.19:0/1119132849 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9da8010ac0 con 0x7f9dac100e20 2024-09-12T03:30:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.136+0000 7f9db6418740 1 Processor -- start 2024-09-12T03:30:52.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 -- start start 2024-09-12T03:30:52.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e4ef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e4fca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55706e4552e0 con 0x55706e4fa680 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55706e3fcce0 con 0x55706e4efdc0 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db6418740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55706e4512a0 con 0x55706e4ef4a0 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:41712/0 (socket says 172.21.15.19:41712) 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1459867068 learned_addr learned my addr 172.21.15.19:0/1459867068 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9da2ffd640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 0x7f9d980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db25dc640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e4ef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.137+0000 7f9db15da640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e4fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1459867068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 msgr2=0x55706e4ef880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e4ef880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1459867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 msgr2=0x55706e4fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e4fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 -- 172.21.15.19:0/1459867068 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706e3e1440 con 0x55706e4efdc0 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db25dc640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e4ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db25dc640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 0x7f9d980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9da2ffd640 1 -- 172.21.15.19:0/1119132849 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9da80cf3a0 con 0x7f9dac100e20 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9da8015b50 tx=0x7f9da8024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e898dd46756eca9a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9d97fff640 1 -- 172.21.15.19:0/1459867068 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8015960 con 0x55706e4efdc0 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9d97fff640 1 -- 172.21.15.19:0/1459867068 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da80851f0 con 0x55706e4efdc0 2024-09-12T03:30:52.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.138+0000 7f9d97fff640 1 -- 172.21.15.19:0/1459867068 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8013190 con 0x55706e4efdc0 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 -- 172.21.15.19:0/1459867068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 msgr2=0x55706e4fa140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db25dc640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 0x7f9d980a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006010 tx=0x7f9da40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9da8015b50 tx=0x7f9da8024000 comp rx=0 tx=0).stop 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 -- 172.21.15.19:0/1459867068 shutdown_connections 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e4fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e4fa140 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 --2- 172.21.15.19:0/1459867068 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e4ef880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 -- 172.21.15.19:0/1459867068 >> 172.21.15.19:0/1459867068 conn(0x55706e4eba30 msgr2=0x55706e4ebe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 -- 172.21.15.19:0/1459867068 shutdown_connections 2024-09-12T03:30:52.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.139+0000 7f9db6418740 1 -- 172.21.15.19:0/1459867068 wait complete. 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 -- start start 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 Processor -- start 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4efdc0 0x55706e4f7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4fa680 0x55706e4f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55706e4fd250 con 0x55706e4efdc0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55706e410050 con 0x55706e4ef4a0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.140+0000 7f9db6418740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55706e4fd3d0 con 0x55706e4fa680 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db6418740 10 client.0 _ll_register_callbacks cb 0x55706e450160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db25dc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4fa680 0x55706e4f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db15da640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4efdc0 0x55706e4f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:41724/0 (socket says 172.21.15.19:41724) 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 -- 172.21.15.19:0/3173144572 learned_addr learned my addr 172.21.15.19:0/3173144572 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 -- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4fa680 msgr2=0x55706e4f7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4fa680 0x55706e4f7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 -- 172.21.15.19:0/3173144572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4efdc0 msgr2=0x55706e4f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4efdc0 0x55706e4f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 -- 172.21.15.19:0/3173144572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706e1c63c0 con 0x55706e4ef4a0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9da809bbf0 tx=0x7f9da809bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c25625b642366070 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80a4820 con 0x55706e4ef4a0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.141+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9da80a4390 con 0x55706e4ef4a0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da80851f0 con 0x55706e4ef4a0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9da80b2020 con 0x55706e4ef4a0 2024-09-12T03:30:52.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 10 client.27155 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55706e5113e0 con 0x55706e4ef4a0 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da80851f0 con 0x55706e4ef4a0 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f9da809ab20 con 0x55706e4ef4a0 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 10 client.27155 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 10 client.27155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 20 client.27155 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e512730 0x55706e514b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 4 client.27155 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55706e462850 con 0x55706e512730 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.142+0000 7f9db15da640 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e512730 0x55706e514b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.143+0000 7f9d957fa640 10 client.27155 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:52.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.143+0000 7f9db15da640 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e512730 0x55706e514b20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3564d5dd0896a2f3 server_cookie=197c79ae103f9217 in_seq=0 out_seq=0 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.144+0000 7f9d957fa640 1 -- 172.21.15.19:0/3173144572 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55706e462850 con 0x55706e512730 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.144+0000 7f9d957fa640 10 client.27155 handle_command_reply: tid=0 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.144+0000 7f9db6418740 1 client.27155 shutdown 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.144+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 msgr2=0x55706e4f9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.144+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9da809bbf0 tx=0x7f9da809bd50 comp rx=0 tx=0).stop 2024-09-12T03:30:52.149 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 shutdown_connections 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4fa680 0x55706e4f7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4efdc0 0x55706e4f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e512730 0x55706e514b20 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 --2- 172.21.15.19:0/3173144572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4ef4a0 0x55706e4f9f90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 >> 172.21.15.19:0/3173144572 conn(0x55706e4eba30 msgr2=0x55706e4f1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9d957fa640 0 client.27155 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 shutdown_connections 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 1 -- 172.21.15.19:0/3173144572 wait complete. 2024-09-12T03:30:52.150 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.145+0000 7f9db6418740 20 client.27155 trim_cache size 0 max 16384 2024-09-12T03:30:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.166+0000 7f9db6418740 1 Processor -- start 2024-09-12T03:30:52.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.166+0000 7f9db6418740 1 -- start start 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35760/0 (socket says 172.21.15.19:35760) 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4efdc0 0x55706e51a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 -- 172.21.15.19:0/2976030656 learned_addr learned my addr 172.21.15.19:0/2976030656 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4fa680 0x55706e51a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ab410 con 0x55706e4ef4a0 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ad190 con 0x55706e4fa680 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1abae0 con 0x55706e4efdc0 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 -- 172.21.15.19:0/2976030656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4efdc0 msgr2=0x55706e51a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.168 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4efdc0 0x55706e51a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 -- 172.21.15.19:0/2976030656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4fa680 msgr2=0x55706e51a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4fa680 0x55706e51a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 -- 172.21.15.19:0/2976030656 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55706e4f61f0 con 0x55706e4ef4a0 2024-09-12T03:30:52.169 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.167+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9da80980c0 tx=0x7f9da80cf620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705df88a115a3165 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.173 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.168+0000 7f9d977fe640 1 -- 172.21.15.19:0/2976030656 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da809ff60 con 0x55706e4ef4a0 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.168+0000 7f9d977fe640 1 -- 172.21.15.19:0/2976030656 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da809ab20 con 0x55706e4ef4a0 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.168+0000 7f9d977fe640 1 -- 172.21.15.19:0/2976030656 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da80a45f0 con 0x55706e4ef4a0 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.168+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 msgr2=0x55706e4f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.168+0000 7f9db6418740 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f9da80980c0 tx=0x7f9da80cf620 comp rx=0 tx=0).stop 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 shutdown_connections 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4fa680 0x55706e51a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4efdc0 0x55706e51a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 --2- 172.21.15.19:0/2976030656 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4ef4a0 0x55706e4f5b10 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.174 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 >> 172.21.15.19:0/2976030656 conn(0x55706e4eba30 msgr2=0x55706e4f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 shutdown_connections 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 -- 172.21.15.19:0/2976030656 wait complete. 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 -- start start 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 Processor -- start 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.169+0000 7f9db6418740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e50b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e508a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e50b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1aca00 con 0x55706e4fa680 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.175 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:35772/0 (socket says 172.21.15.19:35772) 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db1ddb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e50b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45948/0 (socket says 172.21.15.19:45948) 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1ac270 con 0x55706e4efdc0 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac1a9350 con 0x55706e4ef4a0 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db1ddb640 1 -- 172.21.15.19:0/2529661548 learned_addr learned my addr 172.21.15.19:0/2529661548 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db6418740 10 client.0 _ll_register_callbacks cb 0x7f9da809b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db15da640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e508a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 -- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 msgr2=0x55706e50b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e50b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 -- 172.21.15.19:0/2529661548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 msgr2=0x55706e508a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e508a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db25dc640 1 -- 172.21.15.19:0/2529661548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55706e509890 con 0x55706e4fa680 2024-09-12T03:30:52.176 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.170+0000 7f9db15da640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e508a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9db25dc640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005e30 tx=0x7f9da4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8deef0804dee1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9d957fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4034040 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da809ab20 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9da4030aa0 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9da4030ca0 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9db6418740 10 client.17820 fetch_fsmap learned FSMap version 58 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.171+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55706e57d270 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9da4003e50 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f9da403a430 con 0x55706e4fa680 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 10 client.17820 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 10 client.17820 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:30:52.177 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 20 client.17820 populate_metadata read hostname 'smithi019' 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e57da50 0x55706e57fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 4 client.17820 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9da809ab20 con 0x55706e57da50 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.172+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e57da50 0x55706e57fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.173+0000 7f9db1ddb640 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e57da50 0x55706e57fe70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98f765955081462a server_cookie=7b43d08a5552ee38 in_seq=0 out_seq=0 2024-09-12T03:30:52.178 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.173+0000 7f9d957fa640 10 client.17820 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:52.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9d957fa640 1 -- 172.21.15.19:0/2529661548 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f9da809ab20 con 0x55706e57da50 2024-09-12T03:30:52.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9d957fa640 10 client.17820 handle_command_reply: tid=0 2024-09-12T03:30:52.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 client.17820 shutdown 2024-09-12T03:30:52.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 msgr2=0x55706e509060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.179 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f9da4005e30 tx=0x7f9da4005d80 comp rx=0 tx=0).stop 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 shutdown_connections 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55706e4fa680 0x55706e509060 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55706e57da50 0x55706e57fe70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55706e4efdc0 0x55706e508a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 --2- 172.21.15.19:0/2529661548 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55706e4ef4a0 0x55706e50b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 >> 172.21.15.19:0/2529661548 conn(0x55706e4eba30 msgr2=0x55706e4f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9d957fa640 0 client.17820 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.175+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 shutdown_connections 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db6418740 1 -- 172.21.15.19:0/2529661548 wait complete. 2024-09-12T03:30:52.180 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db6418740 20 client.17820 trim_cache size 0 max 16384 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "3e614ca2-e9b3-432b-ad7a-277b20b62776": { 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "3e614ca2-e9b3-432b-ad7a-277b20b62776", 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:30:52.181 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:30:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 msgr2=0x7f9d980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 0x7f9d980a50c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9da4006010 tx=0x7f9da40095c0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 msgr2=0x7f9dac199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:30:52.182 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.176+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9da800e9f0 tx=0x7f9da800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 shutdown_connections 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9dac102ef0 0x7f9dac1a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9dac102570 0x7f9dac199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9d980a2c00 0x7f9d980a50c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 --2- 172.21.15.19:0/1119132849 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9dac100e20 0x7f9dac199300 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 >> 172.21.15.19:0/1119132849 conn(0x7f9dac0fc7d0 msgr2=0x7f9dac106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 shutdown_connections 2024-09-12T03:30:52.183 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:30:52.177+0000 7f9db403f640 1 -- 172.21.15.19:0/1119132849 wait complete. 2024-09-12T03:30:52.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:52 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:52.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:52 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:52.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:52 smithi121 ceph-mon[34963]: pgmap v692: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 KiB/s wr, 34 op/s 2024-09-12T03:30:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:52 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:52 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:52.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:52 smithi019 ceph-mon[30953]: pgmap v692: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 KiB/s wr, 34 op/s 2024-09-12T03:30:53.007 INFO:tasks.cephfs.filesystem:scrub status for tag:3e614ca2-e9b3-432b-ad7a-277b20b62776 - {'path': '/', 'tag': '3e614ca2-e9b3-432b-ad7a-277b20b62776', 'options': 'recursive,force'} 2024-09-12T03:30:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:52 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:52 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:53.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:52 smithi135 ceph-mon[35474]: pgmap v692: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.5 KiB/s wr, 34 op/s 2024-09-12T03:30:54.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:54 smithi121 ceph-mon[34963]: pgmap v693: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 240 KiB/s wr, 71 op/s 2024-09-12T03:30:54.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:54 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:54.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:54 smithi019 ceph-mon[30953]: pgmap v693: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 240 KiB/s wr, 71 op/s 2024-09-12T03:30:54.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:54 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:55.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:54 smithi135 ceph-mon[35474]: pgmap v693: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 240 KiB/s wr, 71 op/s 2024-09-12T03:30:55.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:54 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:30:56.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:56 smithi121 ceph-mon[34963]: pgmap v694: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 240 KiB/s wr, 125 op/s 2024-09-12T03:30:56.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:56 smithi019 ceph-mon[30953]: pgmap v694: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 240 KiB/s wr, 125 op/s 2024-09-12T03:30:57.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:56 smithi135 ceph-mon[35474]: pgmap v694: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 240 KiB/s wr, 125 op/s 2024-09-12T03:30:58.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:30:58 smithi121 ceph-mon[34963]: pgmap v695: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 457 KiB/s wr, 222 op/s 2024-09-12T03:30:58.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:30:58 smithi019 ceph-mon[30953]: pgmap v695: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 457 KiB/s wr, 222 op/s 2024-09-12T03:30:59.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:30:58 smithi135 ceph-mon[35474]: pgmap v695: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 457 KiB/s wr, 222 op/s 2024-09-12T03:31:00.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:00 smithi019 ceph-mon[30953]: pgmap v696: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 834 KiB/s wr, 264 op/s 2024-09-12T03:31:01.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:00 smithi135 ceph-mon[35474]: pgmap v696: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 834 KiB/s wr, 264 op/s 2024-09-12T03:31:01.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:00 smithi121 ceph-mon[34963]: pgmap v696: 97 pgs: 97 active+clean; 1.8 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 834 KiB/s wr, 264 op/s 2024-09-12T03:31:02.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:02 smithi019 ceph-mon[30953]: pgmap v697: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 836 KiB/s wr, 343 op/s 2024-09-12T03:31:03.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:02 smithi135 ceph-mon[35474]: pgmap v697: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 836 KiB/s wr, 343 op/s 2024-09-12T03:31:03.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:02 smithi121 ceph-mon[34963]: pgmap v697: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 836 KiB/s wr, 343 op/s 2024-09-12T03:31:04.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:04 smithi019 ceph-mon[30953]: pgmap v698: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 391 op/s 2024-09-12T03:31:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:04.915 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:04 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:04 smithi135 ceph-mon[35474]: pgmap v698: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 391 op/s 2024-09-12T03:31:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:05.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:04 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:05.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:04 smithi121 ceph-mon[34963]: pgmap v698: 97 pgs: 97 active+clean; 1.7 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 391 op/s 2024-09-12T03:31:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:05.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:04 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:06.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:06 smithi019 ceph-mon[30953]: pgmap v699: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 432 op/s 2024-09-12T03:31:07.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:06 smithi135 ceph-mon[35474]: pgmap v699: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 432 op/s 2024-09-12T03:31:07.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:06 smithi121 ceph-mon[34963]: pgmap v699: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 432 op/s 2024-09-12T03:31:08.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:08 smithi019 ceph-mon[30953]: pgmap v700: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-09-12T03:31:09.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:08 smithi135 ceph-mon[35474]: pgmap v700: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-09-12T03:31:09.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:08 smithi121 ceph-mon[34963]: pgmap v700: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 568 op/s 2024-09-12T03:31:10.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:10 smithi019 ceph-mon[30953]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-12T03:31:11.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:10 smithi135 ceph-mon[35474]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-12T03:31:11.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:10 smithi121 ceph-mon[34963]: pgmap v701: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 534 op/s 2024-09-12T03:31:13.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:12 smithi135 ceph-mon[35474]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 650 op/s 2024-09-12T03:31:13.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:12 smithi121 ceph-mon[34963]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 650 op/s 2024-09-12T03:31:13.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:12 smithi019 ceph-mon[30953]: pgmap v702: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 650 op/s 2024-09-12T03:31:15.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:14 smithi135 ceph-mon[35474]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 694 op/s 2024-09-12T03:31:15.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:14 smithi121 ceph-mon[34963]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 694 op/s 2024-09-12T03:31:15.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:14 smithi019 ceph-mon[30953]: pgmap v703: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 694 op/s 2024-09-12T03:31:17.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:16 smithi135 ceph-mon[35474]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-09-12T03:31:17.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:16 smithi121 ceph-mon[34963]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-09-12T03:31:17.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:16 smithi019 ceph-mon[30953]: pgmap v704: 97 pgs: 97 active+clean; 1.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 690 op/s 2024-09-12T03:31:19.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:18 smithi135 ceph-mon[35474]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 715 op/s 2024-09-12T03:31:19.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:18 smithi121 ceph-mon[34963]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 715 op/s 2024-09-12T03:31:19.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:18 smithi019 ceph-mon[30953]: pgmap v705: 97 pgs: 97 active+clean; 1.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 715 op/s 2024-09-12T03:31:21.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:20 smithi135 ceph-mon[35474]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:31:21.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:20 smithi121 ceph-mon[34963]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:31:21.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:20 smithi019 ceph-mon[30953]: pgmap v706: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 571 op/s 2024-09-12T03:31:23.008 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:31:23.033 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:22 smithi019 ceph-mon[30953]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 581 op/s 2024-09-12T03:31:23.043 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:22 smithi135 ceph-mon[35474]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 581 op/s 2024-09-12T03:31:23.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:22 smithi121 ceph-mon[34963]: pgmap v707: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 581 op/s 2024-09-12T03:31:23.269 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:23.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 -- 172.21.15.19:0/3211060858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181036e0 msgr2=0x7f6c18107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/3211060858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181036e0 0x7f6c18107b90 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009930 tx=0x7f6c0c02a420 comp rx=0 tx=0).stop 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 -- 172.21.15.19:0/3211060858 shutdown_connections 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/3211060858 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181082a0 0x7f6c1810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/3211060858 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181036e0 0x7f6c18107b90 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/3211060858 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 0x7f6c18103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 -- 172.21.15.19:0/3211060858 >> 172.21.15.19:0/3211060858 conn(0x7f6c180fc7d0 msgr2=0x7f6c180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 -- 172.21.15.19:0/3211060858 shutdown_connections 2024-09-12T03:31:23.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.818+0000 7f6c1d96f640 1 -- 172.21.15.19:0/3211060858 wait complete. 2024-09-12T03:31:23.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.819+0000 7f6c1d96f640 1 Processor -- start 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.819+0000 7f6c1d96f640 1 -- start start 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.819+0000 7f6c1d96f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 0x7f6c181a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c1d96f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 0x7f6c181a4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c1d96f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c1d96f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c181120a0 con 0x7f6c181082a0 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c1d96f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18111f20 con 0x7f6c181036e0 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c1d96f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18112220 con 0x7f6c18102d30 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39954/0 (socket says 172.21.15.19:39954) 2024-09-12T03:31:23.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c177fe640 1 -- 172.21.15.19:0/2474131507 learned_addr learned my addr 172.21.15.19:0/2474131507 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 0x7f6c181a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c167fc640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 0x7f6c181a4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.820+0000 7f6c177fe640 1 -- 172.21.15.19:0/2474131507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 msgr2=0x7f6c181a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c177fe640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 0x7f6c181a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c177fe640 1 -- 172.21.15.19:0/2474131507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 msgr2=0x7f6c181a4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c177fe640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 0x7f6c181a4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c177fe640 1 -- 172.21.15.19:0/2474131507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1810e5d0 con 0x7f6c181082a0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c167fc640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 0x7f6c181a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c177fe640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009fd0 tx=0x7f6c1000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c1c96d640 1 -- 172.21.15.19:0/2474131507 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10017070 con 0x7f6c181082a0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1810e8c0 con 0x7f6c181082a0 2024-09-12T03:31:23.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c1c96d640 1 -- 172.21.15.19:0/2474131507 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c10003e50 con 0x7f6c181082a0 2024-09-12T03:31:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c181ab700 con 0x7f6c181082a0 2024-09-12T03:31:23.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.821+0000 7f6c1c96d640 1 -- 172.21.15.19:0/2474131507 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c100106c0 con 0x7f6c181082a0 2024-09-12T03:31:23.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.825+0000 7f6c1c96d640 1 -- 172.21.15.19:0/2474131507 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c10010860 con 0x7f6c181082a0 2024-09-12T03:31:23.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 Processor -- start 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 -- start start 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1c96d640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 0x7f6c000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df63b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df646140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1c96d640 1 -- 172.21.15.19:0/2474131507 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c100ceab0 con 0x7f6c181082a0 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b2df5a12e0 con 0x55b2df646680 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2df548ce0 con 0x55b2df63b4a0 2024-09-12T03:31:23.830 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.826+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2df59d2a0 con 0x55b2df63bdc0 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39958/0 (socket says 172.21.15.19:39958) 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 -- 172.21.15.19:0/101399134 learned_addr learned my addr 172.21.15.19:0/101399134 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df63b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 -- 172.21.15.19:0/101399134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 msgr2=0x55b2df646140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df646140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 -- 172.21.15.19:0/101399134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 msgr2=0x55b2df63b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df63b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 -- 172.21.15.19:0/101399134 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df52d440 con 0x55b2df646680 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.827+0000 7f6c177fe640 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6c1009c2e0 tx=0x7f6c1009c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f547313e54e3a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.828+0000 7f6c167fc640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 0x7f6c000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.828+0000 7f6c167fc640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 0x7f6c000a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005940 tx=0x7f6c0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.829+0000 7f6bfdffb640 1 -- 172.21.15.19:0/101399134 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10020070 con 0x55b2df646680 2024-09-12T03:31:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.829+0000 7f6bfdffb640 1 -- 172.21.15.19:0/101399134 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c100a51a0 con 0x55b2df646680 2024-09-12T03:31:23.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.829+0000 7f6bfdffb640 1 -- 172.21.15.19:0/101399134 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10010b50 con 0x55b2df646680 2024-09-12T03:31:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 -- 172.21.15.19:0/101399134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 msgr2=0x55b2df648a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6c1009c2e0 tx=0x7f6c1009c210 comp rx=0 tx=0).stop 2024-09-12T03:31:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 -- 172.21.15.19:0/101399134 shutdown_connections 2024-09-12T03:31:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df646680 0x55b2df648a70 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df646140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/101399134 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df63b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 -- 172.21.15.19:0/101399134 >> 172.21.15.19:0/101399134 conn(0x55b2df637a30 msgr2=0x55b2df637e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 -- 172.21.15.19:0/101399134 shutdown_connections 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.830+0000 7f6c1fd48740 1 -- 172.21.15.19:0/101399134 wait complete. 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 -- start start 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 Processor -- start 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df646050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df643300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55b2df65a2b0 con 0x55b2df63bdc0 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.831+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55b2df55c050 con 0x55b2df63b4a0 2024-09-12T03:31:23.834 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55b2df65a430 con 0x55b2df646680 2024-09-12T03:31:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c1fd48740 10 client.0 _ll_register_callbacks cb 0x55b2df59c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:23.835 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c16ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df646050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c177fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45110/0 (socket says 172.21.15.19:45110) 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.832+0000 7f6c177fe640 1 -- 172.21.15.19:0/926922429 learned_addr learned my addr 172.21.15.19:0/926922429 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 -- 172.21.15.19:0/926922429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 msgr2=0x55b2df646050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df646050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 -- 172.21.15.19:0/926922429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 msgr2=0x55b2df643300 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df643300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 -- 172.21.15.19:0/926922429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df3123c0 con 0x55b2df646680 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6c177fe640 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c100a4c70 tx=0x7f6c10098c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e281fb11ffb9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.833+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10020070 con 0x55b2df646680 2024-09-12T03:31:23.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c100a51a0 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c10010b50 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6c10010d50 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 10 client.27133 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b2df65a2b0 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c100a51a0 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6c10003970 con 0x55b2df646680 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 10 client.27133 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 20 client.27133 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df678300 0x55b2df67a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 4 client.27133 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.834+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2df5ae850 con 0x55b2df678300 2024-09-12T03:31:23.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.835+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df678300 0x55b2df67a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.835+0000 7f6bf6ffd640 10 client.27133 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.835+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df678300 0x55b2df67a760 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b2fa649bde88ee server_cookie=9e894152901cc928 in_seq=0 out_seq=0 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.836+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/926922429 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2df5ae850 con 0x55b2df678300 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.836+0000 7f6bf6ffd640 10 client.27133 handle_command_reply: tid=0 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.836+0000 7f6c1fd48740 1 client.27133 shutdown 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 msgr2=0x55b2df6438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6c100a4c70 tx=0x7f6c10098c50 comp rx=0 tx=0).stop 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 shutdown_connections 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df646680 0x55b2df6438f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df643300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df678300 0x55b2df67a760 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/926922429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df63b4a0 0x55b2df646050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 >> 172.21.15.19:0/926922429 conn(0x55b2df637a30 msgr2=0x55b2df63d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6bf6ffd640 0 client.27133 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 shutdown_connections 2024-09-12T03:31:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 1 -- 172.21.15.19:0/926922429 wait complete. 2024-09-12T03:31:23.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.837+0000 7f6c1fd48740 20 client.27133 trim_cache size 0 max 16384 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.867+0000 7f6c1fd48740 1 Processor -- start 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 -- start start 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 0x55b2df680030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400e4c0 con 0x55b2df63b4a0 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c04010240 con 0x55b2df646680 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400eb90 con 0x55b2df63bdc0 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c167fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c167fc640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45114/0 (socket says 172.21.15.19:45114) 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c167fc640 1 -- 172.21.15.19:0/2224520979 learned_addr learned my addr 172.21.15.19:0/2224520979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:23.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 0x55b2df680030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.868+0000 7f6c177fe640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 -- 172.21.15.19:0/2224520979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 msgr2=0x55b2df680570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 -- 172.21.15.19:0/2224520979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 msgr2=0x55b2df63e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 -- 172.21.15.19:0/2224520979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2df63b2d0 con 0x55b2df63b4a0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c167fc640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 0x55b2df680030 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x55b2df6786e0 tx=0x7f6c08004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca1d8d1f8be7e80 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6bfd7fa640 1 -- 172.21.15.19:0/2224520979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08003d20 con 0x55b2df63b4a0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6bfd7fa640 1 -- 172.21.15.19:0/2224520979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0801d900 con 0x55b2df63b4a0 2024-09-12T03:31:23.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.869+0000 7f6bfd7fa640 1 -- 172.21.15.19:0/2224520979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08010980 con 0x55b2df63b4a0 2024-09-12T03:31:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 -- 172.21.15.19:0/2224520979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 msgr2=0x55b2df680030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 0x55b2df680030 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x55b2df6786e0 tx=0x7f6c08004410 comp rx=0 tx=0).stop 2024-09-12T03:31:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 -- 172.21.15.19:0/2224520979 shutdown_connections 2024-09-12T03:31:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.878 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63bdc0 0x55b2df680570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/2224520979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63b4a0 0x55b2df680030 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 -- 172.21.15.19:0/2224520979 >> 172.21.15.19:0/2224520979 conn(0x55b2df637a30 msgr2=0x55b2df645380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 -- 172.21.15.19:0/2224520979 shutdown_connections 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.870+0000 7f6c1fd48740 1 -- 172.21.15.19:0/2224520979 wait complete. 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.871+0000 7f6c1fd48740 1 -- start start 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.871+0000 7f6c1fd48740 1 Processor -- start 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.871+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.871+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df65fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.871+0000 7f6c1fd48740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400fab0 con 0x55b2df63bdc0 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400f320 con 0x55b2df646680 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c1fd48740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c0400c3e0 con 0x55b2df63b4a0 2024-09-12T03:31:23.879 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c1fd48740 10 client.0 _ll_register_callbacks cb 0x7f6c10098970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45124/0 (socket says 172.21.15.19:45124) 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 -- 172.21.15.19:0/3817514019 learned_addr learned my addr 172.21.15.19:0/3817514019 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c177fe640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c167fc640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df65fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 -- 172.21.15.19:0/3817514019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 msgr2=0x55b2df63fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 -- 172.21.15.19:0/3817514019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 msgr2=0x55b2df65fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df65fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c16ffd640 1 -- 172.21.15.19:0/3817514019 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2df6405b0 con 0x55b2df63b4a0 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c177fe640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.872+0000 7f6c167fc640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df65fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:23.880 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6c16ffd640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005bc0 tx=0x7f6c08010810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a887873e14a056 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6bf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0801d620 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c08003b20 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c0801d900 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6c08039050 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6c1fd48740 10 client.27139 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.873+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b2df6e2c30 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c0801dcc0 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6c08025070 con 0x55b2df63b4a0 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 10 client.27139 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 10 client.27139 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:23.881 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 20 client.27139 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df6e3370 0x55b2df6e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 4 client.27139 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6c0801d900 con 0x55b2df6e3370 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.874+0000 7f6c167fc640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df6e3370 0x55b2df6e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.875+0000 7f6c167fc640 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df6e3370 0x55b2df6e5790 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca60a043f22a59f server_cookie=f6de6f9d2f6dab64 in_seq=0 out_seq=0 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.875+0000 7f6bf6ffd640 10 client.27139 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.875+0000 7f6bf6ffd640 1 -- 172.21.15.19:0/3817514019 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f6c0801d900 con 0x55b2df6e3370 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.875+0000 7f6bf6ffd640 10 client.27139 handle_command_reply: tid=0 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.875+0000 7f6c1fd48740 1 client.27139 shutdown 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 msgr2=0x55b2df65f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005bc0 tx=0x7f6c08010810 comp rx=0 tx=0).stop 2024-09-12T03:31:23.882 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 shutdown_connections 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55b2df6e3370 0x55b2df6e5790 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55b2df646680 0x55b2df63fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55b2df63bdc0 0x55b2df65fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 --2- 172.21.15.19:0/3817514019 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55b2df63b4a0 0x55b2df65f9b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 >> 172.21.15.19:0/3817514019 conn(0x55b2df637a30 msgr2=0x55b2df63e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6bf6ffd640 0 client.27139 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 shutdown_connections 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 1 -- 172.21.15.19:0/3817514019 wait complete. 2024-09-12T03:31:23.883 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.876+0000 7f6c1fd48740 20 client.27139 trim_cache size 0 max 16384 2024-09-12T03:31:23.884 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:31:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:31:23.884 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:31:23.884 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:31:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 msgr2=0x7f6c000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 0x7f6c000a5040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c005940 tx=0x7f6c0c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 msgr2=0x7f6c1810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009fd0 tx=0x7f6c1000b260 comp rx=0 tx=0).stop 2024-09-12T03:31:23.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 shutdown_connections 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c000a2b80 0x7f6c000a5040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c181082a0 0x7f6c1810de50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c181036e0 0x7f6c181a4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 --2- 172.21.15.19:0/2474131507 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c18102d30 0x7f6c181a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 >> 172.21.15.19:0/2474131507 conn(0x7f6c180fc7d0 msgr2=0x7f6c18103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 shutdown_connections 2024-09-12T03:31:23.886 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:23.877+0000 7f6c1d96f640 1 -- 172.21.15.19:0/2474131507 wait complete. 2024-09-12T03:31:24.804 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:31:24.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:31:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:24 smithi135 ceph-mon[35474]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 486 op/s 2024-09-12T03:31:25.142 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:24 smithi121 ceph-mon[34963]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 486 op/s 2024-09-12T03:31:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:24 smithi019 ceph-mon[30953]: pgmap v708: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 486 op/s 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 -- 172.21.15.19:0/361114839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c102d30 msgr2=0x7f840c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 --2- 172.21.15.19:0/361114839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c102d30 0x7f840c103110 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009930 tx=0x7f83fc02a420 comp rx=0 tx=0).stop 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 -- 172.21.15.19:0/361114839 shutdown_connections 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 --2- 172.21.15.19:0/361114839 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 0x7f840c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 --2- 172.21.15.19:0/361114839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c1036e0 0x7f840c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.336 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 --2- 172.21.15.19:0/361114839 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c102d30 0x7f840c103110 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 -- 172.21.15.19:0/361114839 >> 172.21.15.19:0/361114839 conn(0x7f840c0fc7d0 msgr2=0x7f840c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 -- 172.21.15.19:0/361114839 shutdown_connections 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 -- 172.21.15.19:0/361114839 wait complete. 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.318+0000 7f84119c5640 1 Processor -- start 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 -- start start 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c102d30 0x7f840c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 0x7f840c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f840c1120a0 con 0x7f840c1036e0 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f840c111f20 con 0x7f840c102d30 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f84119c5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f840c112220 con 0x7f840c1082a0 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:26.337 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39994/0 (socket says 172.21.15.19:39994) 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 -- 172.21.15.19:0/789554865 learned_addr learned my addr 172.21.15.19:0/789554865 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c102d30 0x7f840c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840b7fe640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 0x7f840c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 -- 172.21.15.19:0/789554865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 msgr2=0x7f840c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 0x7f840c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 -- 172.21.15.19:0/789554865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c102d30 msgr2=0x7f840c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c102d30 0x7f840c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 -- 172.21.15.19:0/789554865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840c1a6a80 con 0x7f840c1036e0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.319+0000 7f840a7fc640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f840000e9f0 tx=0x7f840000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.320+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840000ccc0 con 0x7f840c1036e0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.320+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8400010910 con 0x7f840c1036e0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.320+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840c1a6d70 con 0x7f840c1036e0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.320+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000188a0 con 0x7f840c1036e0 2024-09-12T03:31:26.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.320+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840c1a7270 con 0x7f840c1036e0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.321+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f840c06a850 con 0x7f840c1036e0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.322+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8400010ac0 con 0x7f840c1036e0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.323+0000 7f84109c3640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 0x7f83f40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.323+0000 7f840affd640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 0x7f83f40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.323+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f84000cf7e0 con 0x7f840c1036e0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.323+0000 7f840affd640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 0x7f83f40a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009900 tx=0x7f83fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:26.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.326+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f840009c750 con 0x7f840c1036e0 2024-09-12T03:31:26.542 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.541+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f840c103110 con 0x7f840c1036e0 2024-09-12T03:31:26.543 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.542+0000 7f84109c3640 1 -- 172.21.15.19:0/789554865 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+12009 (secure 0 0 0) 0x7f840c103110 con 0x7f840c1036e0 2024-09-12T03:31:26.543 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:31:26.544 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":58,"btime":"2024-09-12T03:29:39:011743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:29:38.690241+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:31:26.544 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 58 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.544+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 msgr2=0x7f83f40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.544+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 0x7f83f40a50c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f83fc009900 tx=0x7f83fc0058d0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.544+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 msgr2=0x7f840c100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.544+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f840000e9f0 tx=0x7f840000edf0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 shutdown_connections 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f840c1082a0 0x7f840c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f83f40a2c00 0x7f83f40a50c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f840c1036e0 0x7f840c100ed0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 --2- 172.21.15.19:0/789554865 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f840c102d30 0x7f840c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:26.546 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 >> 172.21.15.19:0/789554865 conn(0x7f840c0fc7d0 msgr2=0x7f840c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 shutdown_connections 2024-09-12T03:31:26.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:26.545+0000 7f84119c5640 1 -- 172.21.15.19:0/789554865 wait complete. 2024-09-12T03:31:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:27 smithi135 ceph-mon[35474]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 420 op/s 2024-09-12T03:31:27.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/789554865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:31:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:27 smithi121 ceph-mon[34963]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 420 op/s 2024-09-12T03:31:27.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/789554865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:31:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:27 smithi019 ceph-mon[30953]: pgmap v709: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 420 op/s 2024-09-12T03:31:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/789554865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:31:27.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:31:28.230 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 -- 172.21.15.19:0/4207420223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 msgr2=0x7f58681029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 --2- 172.21.15.19:0/4207420223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f58681029b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5858009900 tx=0x7f585802a480 comp rx=0 tx=0).stop 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 -- 172.21.15.19:0/4207420223 shutdown_connections 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 --2- 172.21.15.19:0/4207420223 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868102ef0 0x7f586810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 --2- 172.21.15.19:0/4207420223 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f58681029b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 --2- 172.21.15.19:0/4207420223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868100e20 0x7f5868101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 -- 172.21.15.19:0/4207420223 >> 172.21.15.19:0/4207420223 conn(0x7f58680fc7d0 msgr2=0x7f58680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 -- 172.21.15.19:0/4207420223 shutdown_connections 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.804+0000 7f586feeb640 1 -- 172.21.15.19:0/4207420223 wait complete. 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 Processor -- start 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 -- start start 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868100e20 0x7f58681984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 0x7f586819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110f80 con 0x7f5868102570 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5868110e00 con 0x7f5868102ef0 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586feeb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5868111100 con 0x7f5868100e20 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586d486640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586d486640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40010/0 (socket says 172.21.15.19:40010) 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586d486640 1 -- 172.21.15.19:0/2549905479 learned_addr learned my addr 172.21.15.19:0/2549905479 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.805+0000 7f586e488640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 0x7f586819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586dc87640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868100e20 0x7f58681984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 -- 172.21.15.19:0/2549905479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868100e20 msgr2=0x7f58681984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868100e20 0x7f58681984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 -- 172.21.15.19:0/2549905479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 msgr2=0x7f586819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 0x7f586819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 -- 172.21.15.19:0/2549905479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586819d490 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586e488640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 0x7f586819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586d486640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f585effd640 1 -- 172.21.15.19:0/2549905479 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585803f070 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586819d720 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f585effd640 1 -- 172.21.15.19:0/2549905479 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5858031c30 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f585effd640 1 -- 172.21.15.19:0/2549905479 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858035030 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.806+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58681a5060 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 Processor -- start 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f585effd640 1 -- 172.21.15.19:0/2549905479 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5858042070 con 0x7f5868102570 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 -- start start 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 0x5646ed1b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1bf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646ed1182f0 con 0x5646ed1b3b40 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646ed0bfcf0 con 0x5646ed1b25d0 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f58722c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646ed1142b0 con 0x5646ed1bd370 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f586e488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f586e488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40018/0 (socket says 172.21.15.19:40018) 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f586e488640 1 -- 172.21.15.19:0/2378778050 learned_addr learned my addr 172.21.15.19:0/2378778050 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.809+0000 7f586dc87640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 0x5646ed1b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586d486640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1bf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 -- 172.21.15.19:0/2378778050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 msgr2=0x5646ed1bf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1bf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 -- 172.21.15.19:0/2378778050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 msgr2=0x5646ed1b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 0x5646ed1b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 -- 172.21.15.19:0/2378778050 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ed0a43a0 con 0x5646ed1b3b40 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586dc87640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 0x5646ed1b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f585effd640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 0x7f58540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586dc87640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 0x7f58540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f586e488640 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5860010750 tx=0x7f5860010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a9387786d23e85 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f585effd640 1 -- 172.21.15.19:0/2549905479 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58580ef400 con 0x7f5868102570 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f5853fff640 1 -- 172.21.15.19:0/2378778050 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586000ead0 con 0x5646ed1b3b40 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f5853fff640 1 -- 172.21.15.19:0/2378778050 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5860010e60 con 0x5646ed1b3b40 2024-09-12T03:31:28.820 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.810+0000 7f5853fff640 1 -- 172.21.15.19:0/2378778050 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5860011640 con 0x5646ed1b3b40 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f586dc87640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 0x7f58540a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x5646ed1afac0 tx=0x7f5864005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- 172.21.15.19:0/2378778050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 msgr2=0x5646ed1b2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5860010750 tx=0x7f5860010c10 comp rx=0 tx=0).stop 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- 172.21.15.19:0/2378778050 shutdown_connections 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1bf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b25d0 0x5646ed1b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 --2- 172.21.15.19:0/2378778050 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b3b40 0x5646ed1b2090 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- 172.21.15.19:0/2378778050 >> 172.21.15.19:0/2378778050 conn(0x5646ed17b310 msgr2=0x5646ed1aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.821 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- 172.21.15.19:0/2378778050 shutdown_connections 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- 172.21.15.19:0/2378778050 wait complete. 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.811+0000 7f58722c4740 1 -- start start 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 Processor -- start 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1ba560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5646ed1c0070 con 0x5646ed1b25d0 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5646ed0d3060 con 0x5646ed1b3b40 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5646ed1c01f0 con 0x5646ed1bd370 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f58722c4740 10 client.0 _ll_register_callbacks cb 0x5646ece268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f586dc87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.822 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f586dc87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40020/0 (socket says 172.21.15.19:40020) 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f586dc87640 1 -- 172.21.15.19:0/4033621915 learned_addr learned my addr 172.21.15.19:0/4033621915 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f586d486640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.812+0000 7f586e488640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1ba560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 -- 172.21.15.19:0/4033621915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 msgr2=0x5646ed1ba560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1ba560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 -- 172.21.15.19:0/4033621915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 msgr2=0x5646ed1bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 -- 172.21.15.19:0/4033621915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ece893c0 con 0x5646ed1b3b40 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586dc87640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f586d486640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1b9fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58580ecf60 tx=0x7f5858009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f230fbbd92fe87d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58517fa640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:31:28.823 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049070 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58580f2d30 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5860010e60 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f58580d9050 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58722c4740 10 client.27182 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5646ed1cbac0 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.813+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5860010e60 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f585804c060 con 0x5646ed1b3b40 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 10 client.27182 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 10 client.27182 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 20 client.27182 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed1cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 4 client.27182 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:28.824 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ed125860 con 0x5646ed1ccf00 2024-09-12T03:31:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f586dc87640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed1cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f586dc87640 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed1cf360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7feeffccc7fda3bb server_cookie=2e4fff02529eb107 in_seq=0 out_seq=0 2024-09-12T03:31:28.825 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.814+0000 7f58517fa640 10 client.27182 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58517fa640 1 -- 172.21.15.19:0/4033621915 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646ed125860 con 0x5646ed1ccf00 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58517fa640 10 client.27182 handle_command_reply: tid=0 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 client.27182 shutdown 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 msgr2=0x5646ed1b9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1b9fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f58580ecf60 tx=0x7f5858009620 comp rx=0 tx=0).stop 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 shutdown_connections 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed1cf360 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1bd370 0x5646ed1ba560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58517fa640 0 client.27182 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:31:28.826 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1b9fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 --2- 172.21.15.19:0/4033621915 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 >> 172.21.15.19:0/4033621915 conn(0x5646ed17b310 msgr2=0x5646ed1b4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 shutdown_connections 2024-09-12T03:31:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 1 -- 172.21.15.19:0/4033621915 wait complete. 2024-09-12T03:31:28.827 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.815+0000 7f58722c4740 20 client.27182 trim_cache size 0 max 16384 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 Processor -- start 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 -- start start 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 0x5646ed1b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1bd370 0x5646ed1b8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58681ad5a0 con 0x5646ed1bd370 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58681af320 con 0x5646ed1b3b40 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f58722c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58681adc70 con 0x5646ed1b25d0 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.831 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586e488640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1bd370 0x5646ed1b8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:53094/0 (socket says 172.21.15.19:53094) 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 -- 172.21.15.19:0/874637179 learned_addr learned my addr 172.21.15.19:0/874637179 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586dc87640 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 0x5646ed1b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 -- 172.21.15.19:0/874637179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 msgr2=0x5646ed1b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 0x5646ed1b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 -- 172.21.15.19:0/874637179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1bd370 msgr2=0x5646ed1b8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1bd370 0x5646ed1b8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.829+0000 7f586d486640 1 -- 172.21.15.19:0/874637179 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ed1bd080 con 0x5646ed1b3b40 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f586dc87640 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 0x5646ed1b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f586d486640 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5858005940 tx=0x7f58580b78b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=614cee297e1dc08c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f58537fe640 1 -- 172.21.15.19:0/874637179 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858049040 con 0x5646ed1b3b40 2024-09-12T03:31:28.832 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f58537fe640 1 -- 172.21.15.19:0/874637179 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58580bc370 con 0x5646ed1b3b40 2024-09-12T03:31:28.833 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f58537fe640 1 -- 172.21.15.19:0/874637179 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580c4450 con 0x5646ed1b3b40 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f58722c4740 1 -- 172.21.15.19:0/874637179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 msgr2=0x5646ed1c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.830+0000 7f58722c4740 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5858005940 tx=0x7f58580b78b0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- 172.21.15.19:0/874637179 shutdown_connections 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1bd370 0x5646ed1b8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1b3b40 0x5646ed1c9310 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- 172.21.15.19:0/874637179 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b25d0 0x5646ed1b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- 172.21.15.19:0/874637179 >> 172.21.15.19:0/874637179 conn(0x5646ed17b310 msgr2=0x5646ed1a9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- 172.21.15.19:0/874637179 shutdown_connections 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- 172.21.15.19:0/874637179 wait complete. 2024-09-12T03:31:28.836 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- start start 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 Processor -- start 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b3b40 0x5646ed1c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1bd370 0x5646ed232be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.831+0000 7f58722c4740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f58681aeb90 con 0x5646ed1b25d0 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58722c4740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f58681ae400 con 0x5646ed1bd370 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58722c4740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f58681ab4c0 con 0x5646ed1b3b40 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58722c4740 10 client.0 _ll_register_callbacks cb 0x7f5858005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40036/0 (socket says 172.21.15.19:40036) 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586d486640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b3b40 0x5646ed1c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 -- 172.21.15.19:0/4146445840 learned_addr learned my addr 172.21.15.19:0/4146445840 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:28.837 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586e488640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1bd370 0x5646ed232be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 -- 172.21.15.19:0/4146445840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b3b40 msgr2=0x5646ed1c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b3b40 0x5646ed1c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 -- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1bd370 msgr2=0x5646ed232be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1bd370 0x5646ed232be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 -- 172.21.15.19:0/4146445840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ed2331a0 con 0x5646ed1b25d0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f586dc87640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5864009d20 tx=0x7f58640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6971ace1a67f207 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58517fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864004020 con 0x5646ed1b25d0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f58640041c0 con 0x5646ed1b25d0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.832+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58580bc370 con 0x5646ed1b25d0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f58640043c0 con 0x5646ed1b25d0 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 10 client.17856 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:28.838 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5646ed1b7080 con 0x5646ed1b25d0 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f58580bc370 con 0x5646ed1b25d0 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f5864003ba0 con 0x5646ed1b25d0 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 10 client.17856 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 10 client.17856 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 20 client.17856 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed23e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 4 client.17856 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646ed11fd70 con 0x5646ed1ccf00 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.833+0000 7f586d486640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed23e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f586d486640 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed23e110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8141945febe4123b server_cookie=bfa4781d0f04a2cc in_seq=0 out_seq=0 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58517fa640 10 client.17856 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58517fa640 1 -- 172.21.15.19:0/4146445840 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646ed11fd70 con 0x5646ed1ccf00 2024-09-12T03:31:28.839 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58517fa640 10 client.17856 handle_command_reply: tid=0 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 client.17856 shutdown 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 msgr2=0x5646ed1c5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5864009d20 tx=0x7f58640046c0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 shutdown_connections 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5646ed1bd370 0x5646ed232be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x5646ed1ccf00 0x5646ed23e110 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5646ed1b3b40 0x5646ed1c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 --2- 172.21.15.19:0/4146445840 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5646ed1b25d0 0x5646ed1c5480 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58517fa640 0 client.17856 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 >> 172.21.15.19:0/4146445840 conn(0x5646ed17b310 msgr2=0x5646ed1b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 shutdown_connections 2024-09-12T03:31:28.840 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 1 -- 172.21.15.19:0/4146445840 wait complete. 2024-09-12T03:31:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.834+0000 7f58722c4740 20 client.17856 trim_cache size 0 max 16384 2024-09-12T03:31:28.841 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:31:28.841 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 msgr2=0x7f58540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 0x7f58540a5040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x5646ed1afac0 tx=0x7f5864005f90 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 msgr2=0x7f5868198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5858006010 tx=0x7f5858003c10 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 shutdown_connections 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5868102ef0 0x7f586819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f58540a2b80 0x7f58540a5040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5868102570 0x7f5868198a20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 --2- 172.21.15.19:0/2549905479 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5868100e20 0x7f58681984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 >> 172.21.15.19:0/2549905479 conn(0x7f58680fc7d0 msgr2=0x7f586806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 shutdown_connections 2024-09-12T03:31:28.842 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:28.835+0000 7f586feeb640 1 -- 172.21.15.19:0/2549905479 wait complete. 2024-09-12T03:31:29.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:29 smithi135 ceph-mon[35474]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 450 op/s 2024-09-12T03:31:29.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:29 smithi121 ceph-mon[34963]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 450 op/s 2024-09-12T03:31:29.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:29 smithi019 ceph-mon[30953]: pgmap v710: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 450 op/s 2024-09-12T03:31:29.563 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:31:29.834 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:30.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 -- 172.21.15.19:0/246485982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f8103660 msgr2=0x7f40f810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 --2- 172.21.15.19:0/246485982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f8103660 0x7f40f810bb50 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009900 tx=0x7f40ec02a480 comp rx=0 tx=0).stop 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 -- 172.21.15.19:0/246485982 shutdown_connections 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 --2- 172.21.15.19:0/246485982 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f810c170 0x7f40f810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 --2- 172.21.15.19:0/246485982 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f8103660 0x7f40f810bb50 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 --2- 172.21.15.19:0/246485982 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8102d40 0x7f40f8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 -- 172.21.15.19:0/246485982 >> 172.21.15.19:0/246485982 conn(0x7f40f80fc7a0 msgr2=0x7f40f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 -- 172.21.15.19:0/246485982 shutdown_connections 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.324+0000 7f40fe806640 1 -- 172.21.15.19:0/246485982 wait complete. 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 Processor -- start 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 -- start start 2024-09-12T03:31:30.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f8102d40 0x7f40f819a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 0x7f40f819ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8102930 con 0x7f40f810c170 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81027b0 con 0x7f40f8103660 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fe806640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40f8114320 con 0x7f40f8102d40 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fcda3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 0x7f40f819ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fcda3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40056/0 (socket says 172.21.15.19:40056) 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40fcda3640 1 -- 172.21.15.19:0/2244595576 learned_addr learned my addr 172.21.15.19:0/2244595576 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.325+0000 7f40f7fff640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f8102d40 0x7f40f819a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 -- 172.21.15.19:0/2244595576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f8102d40 msgr2=0x7f40f819a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f8102d40 0x7f40f819a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 -- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 msgr2=0x7f40f819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 -- 172.21.15.19:0/2244595576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f819f5e0 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40fcda3640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f77fe640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 0x7f40f819ab70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005940 tx=0x7f40ec0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f57fa640 1 -- 172.21.15.19:0/2244595576 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec03f030 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f819f870 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f57fa640 1 -- 172.21.15.19:0/2244595576 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40ec0044c0 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40f57fa640 1 -- 172.21.15.19:0/2244595576 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec035070 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.326+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f81a71b0 con 0x7f40f8103660 2024-09-12T03:31:30.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f57fa640 1 -- 172.21.15.19:0/2244595576 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40ec042030 con 0x7f40f8103660 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 Processor -- start 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 -- start start 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f29370 0x55e6f0f2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0e842f0 con 0x55e6f0f29370 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0e2bcf0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f4100bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0e802b0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40fcda3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f29370 0x55e6f0f2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40fcda3640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f1e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45238/0 (socket says 172.21.15.19:45238) 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40fcda3640 1 -- 172.21.15.19:0/1385179619 learned_addr learned my addr 172.21.15.19:0/1385179619 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f57fa640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 0x7f40e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 -- 172.21.15.19:0/1385179619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 msgr2=0x55e6f0f1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 -- 172.21.15.19:0/1385179619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f29370 msgr2=0x55e6f0f2b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f29370 0x55e6f0f2b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.329+0000 7f40f7fff640 1 -- 172.21.15.19:0/1385179619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f0e103a0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40fcda3640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 0x7f40e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40f57fa640 1 -- 172.21.15.19:0/2244595576 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40ec0ef2d0 con 0x7f40f8103660 2024-09-12T03:31:30.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f1ea10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009c20 tx=0x7f40e8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20c8987c20042cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40de7fc640 1 -- 172.21.15.19:0/1385179619 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004050 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40de7fc640 1 -- 172.21.15.19:0/1385179619 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005500 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40de7fc640 1 -- 172.21.15.19:0/1385179619 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8004dd0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.330+0000 7f40fcda3640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 0x7f40e00a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007fd0 tx=0x7f40f0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- 172.21.15.19:0/1385179619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 msgr2=0x55e6f0f1ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f1ea10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009c20 tx=0x7f40e8011810 comp rx=0 tx=0).stop 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- 172.21.15.19:0/1385179619 shutdown_connections 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f29370 0x55e6f0f2b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f1ea10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1385179619 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- 172.21.15.19:0/1385179619 >> 172.21.15.19:0/1385179619 conn(0x55e6f0ee7310 msgr2=0x55e6f0f1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- 172.21.15.19:0/1385179619 shutdown_connections 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- 172.21.15.19:0/1385179619 wait complete. 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.331+0000 7f4100bdf740 1 -- start start 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 Processor -- start 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f28c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f26470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0f2c070 con 0x55e6f0f1fb40 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0e3f060 con 0x55e6f0f29370 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e6f0f2c1f0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f4100bdf740 10 client.0 _ll_register_callbacks cb 0x55e6f0b928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40084/0 (socket says 172.21.15.19:40084) 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 -- 172.21.15.19:0/1624432116 learned_addr learned my addr 172.21.15.19:0/1624432116 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40fcda3640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f26470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f28c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 -- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 msgr2=0x55e6f0f28c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f28c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 -- 172.21.15.19:0/1624432116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 msgr2=0x55e6f0f26470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f26470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 -- 172.21.15.19:0/1624432116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f0bf53c0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40f77fe640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0bb8f0 tx=0x7f40ec02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f15ae28f46841b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.332+0000 7f40d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec049070 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40ec0b8260 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e8005500 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f40ec0b7e90 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 10 client.17868 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e6f0f37a60 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40e8005500 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f40ec0bb040 con 0x55e6f0f1fb40 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 10 client.17868 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 10 client.17868 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:31:30.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 20 client.17868 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0f38ef0 0x55e6f0f3b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 4 client.17868 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f0e91860 con 0x55e6f0f38ef0 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.333+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0f38ef0 0x55e6f0f3b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.334+0000 7f40f7fff640 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0f38ef0 0x55e6f0f3b330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a9dbb5dc87b5b41 server_cookie=7e1ae3fb367162cc in_seq=0 out_seq=0 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.334+0000 7f40d77fe640 10 client.17868 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f40d77fe640 1 -- 172.21.15.19:0/1624432116 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6f0e91860 con 0x55e6f0f38ef0 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f40d77fe640 10 client.17868 handle_command_reply: tid=0 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 client.17868 shutdown 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 msgr2=0x55e6f0f25f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0bb8f0 tx=0x7f40ec02a480 comp rx=0 tx=0).stop 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 shutdown_connections 2024-09-12T03:31:30.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0f38ef0 0x55e6f0f3b330 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f26470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f25f30 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f40d77fe640 0 client.17868 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 --2- 172.21.15.19:0/1624432116 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f28c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 >> 172.21.15.19:0/1624432116 conn(0x55e6f0ee7310 msgr2=0x55e6f0f20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 shutdown_connections 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 1 -- 172.21.15.19:0/1624432116 wait complete. 2024-09-12T03:31:30.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.335+0000 7f4100bdf740 20 client.17868 trim_cache size 0 max 16384 2024-09-12T03:31:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 Processor -- start 2024-09-12T03:31:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 -- start start 2024-09-12T03:31:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f27ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f21120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f21660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81af7c0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b1540 con 0x55e6f0f29370 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f4100bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81afe90 con 0x55e6f0f1fb40 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f27ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f40fcda3640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f21660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f40f7fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f27ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40094/0 (socket says 172.21.15.19:40094) 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f40f7fff640 1 -- 172.21.15.19:0/225276058 learned_addr learned my addr 172.21.15.19:0/225276058 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.349+0000 7f40f77fe640 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f21120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 -- 172.21.15.19:0/225276058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 msgr2=0x55e6f0f21120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f21120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 -- 172.21.15.19:0/225276058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 msgr2=0x55e6f0f27ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f27ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 -- 172.21.15.19:0/225276058 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f0f24f00 con 0x55e6f0f29370 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40fcda3640 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f21660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007ef0 tx=0x7f40f0015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb61e36d41c97a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40ddffb640 1 -- 172.21.15.19:0/225276058 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003c40 con 0x55e6f0f29370 2024-09-12T03:31:30.359 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40ddffb640 1 -- 172.21.15.19:0/225276058 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f0003de0 con 0x55e6f0f29370 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.350+0000 7f40ddffb640 1 -- 172.21.15.19:0/225276058 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f001b450 con 0x55e6f0f29370 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- 172.21.15.19:0/225276058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 msgr2=0x55e6f0f21660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f21660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007ef0 tx=0x7f40f0015000 comp rx=0 tx=0).stop 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- 172.21.15.19:0/225276058 shutdown_connections 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f21660 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f21120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 --2- 172.21.15.19:0/225276058 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f27ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- 172.21.15.19:0/225276058 >> 172.21.15.19:0/225276058 conn(0x55e6f0ee7310 msgr2=0x55e6f0f27790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- 172.21.15.19:0/225276058 shutdown_connections 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- 172.21.15.19:0/225276058 wait complete. 2024-09-12T03:31:30.360 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 -- start start 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.351+0000 7f4100bdf740 1 Processor -- start 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f2e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b0db0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81b0620 con 0x55e6f0f29370 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40f81ad6e0 con 0x55e6f0f1e5d0 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f4100bdf740 10 client.0 _ll_register_callbacks cb 0x7f40ec009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:40098/0 (socket says 172.21.15.19:40098) 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 -- 172.21.15.19:0/3933897494 learned_addr learned my addr 172.21.15.19:0/3933897494 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40fcda3640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f2e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.361 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f7fff640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 -- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 msgr2=0x55e6f0f314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 -- 172.21.15.19:0/3933897494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 msgr2=0x55e6f0f2e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f2e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 -- 172.21.15.19:0/3933897494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f0f2e9c0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f7fff640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40f77fe640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0b85c0 tx=0x7f40ec03b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddc3a4ed382bbe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.352+0000 7f40d77fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:30.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec049040 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40ec03bc70 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f0003de0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f40ec0bb550 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 10 client.17874 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e6f0fa4210 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40f0003de0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f40ec0da1d0 con 0x55e6f0f1fb40 2024-09-12T03:31:30.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 10 client.17874 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 10 client.17874 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 20 client.17874 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0fa48d0 0x55e6f0fa6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 4 client.17874 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e6f0e8bd70 con 0x55e6f0fa48d0 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.353+0000 7f40f7fff640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0fa48d0 0x55e6f0fa6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f40f7fff640 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0fa48d0 0x55e6f0fa6cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b6aeb13e334f69f server_cookie=4fd50a0697b791dd in_seq=0 out_seq=0 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f40d77fe640 10 client.17874 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f40d77fe640 1 -- 172.21.15.19:0/3933897494 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e6f0e8bd70 con 0x55e6f0fa48d0 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f40d77fe640 10 client.17874 handle_command_reply: tid=0 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 client.17874 shutdown 2024-09-12T03:31:30.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 msgr2=0x55e6f0f319f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0b85c0 tx=0x7f40ec03b5d0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 shutdown_connections 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55e6f0fa48d0 0x55e6f0fa6cf0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e6f0f29370 0x55e6f0f2e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e6f0f1fb40 0x55e6f0f319f0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 --2- 172.21.15.19:0/3933897494 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e6f0f1e5d0 0x55e6f0f314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 >> 172.21.15.19:0/3933897494 conn(0x55e6f0ee7310 msgr2=0x55e6f0f27fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f40d77fe640 0 client.17874 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 shutdown_connections 2024-09-12T03:31:30.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 1 -- 172.21.15.19:0/3933897494 wait complete. 2024-09-12T03:31:30.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.354+0000 7f4100bdf740 20 client.17874 trim_cache size 0 max 16384 2024-09-12T03:31:30.366 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:31:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 msgr2=0x7f40e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 0x7f40e00a5040 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f40f0007fd0 tx=0x7f40f0004a40 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 msgr2=0x7f40f819ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 0x7f40f819ab70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f40ec005940 tx=0x7f40ec0040c0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 shutdown_connections 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40e00a2b80 0x7f40e00a5040 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40f810c170 0x7f40f819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f8103660 0x7f40f819ab70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 --2- 172.21.15.19:0/2244595576 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40f8102d40 0x7f40f819a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 >> 172.21.15.19:0/2244595576 conn(0x7f40f80fc7a0 msgr2=0x7f40f80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:30.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 shutdown_connections 2024-09-12T03:31:30.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:30.355+0000 7f40fe806640 1 -- 172.21.15.19:0/2244595576 wait complete. 2024-09-12T03:31:31.139 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:31 smithi019 ceph-mon[30953]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 392 op/s 2024-09-12T03:31:31.167 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:31:31.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:31 smithi135 ceph-mon[35474]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 392 op/s 2024-09-12T03:31:31.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:31 smithi121 ceph-mon[34963]: pgmap v711: 97 pgs: 97 active+clean; 1.6 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 392 op/s 2024-09-12T03:31:31.427 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 -- 172.21.15.19:0/3162981994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4102790 msgr2=0x7f87b41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 --2- 172.21.15.19:0/3162981994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4102790 0x7f87b41067e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a802a410 comp rx=0 tx=0).stop 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 -- 172.21.15.19:0/3162981994 shutdown_connections 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 --2- 172.21.15.19:0/3162981994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4106f10 0x7f87b410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 --2- 172.21.15.19:0/3162981994 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4102790 0x7f87b41067e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 --2- 172.21.15.19:0/3162981994 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4101de0 0x7f87b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.889+0000 7f87b9da6640 1 -- 172.21.15.19:0/3162981994 >> 172.21.15.19:0/3162981994 conn(0x7f87b40fd8c0 msgr2=0x7f87b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.890+0000 7f87b9da6640 1 -- 172.21.15.19:0/3162981994 shutdown_connections 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.890+0000 7f87b9da6640 1 -- 172.21.15.19:0/3162981994 wait complete. 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.890+0000 7f87b9da6640 1 Processor -- start 2024-09-12T03:31:31.911 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.890+0000 7f87b9da6640 1 -- start start 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 0x7f87b4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4106f10 0x7f87b419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4111f30 con 0x7f87b4101de0 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87b4111db0 con 0x7f87b4102790 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b9da6640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41120b0 con 0x7f87b4106f10 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49430/0 (socket says 172.21.15.19:49430) 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 -- 172.21.15.19:0/300275601 learned_addr learned my addr 172.21.15.19:0/300275601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 0x7f87b4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b3fff640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4106f10 0x7f87b419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 -- 172.21.15.19:0/300275601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4106f10 msgr2=0x7f87b419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.912 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4106f10 0x7f87b419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 -- 172.21.15.19:0/300275601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 msgr2=0x7f87b4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 0x7f87b4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 -- 172.21.15.19:0/300275601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b419d3e0 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 0x7f87b4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.891+0000 7f87b37fe640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f87a400d950 tx=0x7f87a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.892+0000 7f87b0ff9640 1 -- 172.21.15.19:0/300275601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a4017030 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.892+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b419d6d0 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.892+0000 7f87b0ff9640 1 -- 172.21.15.19:0/300275601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a4004020 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.892+0000 7f87b0ff9640 1 -- 172.21.15.19:0/300275601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a40047a0 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.892+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b41a4fb0 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87b0ff9640 1 -- 172.21.15.19:0/300275601 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87a40041d0 con 0x7f87b4101de0 2024-09-12T03:31:31.913 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87bc17f740 1 Processor -- start 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87bc17f740 1 -- start start 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb28b40 0x56463fb27090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.894+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87bc17f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56463fa8d2f0 con 0x56463fb32370 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87bc17f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56463fa34cf0 con 0x56463fb275d0 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87bc17f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56463fa892b0 con 0x56463fb28b40 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39666/0 (socket says 172.21.15.19:39666) 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 -- 172.21.15.19:0/3791346528 learned_addr learned my addr 172.21.15.19:0/3791346528 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49442/0 (socket says 172.21.15.19:49442) 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b3fff640 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb28b40 0x56463fb27090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b0ff9640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 0x7f879c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 -- 172.21.15.19:0/3791346528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb28b40 msgr2=0x56463fb27090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb28b40 0x56463fb27090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 -- 172.21.15.19:0/3791346528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 msgr2=0x56463fb34760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 -- 172.21.15.19:0/3791346528 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463fa193a0 con 0x56463fb275d0 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b3fff640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 0x7f879c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b0ff9640 1 -- 172.21.15.19:0/300275601 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87a40cea20 con 0x7f87b4101de0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.895+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87a400fc70 tx=0x7f87a4017ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94c03a4f5da4e8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f8799ffb640 1 -- 172.21.15.19:0/3791346528 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a409b060 con 0x56463fb275d0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f8799ffb640 1 -- 172.21.15.19:0/3791346528 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a4066370 con 0x56463fb275d0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f8799ffb640 1 -- 172.21.15.19:0/3791346528 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a400fed0 con 0x56463fb275d0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 -- 172.21.15.19:0/3791346528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 msgr2=0x56463fb27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f87a400fc70 tx=0x7f87a4017ba0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87b3fff640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 0x7f879c0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f87ac005fd0 tx=0x7f87ac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 -- 172.21.15.19:0/3791346528 shutdown_connections 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb34760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb27a10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3791346528 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb28b40 0x56463fb27090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 -- 172.21.15.19:0/3791346528 >> 172.21.15.19:0/3791346528 conn(0x56463faf0310 msgr2=0x56463fb23c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 -- 172.21.15.19:0/3791346528 shutdown_connections 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.896+0000 7f87bc17f740 1 -- 172.21.15.19:0/3791346528 wait complete. 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 -- start start 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 Processor -- start 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb31d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x56463fb35010 con 0x56463fb28b40 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x56463fa48060 con 0x56463fb275d0 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x56463fb35190 con 0x56463fb32370 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87bc17f740 10 client.0 _ll_register_callbacks cb 0x56463f79b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb31d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49446/0 (socket says 172.21.15.19:49446) 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87b2ffd640 1 -- 172.21.15.19:0/3335521661 learned_addr learned my addr 172.21.15.19:0/3335521661 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.897+0000 7f87b3fff640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 -- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 msgr2=0x56463fb2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 -- 172.21.15.19:0/3335521661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 msgr2=0x56463fb31d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb31d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 -- 172.21.15.19:0/3335521661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463f7fe3c0 con 0x56463fb28b40 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b3fff640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb31d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a8031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5dac4a823e3b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f8792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:31.918 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8035040 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87a8004450 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a4066370 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f87a8005020 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87bc17f740 10 client.17886 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56463fb40a60 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.898+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a4066370 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f87a80035f0 con 0x56463fb28b40 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 10 client.17886 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 10 client.17886 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 20 client.17886 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fb41fc0 0x56463fb443d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 4 client.17886 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:31.919 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56463fa9a860 con 0x56463fb41fc0 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fb41fc0 0x56463fb443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.899+0000 7f87b37fe640 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fb41fc0 0x56463fb443d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f11708a57a18f9c6 server_cookie=fd8624816d78a81f in_seq=0 out_seq=0 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f8792ffd640 10 client.17886 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f8792ffd640 1 -- 172.21.15.19:0/3335521661 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56463fa9a860 con 0x56463fb41fc0 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f8792ffd640 10 client.17886 handle_command_reply: tid=0 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 client.17886 shutdown 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 msgr2=0x56463fb2efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a8031850 comp rx=0 tx=0).stop 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 shutdown_connections 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fb41fc0 0x56463fb443d0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb2efc0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f8792ffd640 0 client.17886 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:31:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 --2- 172.21.15.19:0/3335521661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb31d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 >> 172.21.15.19:0/3335521661 conn(0x56463faf0310 msgr2=0x56463fb299c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.900+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 shutdown_connections 2024-09-12T03:31:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.901+0000 7f87bc17f740 1 -- 172.21.15.19:0/3335521661 wait complete. 2024-09-12T03:31:31.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.901+0000 7f87bc17f740 20 client.17886 trim_cache size 0 max 16384 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 Processor -- start 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 -- start start 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb28350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ad630 con 0x56463fb28b40 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41af3b0 con 0x56463fb275d0 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.914+0000 7f87bc17f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41add00 con 0x56463fb32370 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb28350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49458/0 (socket says 172.21.15.19:49458) 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 -- 172.21.15.19:0/2847554968 learned_addr learned my addr 172.21.15.19:0/2847554968 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b3fff640 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 -- 172.21.15.19:0/2847554968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 msgr2=0x56463fb2a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 -- 172.21.15.19:0/2847554968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 msgr2=0x56463fb28350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb28350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 -- 172.21.15.19:0/2847554968 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463fb2d570 con 0x56463fb28b40 2024-09-12T03:31:31.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f87a8003b60 tx=0x7f87a802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5d9fc7cfd74e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87997fa640 1 -- 172.21.15.19:0/2847554968 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8035070 con 0x56463fb28b40 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87997fa640 1 -- 172.21.15.19:0/2847554968 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a80037f0 con 0x56463fb28b40 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.915+0000 7f87997fa640 1 -- 172.21.15.19:0/2847554968 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8004920 con 0x56463fb28b40 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- 172.21.15.19:0/2847554968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 msgr2=0x56463fb22bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f87a8003b60 tx=0x7f87a802a410 comp rx=0 tx=0).stop 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- 172.21.15.19:0/2847554968 shutdown_connections 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb32370 0x56463fb2a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb28b40 0x56463fb22bf0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 --2- 172.21.15.19:0/2847554968 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb275d0 0x56463fb28350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- 172.21.15.19:0/2847554968 >> 172.21.15.19:0/2847554968 conn(0x56463faf0310 msgr2=0x56463fb224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- 172.21.15.19:0/2847554968 shutdown_connections 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- 172.21.15.19:0/2847554968 wait complete. 2024-09-12T03:31:31.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.916+0000 7f87bc17f740 1 -- start start 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 Processor -- start 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb275d0 0x56463fb3aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb2c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41aec20 con 0x56463fb32370 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ae490 con 0x56463fb28b40 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f87b41ab550 con 0x56463fb275d0 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87bc17f740 10 client.0 _ll_register_callbacks cb 0x7f87a8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39690/0 (socket says 172.21.15.19:39690) 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb2c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.928 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b2ffd640 1 -- 172.21.15.19:0/4287330133 learned_addr learned my addr 172.21.15.19:0/4287330133 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b37fe640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb275d0 0x56463fb3aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b2ffd640 1 -- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb275d0 msgr2=0x56463fb3aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.917+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb275d0 0x56463fb3aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87b2ffd640 1 -- 172.21.15.19:0/4287330133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 msgr2=0x56463fb2c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb2c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87b2ffd640 1 -- 172.21.15.19:0/4287330133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463fb2c860 con 0x56463fb28b40 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87b2ffd640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87a80315f0 tx=0x7f87a8003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03c164b90c9c5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f8792ffd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8035040 con 0x56463fb28b40 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a80037f0 con 0x56463fb28b40 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87a80055a0 con 0x56463fb28b40 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f87a8003be0 con 0x56463fb28b40 2024-09-12T03:31:31.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87bc17f740 10 client.27224 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56463fbacb70 con 0x56463fb28b40 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.918+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87a80037f0 con 0x56463fb28b40 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f87a8059070 con 0x56463fb28b40 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 10 client.27224 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 10 client.27224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 20 client.27224 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fbad340 0x56463fbaf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 4 client.27224 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56463fa94d70 con 0x56463fbad340 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87b37fe640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fbad340 0x56463fbaf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f87b37fe640 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fbad340 0x56463fbaf760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28fa0e384d8f7184 server_cookie=c35111f1f1a11d24 in_seq=0 out_seq=0 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.919+0000 7f8792ffd640 10 client.27224 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:31:31.930 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f8792ffd640 1 -- 172.21.15.19:0/4287330133 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56463fa94d70 con 0x56463fbad340 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f8792ffd640 10 client.27224 handle_command_reply: tid=0 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 client.27224 shutdown 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 msgr2=0x56463fb2bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87a80315f0 tx=0x7f87a8003f10 comp rx=0 tx=0).stop 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 shutdown_connections 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x56463fbad340 0x56463fbaf760 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x56463fb32370 0x56463fb2c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x56463fb28b40 0x56463fb2bc30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 --2- 172.21.15.19:0/4287330133 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x56463fb275d0 0x56463fb3aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 >> 172.21.15.19:0/4287330133 conn(0x56463faf0310 msgr2=0x56463fb31040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f8792ffd640 0 client.27224 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 shutdown_connections 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 1 -- 172.21.15.19:0/4287330133 wait complete. 2024-09-12T03:31:31.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.920+0000 7f87bc17f740 20 client.27224 trim_cache size 0 max 16384 2024-09-12T03:31:31.932 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:31:31.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 msgr2=0x7f879c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 0x7f879c0a5040 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f87ac005fd0 tx=0x7f87ac004a40 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 msgr2=0x7f87b4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f87a400d950 tx=0x7f87a400de10 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 shutdown_connections 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f87b4106f10 0x7f87b419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f87b4102790 0x7f87b4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f879c0a2b80 0x7f879c0a5040 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 --2- 172.21.15.19:0/300275601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f87b4101de0 0x7f87b4198430 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 >> 172.21.15.19:0/300275601 conn(0x7f87b40fd8c0 msgr2=0x7f87b40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 shutdown_connections 2024-09-12T03:31:31.933 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:31.921+0000 7f87b9da6640 1 -- 172.21.15.19:0/300275601 wait complete. 2024-09-12T03:31:32.662 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:31:32.981 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:33.288 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:33 smithi019 ceph-mon[30953]: pgmap v712: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 421 op/s 2024-09-12T03:31:33.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:33 smithi135 ceph-mon[35474]: pgmap v712: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 421 op/s 2024-09-12T03:31:33.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:33 smithi121 ceph-mon[34963]: pgmap v712: 97 pgs: 97 active+clean; 1.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 421 op/s 2024-09-12T03:31:33.547 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- 172.21.15.19:0/2348668150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 msgr2=0x7f9c9810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 --2- 172.21.15.19:0/2348668150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9810da40 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f9c90009ca0 tx=0x7f9c9002a950 comp rx=0 tx=0).stop 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- 172.21.15.19:0/2348668150 shutdown_connections 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 --2- 172.21.15.19:0/2348668150 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9810da40 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 --2- 172.21.15.19:0/2348668150 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98102570 0x7f9c981029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 --2- 172.21.15.19:0/2348668150 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98100e20 0x7f9c98101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- 172.21.15.19:0/2348668150 >> 172.21.15.19:0/2348668150 conn(0x7f9c980fc7d0 msgr2=0x7f9c980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- 172.21.15.19:0/2348668150 shutdown_connections 2024-09-12T03:31:33.548 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- 172.21.15.19:0/2348668150 wait complete. 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 Processor -- start 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.528+0000 7f9ca039d640 1 -- start start 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 0x7f9c98197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 0x7f9c98197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110f80 con 0x7f9c98102ef0 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98110e00 con 0x7f9c98100e20 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9ca039d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c98111100 con 0x7f9c98102570 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.549 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 0x7f9c98197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49494/0 (socket says 172.21.15.19:49494) 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 -- 172.21.15.19:0/1500558470 learned_addr learned my addr 172.21.15.19:0/1500558470 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9d938640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 0x7f9c98197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 -- 172.21.15.19:0/1500558470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 msgr2=0x7f9c98197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 0x7f9c98197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 -- 172.21.15.19:0/1500558470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 msgr2=0x7f9c98197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 0x7f9c98197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e93a640 1 -- 172.21.15.19:0/1500558470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9819c2f0 con 0x7f9c98102ef0 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9e139640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 0x7f9c98197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:33.550 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.529+0000 7f9c9d938640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 0x7f9c98197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9c9e93a640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9c90004af0 tx=0x7f9c9002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9c8f7fe640 1 -- 172.21.15.19:0/1500558470 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90034040 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9c8f7fe640 1 -- 172.21.15.19:0/1500558470 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c90030920 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9c8f7fe640 1 -- 172.21.15.19:0/1500558470 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c90004820 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9819c580 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.530+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c9819ca10 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.532+0000 7f9c8f7fe640 1 -- 172.21.15.19:0/1500558470 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9c90037020 con 0x7f9c98102ef0 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.532+0000 7f9ca2776740 1 Processor -- start 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 -- start start 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.551 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c42970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e860b992f0 con 0x55e860c335d0 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e860b40cf0 con 0x55e860c40580 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9ca2776740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e860b952b0 con 0x55e860c34b40 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:51710/0 (socket says 172.21.15.19:51710) 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 -- 172.21.15.19:0/766152271 learned_addr learned my addr 172.21.15.19:0/766152271 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c42970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c8f7fe640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 0x7f9c840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 -- 172.21.15.19:0/766152271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 msgr2=0x55e860c42970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.552 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c42970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c8f7fe640 1 -- 172.21.15.19:0/1500558470 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c900eff80 con 0x7f9c98102ef0 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 -- 172.21.15.19:0/766152271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 msgr2=0x55e860c33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.533+0000 7f9c9d938640 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c9d938640 1 -- 172.21.15.19:0/766152271 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e860b253a0 con 0x55e860c34b40 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c9e139640 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c9d938640 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009560 tx=0x7f9c880096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5be80b13cebfd4a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c9e93a640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 0x7f9c840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c8cdea640 1 -- 172.21.15.19:0/766152271 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88012d00 con 0x55e860c34b40 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c8cdea640 1 -- 172.21.15.19:0/766152271 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88010cb0 con 0x55e860c34b40 2024-09-12T03:31:33.553 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.534+0000 7f9c8cdea640 1 -- 172.21.15.19:0/766152271 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8801eb50 con 0x55e860c34b40 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 -- 172.21.15.19:0/766152271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 msgr2=0x55e860c33090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9c9e93a640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 0x7f9c840a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9c9002ae70 tx=0x7f9c900042c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009560 tx=0x7f9c880096e0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 -- 172.21.15.19:0/766152271 shutdown_connections 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c42970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 --2- 172.21.15.19:0/766152271 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c34b40 0x55e860c33090 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 -- 172.21.15.19:0/766152271 >> 172.21.15.19:0/766152271 conn(0x55e860bfc310 msgr2=0x55e860c2fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 -- 172.21.15.19:0/766152271 shutdown_connections 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.535+0000 7f9ca2776740 1 -- 172.21.15.19:0/766152271 wait complete. 2024-09-12T03:31:33.554 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 -- start start 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 Processor -- start 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c335d0 0x55e860c3ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c3da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55e860c43280 con 0x55e860c34b40 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e860b54060 con 0x55e860c40580 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55e860c43400 con 0x55e860c335d0 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9ca2776740 10 client.0 _ll_register_callbacks cb 0x55e8608a78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c3da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49512/0 (socket says 172.21.15.19:49512) 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c3da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39728/0 (socket says 172.21.15.19:39728) 2024-09-12T03:31:33.555 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 -- 172.21.15.19:0/2818777268 learned_addr learned my addr 172.21.15.19:0/2818777268 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c335d0 0x55e860c3ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 -- 172.21.15.19:0/2818777268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c335d0 msgr2=0x55e860c3ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c335d0 0x55e860c3ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 -- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 msgr2=0x55e860c3da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c3da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.536+0000 7f9c9e139640 1 -- 172.21.15.19:0/2818777268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e86090a3c0 con 0x55e860c34b40 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9c940118c0 tx=0x7f9c94011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f7e5a62a2f1acd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9401f060 con 0x55e860c34b40 2024-09-12T03:31:33.556 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c94010b40 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c88010cb0 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9c94010d40 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9ca2776740 10 client.17904 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e860c4edf0 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88010cb0 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f9c9400f4c0 con 0x55e860c34b40 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9ca2776740 10 client.17904 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:33.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.537+0000 7f9ca2776740 10 client.17904 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:31:33.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9ca2776740 20 client.17904 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:33.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860c50380 0x55e860c527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9ca2776740 4 client.17904 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:33.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e860ba6860 con 0x55e860c50380 2024-09-12T03:31:33.558 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860c50380 0x55e860c527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860c50380 0x55e860c527a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25b12196896462f7 server_cookie=1c43fd087622dc9a in_seq=0 out_seq=0 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.538+0000 7f9c81ffb640 10 client.17904 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2818777268 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e860ba6860 con 0x55e860c50380 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9c81ffb640 10 client.17904 handle_command_reply: tid=0 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 client.17904 shutdown 2024-09-12T03:31:33.559 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 msgr2=0x55e860c3d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f9c940118c0 tx=0x7f9c94011d80 comp rx=0 tx=0).stop 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 shutdown_connections 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860c50380 0x55e860c527a0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c40580 0x55e860c3da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860c3d280 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2818777268 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c335d0 0x55e860c3ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 >> 172.21.15.19:0/2818777268 conn(0x55e860bfc310 msgr2=0x55e860c35960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9c81ffb640 0 client.17904 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:31:33.560 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 shutdown_connections 2024-09-12T03:31:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 1 -- 172.21.15.19:0/2818777268 wait complete. 2024-09-12T03:31:33.561 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.540+0000 7f9ca2776740 20 client.17904 trim_cache size 0 max 16384 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 Processor -- start 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 -- start start 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 0x55e860c58070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981ab2f0 con 0x55e860c335d0 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981ad070 con 0x55e860c34b40 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.553+0000 7f9ca2776740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981ab9c0 con 0x55e860c40580 2024-09-12T03:31:33.567 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49526/0 (socket says 172.21.15.19:49526) 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 -- 172.21.15.19:0/439592078 learned_addr learned my addr 172.21.15.19:0/439592078 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9e139640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 0x55e860c58070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 -- 172.21.15.19:0/439592078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 msgr2=0x55e860c585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 -- 172.21.15.19:0/439592078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 msgr2=0x55e860c58070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 0x55e860c58070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 -- 172.21.15.19:0/439592078 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e860c30960 con 0x55e860c335d0 2024-09-12T03:31:33.568 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9e139640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 0x55e860c58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c9d938640 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9c880096b0 tx=0x7f9c8801b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3efb9e2cf6dfc71 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c83fff640 1 -- 172.21.15.19:0/439592078 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017040 con 0x55e860c335d0 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c83fff640 1 -- 172.21.15.19:0/439592078 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c88012dc0 con 0x55e860c335d0 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.554+0000 7f9c83fff640 1 -- 172.21.15.19:0/439592078 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88010a00 con 0x55e860c335d0 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 -- 172.21.15.19:0/439592078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 msgr2=0x55e860c3d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f9c880096b0 tx=0x7f9c8801b040 comp rx=0 tx=0).stop 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 -- 172.21.15.19:0/439592078 shutdown_connections 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c34b40 0x55e860c58070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 --2- 172.21.15.19:0/439592078 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c335d0 0x55e860c3d250 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.569 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 -- 172.21.15.19:0/439592078 >> 172.21.15.19:0/439592078 conn(0x55e860bfc310 msgr2=0x55e860c3cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 -- 172.21.15.19:0/439592078 shutdown_connections 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.555+0000 7f9ca2776740 1 -- 172.21.15.19:0/439592078 wait complete. 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 -- start start 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 Processor -- start 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c335d0 0x55e860cb31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c48320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981ac8e0 con 0x55e860c34b40 2024-09-12T03:31:33.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981ac150 con 0x55e860c335d0 2024-09-12T03:31:33.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9c981a9230 con 0x55e860c40580 2024-09-12T03:31:33.572 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9ca2776740 10 client.0 _ll_register_callbacks cb 0x7f9c9400bcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49540/0 (socket says 172.21.15.19:49540) 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 -- 172.21.15.19:0/2095877162 learned_addr learned my addr 172.21.15.19:0/2095877162 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c335d0 0x55e860cb31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c48320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 -- 172.21.15.19:0/2095877162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 msgr2=0x55e860c48320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.573 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c48320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 -- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c335d0 msgr2=0x55e860cb31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c335d0 0x55e860cb31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.556+0000 7f9c9e139640 1 -- 172.21.15.19:0/2095877162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e860c48990 con 0x55e860c34b40 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c9e139640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9c94011890 tx=0x7f9c9400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf921b80847b42a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9402a040 con 0x55e860c34b40 2024-09-12T03:31:33.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9c94010b40 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e860ba0d70 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9c94011440 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9ca2776740 10 client.17910 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e860cbaa10 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e860ba0d70 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.557+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f9c94004660 con 0x55e860c34b40 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 10 client.17910 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:33.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 10 client.17910 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 20 client.17910 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860cbb1f0 0x55e860cbd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 4 client.17910 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e860b1d170 con 0x55e860cbb1f0 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860cbb1f0 0x55e860cbd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9c81ffb640 10 client.17910 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:31:33.576 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.558+0000 7f9c9d938640 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860cbb1f0 0x55e860cbd610 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=814131643920c308 server_cookie=ba6e15c138f71adb in_seq=0 out_seq=0 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.559+0000 7f9c81ffb640 1 -- 172.21.15.19:0/2095877162 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e860b1d170 con 0x55e860cbb1f0 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.559+0000 7f9c81ffb640 10 client.17910 handle_command_reply: tid=0 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 client.17910 shutdown 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 msgr2=0x55e860cb3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f9c94011890 tx=0x7f9c9400fd50 comp rx=0 tx=0).stop 2024-09-12T03:31:33.577 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 shutdown_connections 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x55e860cbb1f0 0x55e860cbd610 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55e860c40580 0x55e860c48320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55e860c34b40 0x55e860cb3720 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 --2- 172.21.15.19:0/2095877162 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e860c335d0 0x55e860cb31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 >> 172.21.15.19:0/2095877162 conn(0x55e860bfc310 msgr2=0x55e860c3ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9c81ffb640 0 client.17910 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 shutdown_connections 2024-09-12T03:31:33.578 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 1 -- 172.21.15.19:0/2095877162 wait complete. 2024-09-12T03:31:33.579 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca2776740 20 client.17910 trim_cache size 0 max 16384 2024-09-12T03:31:33.579 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 msgr2=0x7f9c840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 0x7f9c840a5040 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9c9002ae70 tx=0x7f9c900042c0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 msgr2=0x7f9c9819bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:33.580 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.560+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f9c90004af0 tx=0x7f9c9002f5b0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 shutdown_connections 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9c840a2b80 0x7f9c840a5040 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9c98102ef0 0x7f9c9819bc10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9c98102570 0x7f9c98197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 --2- 172.21.15.19:0/1500558470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c98100e20 0x7f9c98197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 >> 172.21.15.19:0/1500558470 conn(0x7f9c980fc7d0 msgr2=0x7f9c98106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 shutdown_connections 2024-09-12T03:31:33.581 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:33.561+0000 7f9ca039d640 1 -- 172.21.15.19:0/1500558470 wait complete. 2024-09-12T03:31:34.348 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:31:34.605 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 -- 172.21.15.19:0/355837459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac103660 msgr2=0x7f40ac10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 --2- 172.21.15.19:0/355837459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac103660 0x7f40ac10bb50 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a002a480 comp rx=0 tx=0).stop 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 -- 172.21.15.19:0/355837459 shutdown_connections 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 --2- 172.21.15.19:0/355837459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac10c170 0x7f40ac10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 --2- 172.21.15.19:0/355837459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac103660 0x7f40ac10bb50 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 --2- 172.21.15.19:0/355837459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac102d40 0x7f40ac103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 -- 172.21.15.19:0/355837459 >> 172.21.15.19:0/355837459 conn(0x7f40ac0fc7a0 msgr2=0x7f40ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 -- 172.21.15.19:0/355837459 shutdown_connections 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.118+0000 7f40b20c9640 1 -- 172.21.15.19:0/355837459 wait complete. 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 Processor -- start 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 -- start start 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac103660 0x7f40ac100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac10c170 0x7f40ac101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac111f70 con 0x7f40ac102d40 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac111df0 con 0x7f40ac10c170 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40b20c9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1120f0 con 0x7f40ac103660 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.119+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40abfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac10c170 0x7f40ac101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49550/0 (socket says 172.21.15.19:49550) 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 -- 172.21.15.19:0/169072077 learned_addr learned my addr 172.21.15.19:0/169072077 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40aaffd640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac103660 0x7f40ac100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 -- 172.21.15.19:0/169072077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac103660 msgr2=0x7f40ac100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac103660 0x7f40ac100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 -- 172.21.15.19:0/169072077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac10c170 msgr2=0x7f40ac101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac10c170 0x7f40ac101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 -- 172.21.15.19:0/169072077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ac1a6930 con 0x7f40ac102d40 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e9f0 tx=0x7f409c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40a8ff9640 1 -- 172.21.15.19:0/169072077 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c00cd80 con 0x7f40ac102d40 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ac1a6c20 con 0x7f40ac102d40 2024-09-12T03:31:35.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40a8ff9640 1 -- 172.21.15.19:0/169072077 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f409c010910 con 0x7f40ac102d40 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40a8ff9640 1 -- 172.21.15.19:0/169072077 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f409c0188f0 con 0x7f40ac102d40 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.120+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ac1a7180 con 0x7f40ac102d40 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.123+0000 7f40a8ff9640 1 -- 172.21.15.19:0/169072077 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f409c010ac0 con 0x7f40ac102d40 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.123+0000 7f40b44a2740 1 Processor -- start 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.123+0000 7f40b44a2740 1 -- start start 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee1e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee1ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55db5ed842f0 con 0x55db5ee29370 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55db5ed2bcf0 con 0x55db5ee1e5d0 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40b44a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db5ed802b0 con 0x55db5ee1fb40 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee1ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40abfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee1e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49566/0 (socket says 172.21.15.19:49566) 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 -- 172.21.15.19:0/1159317902 learned_addr learned my addr 172.21.15.19:0/1159317902 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40a8ff9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 -- 172.21.15.19:0/1159317902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 msgr2=0x55db5ee1e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee1e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 -- 172.21.15.19:0/1159317902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 msgr2=0x55db5ee1ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee1ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 -- 172.21.15.19:0/1159317902 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5ed103a0 con 0x55db5ee29370 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40abfff640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40a8ff9640 1 -- 172.21.15.19:0/169072077 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f409c0cf3c0 con 0x7f40ac102d40 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4214660f8cc702 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.124+0000 7f4091ffb640 1 -- 172.21.15.19:0/1159317902 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0040070 con 0x55db5ee29370 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f4091ffb640 1 -- 172.21.15.19:0/1159317902 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0031870 con 0x55db5ee29370 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 -- 172.21.15.19:0/1159317902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 msgr2=0x55db5ee2b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40abfff640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 0x7f40940a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005fd0 tx=0x7f40a4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009900 tx=0x7f40a0005be0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f4091ffb640 1 -- 172.21.15.19:0/1159317902 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0035040 con 0x55db5ee29370 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 -- 172.21.15.19:0/1159317902 shutdown_connections 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee29370 0x55db5ee2b760 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee1ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1159317902 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee1e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.125+0000 7f40b44a2740 1 -- 172.21.15.19:0/1159317902 >> 172.21.15.19:0/1159317902 conn(0x55db5ede7310 msgr2=0x55db5ee1ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.126+0000 7f40b44a2740 1 -- 172.21.15.19:0/1159317902 shutdown_connections 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.126+0000 7f40b44a2740 1 -- 172.21.15.19:0/1159317902 wait complete. 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.126+0000 7f40b44a2740 1 -- start start 2024-09-12T03:31:35.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.126+0000 7f40b44a2740 1 Processor -- start 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee28c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee25f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee26450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55db5ee2c060 con 0x55db5ee1e5d0 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55db5ed3f060 con 0x55db5ee29370 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55db5ee2c1e0 con 0x55db5ee1fb40 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40b44a2740 10 client.0 _ll_register_callbacks cb 0x55db5ea928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40abfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee26450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40aaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee25f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40abfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee26450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39780/0 (socket says 172.21.15.19:39780) 2024-09-12T03:31:35.142 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee28c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.127+0000 7f40abfff640 1 -- 172.21.15.19:0/2767473883 learned_addr learned my addr 172.21.15.19:0/2767473883 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 -- 172.21.15.19:0/2767473883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 msgr2=0x55db5ee26450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee26450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 -- 172.21.15.19:0/2767473883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 msgr2=0x55db5ee28c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee28c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 -- 172.21.15.19:0/2767473883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5eaf53c0 con 0x55db5ee1fb40 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee28c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f40aaffd640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee25f10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006010 tx=0x7f40a00054c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62898f74602d80d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.128+0000 7f408affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:31:35.143 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a004c020 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40a0004260 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a0031870 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f40a0040070 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f40b44a2740 10 client.27220 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55db5ee2c060 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.129+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a0031870 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f40a00035f0 con 0x55db5ee1fb40 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 10 client.27220 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 10 client.27220 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 20 client.27220 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5ee38c40 0x55db5ee3b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.144 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 4 client.27220 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db5ed91860 con 0x55db5ee38c40 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.130+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5ee38c40 0x55db5ee3b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.131+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5ee38c40 0x55db5ee3b030 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49ad670c87dc8149 server_cookie=dd9c3840fff4f998 in_seq=0 out_seq=0 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.131+0000 7f408affd640 10 client.27220 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f408affd640 1 -- 172.21.15.19:0/2767473883 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db5ed91860 con 0x55db5ee38c40 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f408affd640 10 client.27220 handle_command_reply: tid=0 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 client.27220 shutdown 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 msgr2=0x55db5ee25f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.145 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee25f10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006010 tx=0x7f40a00054c0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 shutdown_connections 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5ee38c40 0x55db5ee3b030 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee26450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee25f10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 --2- 172.21.15.19:0/2767473883 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee28c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 >> 172.21.15.19:0/2767473883 conn(0x55db5ede7310 msgr2=0x55db5ee20960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f408affd640 0 client.27220 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.132+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 shutdown_connections 2024-09-12T03:31:35.146 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.133+0000 7f40b44a2740 1 -- 172.21.15.19:0/2767473883 wait complete. 2024-09-12T03:31:35.147 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.133+0000 7f40b44a2740 20 client.27220 trim_cache size 0 max 16384 2024-09-12T03:31:35.152 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 Processor -- start 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 -- start start 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 0x55db5ee40860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 0x55db5ee40da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1af880 con 0x55db5ee1fb40 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1b1600 con 0x55db5ee1e5d0 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.152+0000 7f40b44a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1aff50 con 0x55db5ee29370 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40ab7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee353b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39784/0 (socket says 172.21.15.19:39784) 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 0x55db5ee40da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.153 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40ab7fe640 1 -- 172.21.15.19:0/3118318447 learned_addr learned my addr 172.21.15.19:0/3118318447 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:35.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 -- 172.21.15.19:0/3118318447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 msgr2=0x55db5ee353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.154 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40aaffd640 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 0x55db5ee40860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 -- 172.21.15.19:0/3118318447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 msgr2=0x55db5ee40860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 0x55db5ee40860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 -- 172.21.15.19:0/3118318447 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5ee28fd0 con 0x55db5ee29370 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40aaffd640 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 0x55db5ee40860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.153+0000 7f40abfff640 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 0x55db5ee40da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55db5ee16620 tx=0x7f40a4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4294f4d5785992d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.154+0000 7f40917fa640 1 -- 172.21.15.19:0/3118318447 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4020600 con 0x55db5ee29370 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.154+0000 7f40917fa640 1 -- 172.21.15.19:0/3118318447 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a4020c80 con 0x55db5ee29370 2024-09-12T03:31:35.155 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.154+0000 7f40917fa640 1 -- 172.21.15.19:0/3118318447 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4010cf0 con 0x55db5ee29370 2024-09-12T03:31:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 -- 172.21.15.19:0/3118318447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 msgr2=0x55db5ee40da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 0x55db5ee40da0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x55db5ee16620 tx=0x7f40a4016000 comp rx=0 tx=0).stop 2024-09-12T03:31:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 -- 172.21.15.19:0/3118318447 shutdown_connections 2024-09-12T03:31:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee29370 0x55db5ee40da0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.156 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1fb40 0x55db5ee40860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 --2- 172.21.15.19:0/3118318447 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee1e5d0 0x55db5ee353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 -- 172.21.15.19:0/3118318447 >> 172.21.15.19:0/3118318447 conn(0x55db5ede7310 msgr2=0x55db5ee25000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 -- 172.21.15.19:0/3118318447 shutdown_connections 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.155+0000 7f40b44a2740 1 -- 172.21.15.19:0/3118318447 wait complete. 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.156+0000 7f40b44a2740 1 -- start start 2024-09-12T03:31:35.157 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.156+0000 7f40b44a2740 1 Processor -- start 2024-09-12T03:31:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.156+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee318f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.158 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.156+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.156+0000 7f40b44a2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee2ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40b44a2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1b0e70 con 0x55db5ee1e5d0 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40b44a2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1b06e0 con 0x55db5ee29370 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40b44a2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f40ac1ad7a0 con 0x55db5ee1fb40 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40b44a2740 10 client.0 _ll_register_callbacks cb 0x7f409c0d2d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:51782/0 (socket says 172.21.15.19:51782) 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 -- 172.21.15.19:0/1160697959 learned_addr learned my addr 172.21.15.19:0/1160697959 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 -- 172.21.15.19:0/1160697959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 msgr2=0x55db5ee2ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee2ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 -- 172.21.15.19:0/1160697959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 msgr2=0x55db5ee318f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:35.159 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee318f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 -- 172.21.15.19:0/1160697959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5ee2d680 con 0x55db5ee1fb40 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f40aaffd640 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40a0003a40 tx=0x7f40a0009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fd5bd4753a92c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.157+0000 7f408affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.158+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a004c070 con 0x55db5ee1fb40 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.158+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a4020c80 con 0x55db5ee1fb40 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.158+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f40a0004460 con 0x55db5ee1fb40 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.158+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f40a0035040 con 0x55db5ee1fb40 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.158+0000 7f40b44a2740 10 client.27226 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:35.160 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.159+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55db5eeabc40 con 0x55db5ee1fb40 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40a4020c80 con 0x55db5ee1fb40 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f40a0064470 con 0x55db5ee1fb40 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 10 client.27226 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 10 client.27226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 20 client.27226 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5eeac3c0 0x55db5eeae800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 4 client.27226 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.160+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55db5ee267e0 con 0x55db5eeac3c0 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.161+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5eeac3c0 0x55db5eeae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.161+0000 7f40ab7fe640 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5eeac3c0 0x55db5eeae800 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266bd7d14a9d001f server_cookie=30b7e624afede952 in_seq=0 out_seq=0 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.161+0000 7f408affd640 10 client.27226 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:35.162 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.162+0000 7f408affd640 1 -- 172.21.15.19:0/1160697959 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55db5ee267e0 con 0x55db5eeac3c0 2024-09-12T03:31:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.162+0000 7f408affd640 10 client.27226 handle_command_reply: tid=0 2024-09-12T03:31:35.163 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.162+0000 7f40b44a2740 1 client.27226 shutdown 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.162+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 msgr2=0x55db5ee31e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.162+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40a0003a40 tx=0x7f40a0009520 comp rx=0 tx=0).stop 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 shutdown_connections 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55db5eeac3c0 0x55db5eeae800 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55db5ee29370 0x55db5ee2ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55db5ee1fb40 0x55db5ee31e30 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f40a0003a40 tx=0x7f40a0009520 comp rx=0 tx=0).stop 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 --2- 172.21.15.19:0/1160697959 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55db5ee1e5d0 0x55db5ee318f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.164 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 >> 172.21.15.19:0/1160697959 conn(0x55db5ede7310 msgr2=0x55db5ee27770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f408affd640 0 client.27226 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 shutdown_connections 2024-09-12T03:31:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 1 -- 172.21.15.19:0/1160697959 wait complete. 2024-09-12T03:31:35.165 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.163+0000 7f40b44a2740 20 client.27226 trim_cache size 0 max 16384 2024-09-12T03:31:35.165 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 msgr2=0x7f40940a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 0x7f40940a5040 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005fd0 tx=0x7f40a4004a40 comp rx=0 tx=0).stop 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 msgr2=0x7f40ac1009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f409c00e9f0 tx=0x7f409c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 shutdown_connections 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40ac10c170 0x7f40ac101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.166 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f40ac103660 0x7f40ac100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f40940a2b80 0x7f40940a5040 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 --2- 172.21.15.19:0/169072077 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f40ac102d40 0x7f40ac1009a0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 >> 172.21.15.19:0/169072077 conn(0x7f40ac0fc7a0 msgr2=0x7f40ac0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 shutdown_connections 2024-09-12T03:31:35.167 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:35.165+0000 7f40b20c9640 1 -- 172.21.15.19:0/169072077 wait complete. 2024-09-12T03:31:35.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:35 smithi135 ceph-mon[35474]: pgmap v713: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 418 op/s 2024-09-12T03:31:35.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:35 smithi121 ceph-mon[34963]: pgmap v713: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 418 op/s 2024-09-12T03:31:35.417 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:35 smithi019 ceph-mon[30953]: pgmap v713: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 418 op/s 2024-09-12T03:31:36.980 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:31:36.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:31:37.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:37 smithi135 ceph-mon[35474]: pgmap v714: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 413 op/s 2024-09-12T03:31:37.310 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:37.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:37 smithi121 ceph-mon[34963]: pgmap v714: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 413 op/s 2024-09-12T03:31:37.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:37 smithi019 ceph-mon[30953]: pgmap v714: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 413 op/s 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.799+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/809869580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec103780 msgr2=0x7f6eec107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.799+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/809869580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec103780 0x7f6eec107bc0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009930 tx=0x7f6ee002a420 comp rx=0 tx=0).stop 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/809869580 shutdown_connections 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/809869580 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec108300 0x7f6eec10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/809869580 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec103780 0x7f6eec107bc0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/809869580 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/809869580 >> 172.21.15.19:0/809869580 conn(0x7f6eec0fc7d0 msgr2=0x7f6eec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.845 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/809869580 shutdown_connections 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/809869580 wait complete. 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.800+0000 7f6ef1cdd640 1 Processor -- start 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 -- start start 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec103780 0x7f6eec198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec112070 con 0x7f6eec108300 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec111ef0 con 0x7f6eec103780 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6ef1cdd640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1121f0 con 0x7f6eec102dd0 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eebfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49622/0 (socket says 172.21.15.19:49622) 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.846 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eebfff640 1 -- 172.21.15.19:0/2429680911 learned_addr learned my addr 172.21.15.19:0/2429680911 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec103780 0x7f6eec198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2429680911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec103780 msgr2=0x7f6eec198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec103780 0x7f6eec198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 msgr2=0x7f6eec19cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2429680911 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec19d580 con 0x7f6eec102dd0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.801+0000 7f6eebfff640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1985d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6edc005fd0 tx=0x7f6edc00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6ee8ff9640 1 -- 172.21.15.19:0/2429680911 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00cd80 con 0x7f6eec102dd0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6ee8ff9640 1 -- 172.21.15.19:0/2429680911 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc005020 con 0x7f6eec102dd0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec19d870 con 0x7f6eec102dd0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6ee8ff9640 1 -- 172.21.15.19:0/2429680911 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0116d0 con 0x7f6eec102dd0 2024-09-12T03:31:37.847 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.802+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec1a5150 con 0x7f6eec102dd0 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.804+0000 7f6ef40b6740 1 Processor -- start 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.804+0000 7f6ee8ff9640 1 -- 172.21.15.19:0/2429680911 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6edc005540 con 0x7f6eec102dd0 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.804+0000 7f6ef40b6740 1 -- start start 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adaa60 0x5631d2adae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2ae9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631d2a40280 con 0x5631d2aea150 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5631d29e7b10 con 0x5631d2adb380 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5631d2a3c240 con 0x5631d2adaa60 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2ae9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.848 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49630/0 (socket says 172.21.15.19:49630) 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3498396378 learned_addr learned my addr 172.21.15.19:0/3498396378 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eebfff640 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adaa60 0x5631d2adae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3498396378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adaa60 msgr2=0x5631d2adae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adaa60 0x5631d2adae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3498396378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 msgr2=0x5631d2ae9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2ae9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3498396378 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d29f41f0 con 0x5631d2aea150 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0000c00 tx=0x7f6ee0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54535200830d52bd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.805+0000 7f6ee8ff9640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 0x7f6ed40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ed1ffb640 1 -- 172.21.15.19:0/3498396378 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0040070 con 0x5631d2aea150 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ed1ffb640 1 -- 172.21.15.19:0/3498396378 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0004030 con 0x5631d2aea150 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ed1ffb640 1 -- 172.21.15.19:0/3498396378 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0035040 con 0x5631d2aea150 2024-09-12T03:31:37.849 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6eebfff640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 0x7f6ed40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ee8ff9640 1 -- 172.21.15.19:0/2429680911 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6edc0cf450 con 0x7f6eec102dd0 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3498396378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 msgr2=0x5631d2aec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6eebfff640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 0x7f6ed40a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4012640 tx=0x7f6ee4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0000c00 tx=0x7f6ee0005e80 comp rx=0 tx=0).stop 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3498396378 shutdown_connections 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2aea150 0x5631d2aec540 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2ae9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3498396378 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adaa60 0x5631d2adae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3498396378 >> 172.21.15.19:0/3498396378 conn(0x5631d2ad70f0 msgr2=0x5631d2ad74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3498396378 shutdown_connections 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.806+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3498396378 wait complete. 2024-09-12T03:31:37.850 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 -- start start 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 Processor -- start 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adaa60 0x5631d2ae7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2ae7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5631d2aecd20 con 0x5631d2adb380 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5631d29faf10 con 0x5631d2adaa60 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.807+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5631d2aecea0 con 0x5631d2aea150 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ef40b6740 10 client.0 _ll_register_callbacks cb 0x5631d29c3a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49634/0 (socket says 172.21.15.19:49634) 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3625575898 learned_addr learned my addr 172.21.15.19:0/3625575898 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adaa60 0x5631d2ae7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.851 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eebfff640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2ae7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 msgr2=0x5631d2ae7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2ae7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3625575898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adaa60 msgr2=0x5631d2ae7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adaa60 0x5631d2ae7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 -- 172.21.15.19:0/3625575898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d27b16a0 con 0x5631d2adb380 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005310 tx=0x7f6ee00053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d973f0e72fcd88f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0040070 con 0x5631d2adb380 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6ee00315b0 con 0x5631d2adb380 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee0004030 con 0x5631d2adb380 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.808+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ee0035040 con 0x5631d2adb380 2024-09-12T03:31:37.852 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 10 client.17931 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5631d2af8990 con 0x5631d2adb380 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6ee0004030 con 0x5631d2adb380 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6ee004c020 con 0x5631d2adb380 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 10 client.17931 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 20 client.17931 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2af9ce0 0x5631d2afc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 4 client.17931 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631d2a4d7f0 con 0x5631d2af9ce0 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.809+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2af9ce0 0x5631d2afc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.810+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2af9ce0 0x5631d2afc0d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816a8e7aefb82f33 server_cookie=14219d1a051c0dd1 in_seq=0 out_seq=0 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.810+0000 7f6ecaffd640 10 client.17931 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:37.853 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.810+0000 7f6ecaffd640 1 -- 172.21.15.19:0/3625575898 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631d2a4d7f0 con 0x5631d2af9ce0 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.810+0000 7f6ecaffd640 10 client.17931 handle_command_reply: tid=0 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.810+0000 7f6ef40b6740 1 client.17931 shutdown 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 msgr2=0x5631d2ae7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005310 tx=0x7f6ee00053b0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 shutdown_connections 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2ae7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adb380 0x5631d2ae7570 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2af9ce0 0x5631d2afc0d0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/3625575898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adaa60 0x5631d2ae7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ecaffd640 0 client.17931 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 >> 172.21.15.19:0/3625575898 conn(0x5631d2ad70f0 msgr2=0x5631d2adce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 shutdown_connections 2024-09-12T03:31:37.854 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 1 -- 172.21.15.19:0/3625575898 wait complete. 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.811+0000 7f6ef40b6740 20 client.17931 trim_cache size 0 max 16384 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.828+0000 7f6ef40b6740 1 Processor -- start 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.828+0000 7f6ef40b6740 1 -- start start 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2b016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2b01c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1ab300 con 0x5631d2adaa60 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1ad080 con 0x5631d2adb380 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1ab9d0 con 0x5631d2aea150 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49636/0 (socket says 172.21.15.19:49636) 2024-09-12T03:31:37.855 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/990315891 learned_addr learned my addr 172.21.15.19:0/990315891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2b016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eebfff640 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2b01c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/990315891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 msgr2=0x5631d2b01c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2b01c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/990315891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 msgr2=0x5631d2b016d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2b016d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/990315891 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631d2ad7ca0 con 0x5631d2adaa60 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6edc09ba70 tx=0x7f6edc0cce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e0dbc74f7a07fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.829+0000 7f6ed17fa640 1 -- 172.21.15.19:0/990315891 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01f040 con 0x5631d2adaa60 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ed17fa640 1 -- 172.21.15.19:0/990315891 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc09bd70 con 0x5631d2adaa60 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ed17fa640 1 -- 172.21.15.19:0/990315891 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0983e0 con 0x5631d2adaa60 2024-09-12T03:31:37.856 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 -- 172.21.15.19:0/990315891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 msgr2=0x5631d2adcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f6edc09ba70 tx=0x7f6edc0cce30 comp rx=0 tx=0).stop 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 -- 172.21.15.19:0/990315891 shutdown_connections 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2aea150 0x5631d2b01c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2adb380 0x5631d2b016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/990315891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2adcd10 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 -- 172.21.15.19:0/990315891 >> 172.21.15.19:0/990315891 conn(0x5631d2ad70f0 msgr2=0x5631d2adc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 -- 172.21.15.19:0/990315891 shutdown_connections 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.830+0000 7f6ef40b6740 1 -- 172.21.15.19:0/990315891 wait complete. 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 -- start start 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 Processor -- start 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.857 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 0x5631d2aee7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2aea150 0x5631d2b636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1ac8f0 con 0x5631d2adaa60 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1ac160 con 0x5631d2aea150 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1a9220 con 0x5631d2adb380 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6ef40b6740 10 client.0 _ll_register_callbacks cb 0x7f6edc0d2e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6eeb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49638/0 (socket says 172.21.15.19:49638) 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2348205747 learned_addr learned my addr 172.21.15.19:0/2348205747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6eebfff640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2aea150 0x5631d2b636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.831+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 0x5631d2aee7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 msgr2=0x5631d2aee7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 0x5631d2aee7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2348205747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2aea150 msgr2=0x5631d2b636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2aea150 0x5631d2b636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 -- 172.21.15.19:0/2348205747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631d2b63be0 con 0x5631d2adaa60 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 0x5631d2aee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6eeb7fe640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0a4400 tx=0x7f6edc0ccf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6076581e718ac14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc01f040 con 0x5631d2adaa60 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6edc0983e0 con 0x5631d2adaa60 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6edc09bd70 con 0x5631d2adaa60 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6edc0a0070 con 0x5631d2adaa60 2024-09-12T03:31:37.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 10 client.17937 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5631d2b64dc0 con 0x5631d2adaa60 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6edc09bd70 con 0x5631d2adaa60 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f6edc00f040 con 0x5631d2adaa60 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 10 client.17937 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 10 client.17937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.832+0000 7f6ef40b6740 20 client.17937 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2b65580 0x5631d2b67970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6ef40b6740 4 client.17937 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5631d2a47d00 con 0x5631d2b65580 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2b65580 0x5631d2b67970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:37.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6eeaffd640 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2b65580 0x5631d2b67970 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0e59f8de3092fd0 server_cookie=7333928530e427db in_seq=0 out_seq=0 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.833+0000 7f6ecaffd640 10 client.17937 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ecaffd640 1 -- 172.21.15.19:0/2348205747 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5631d2a47d00 con 0x5631d2b65580 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ecaffd640 10 client.17937 handle_command_reply: tid=0 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 client.17937 shutdown 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 msgr2=0x5631d2aee2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0a4400 tx=0x7f6edc0ccf20 comp rx=0 tx=0).stop 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 shutdown_connections 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5631d2b65580 0x5631d2b67970 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5631d2aea150 0x5631d2b636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ecaffd640 0 client.17937 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5631d2adb380 0x5631d2aee7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 --2- 172.21.15.19:0/2348205747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5631d2adaa60 0x5631d2aee2b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 >> 172.21.15.19:0/2348205747 conn(0x5631d2ad70f0 msgr2=0x5631d2adcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 shutdown_connections 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "6e03c154-48e2-4034-8193-28561de9f0a1", 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:31:37.862 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 1 -- 172.21.15.19:0/2348205747 wait complete. 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.834+0000 7f6ef40b6740 20 client.17937 trim_cache size 0 max 16384 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 msgr2=0x7f6ed40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 0x7f6ed40a50c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4012640 tx=0x7f6ee4005f90 comp rx=0 tx=0).stop 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 msgr2=0x7f6eec1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1985d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6edc005fd0 tx=0x7f6edc00ee80 comp rx=0 tx=0).stop 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 shutdown_connections 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6ed40a2c00 0x7f6ed40a50c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6eec108300 0x7f6eec19cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6eec103780 0x7f6eec198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 --2- 172.21.15.19:0/2429680911 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6eec102dd0 0x7f6eec1985d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 >> 172.21.15.19:0/2429680911 conn(0x7f6eec0fc7d0 msgr2=0x7f6eec105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 shutdown_connections 2024-09-12T03:31:37.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:37.835+0000 7f6ef1cdd640 1 -- 172.21.15.19:0/2429680911 wait complete. 2024-09-12T03:31:38.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:38 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:31:38.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:38 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:31:38.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:38 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:31:38.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:31:39.055 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: pgmap v715: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 457 op/s 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.294 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:39 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:31:39.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: pgmap v715: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 457 op/s 2024-09-12T03:31:39.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:31:39.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:31:39.352 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:31:39.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.353 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:39 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:31:39.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: pgmap v715: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 457 op/s 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:39 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:31:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 -- 172.21.15.19:0/1389931321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 msgr2=0x7f346c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 --2- 172.21.15.19:0/1389931321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c102e30 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f345c009900 tx=0x7f345c02a410 comp rx=0 tx=0).stop 2024-09-12T03:31:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 -- 172.21.15.19:0/1389931321 shutdown_connections 2024-09-12T03:31:39.744 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 --2- 172.21.15.19:0/1389931321 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 --2- 172.21.15.19:0/1389931321 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c100520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 --2- 172.21.15.19:0/1389931321 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c102e30 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 -- 172.21.15.19:0/1389931321 >> 172.21.15.19:0/1389931321 conn(0x7f346c077230 msgr2=0x7f346c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 -- 172.21.15.19:0/1389931321 shutdown_connections 2024-09-12T03:31:39.745 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.744+0000 7f3472097640 1 -- 172.21.15.19:0/1389931321 wait complete. 2024-09-12T03:31:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 Processor -- start 2024-09-12T03:31:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 -- start start 2024-09-12T03:31:39.746 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f346c114170 con 0x7f346c102a50 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f346c113ff0 con 0x7f346c100c80 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f3472097640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1142f0 con 0x7f346c100100 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:51850/0 (socket says 172.21.15.19:51850) 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346b7fe640 1 -- 172.21.15.19:0/738596721 learned_addr learned my addr 172.21.15.19:0/738596721 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.745+0000 7f346affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39858/0 (socket says 172.21.15.19:39858) 2024-09-12T03:31:39.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 -- 172.21.15.19:0/738596721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 msgr2=0x7f346c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 -- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 msgr2=0x7f346c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 -- 172.21.15.19:0/738596721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f346c19f620 con 0x7f346c100c80 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346b7fe640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346bfff640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f346affd640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9f0 tx=0x7f346000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f3468ff9640 1 -- 172.21.15.19:0/738596721 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346000ccc0 con 0x7f346c100c80 2024-09-12T03:31:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346c19f910 con 0x7f346c100c80 2024-09-12T03:31:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.746+0000 7f3468ff9640 1 -- 172.21.15.19:0/738596721 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3460005020 con 0x7f346c100c80 2024-09-12T03:31:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.747+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346c1a71f0 con 0x7f346c100c80 2024-09-12T03:31:39.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.747+0000 7f3468ff9640 1 -- 172.21.15.19:0/738596721 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600049e0 con 0x7f346c100c80 2024-09-12T03:31:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.750+0000 7f3474470740 1 Processor -- start 2024-09-12T03:31:39.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.750+0000 7f3474470740 1 -- start start 2024-09-12T03:31:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fd3c0 0x55ec311fd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec31210e90 0x55ec31213280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec311632e0 con 0x55ec31210e90 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec3110ace0 con 0x55ec311fd3c0 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f3474470740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ec3115f2a0 con 0x55ec311fdce0 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fd3c0 0x55ec311fd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:51862/0 (socket says 172.21.15.19:51862) 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 -- 172.21.15.19:0/3015732131 learned_addr learned my addr 172.21.15.19:0/3015732131 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 -- 172.21.15.19:0/3015732131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fd3c0 msgr2=0x55ec311fd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fd3c0 0x55ec311fd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 -- 172.21.15.19:0/3015732131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec31210e90 msgr2=0x55ec31213280 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec31210e90 0x55ec31213280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.751+0000 7f346b7fe640 1 -- 172.21.15.19:0/3015732131 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec310ef440 con 0x55ec311fdce0 2024-09-12T03:31:39.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f3468ff9640 1 -- 172.21.15.19:0/738596721 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3460005560 con 0x7f346c100c80 2024-09-12T03:31:39.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f346b7fe640 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f345c005c40 tx=0x7f345c005dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47805538d6b5dbc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f3451ffb640 1 -- 172.21.15.19:0/3015732131 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c040070 con 0x55ec311fdce0 2024-09-12T03:31:39.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f3451ffb640 1 -- 172.21.15.19:0/3015732131 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c0047a0 con 0x55ec311fdce0 2024-09-12T03:31:39.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f3451ffb640 1 -- 172.21.15.19:0/3015732131 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035040 con 0x55ec311fdce0 2024-09-12T03:31:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.752+0000 7f3468ff9640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 0x7f34540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3468ff9640 1 -- 172.21.15.19:0/738596721 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f34600cefb0 con 0x7f346c100c80 2024-09-12T03:31:39.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f346bfff640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 0x7f34540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 -- 172.21.15.19:0/3015732131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 msgr2=0x55ec31210950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f345c005c40 tx=0x7f345c005dd0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f346bfff640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 0x7f34540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f3464009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 -- 172.21.15.19:0/3015732131 shutdown_connections 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec31210e90 0x55ec31213280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec311fdce0 0x55ec31210950 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 --2- 172.21.15.19:0/3015732131 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fd3c0 0x55ec311fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 -- 172.21.15.19:0/3015732131 >> 172.21.15.19:0/3015732131 conn(0x55ec311f9a30 msgr2=0x55ec311f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.753+0000 7f3474470740 1 -- 172.21.15.19:0/3015732131 shutdown_connections 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.754+0000 7f3474470740 1 -- 172.21.15.19:0/3015732131 wait complete. 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.754+0000 7f3474470740 1 -- start start 2024-09-12T03:31:39.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.754+0000 7f3474470740 1 Processor -- start 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49664/0 (socket says 172.21.15.19:49664) 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 -- 172.21.15.19:0/2071327919 learned_addr learned my addr 172.21.15.19:0/2071327919 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ec312139a0 con 0x55ec311fd3c0 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ec3111e050 con 0x55ec311fdce0 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ec31213b20 con 0x55ec31210e90 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346bfff640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f3474470740 10 client.0 _ll_register_callbacks cb 0x55ec3115e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 -- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 msgr2=0x55ec3120e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.759 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 -- 172.21.15.19:0/2071327919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 msgr2=0x55ec3120dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346affd640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.755+0000 7f346b7fe640 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec30ed43c0 con 0x55ec311fd3c0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f346bfff640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f346affd640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f345c003650 tx=0x7f345c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dd4d9238aac008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f344affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c04c020 con 0x55ec311fd3c0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f345c005410 con 0x55ec311fd3c0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c0047a0 con 0x55ec311fd3c0 2024-09-12T03:31:39.760 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f345c040070 con 0x55ec311fd3c0 2024-09-12T03:31:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f3474470740 10 client.17946 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.756+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ec3121f4d0 con 0x55ec311fd3c0 2024-09-12T03:31:39.761 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c0047a0 con 0x55ec311fd3c0 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f345c0316d0 con 0x55ec311fd3c0 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 10 client.17946 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 10 client.17946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 20 client.17946 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec31220820 0x55ec31222c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 4 client.17946 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:31:39.762 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec31170850 con 0x55ec31220820 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.757+0000 7f346affd640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec31220820 0x55ec31222c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.758+0000 7f346affd640 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec31220820 0x55ec31222c10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355b37f39a0080ea server_cookie=f312bc0659a578d9 in_seq=0 out_seq=0 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.758+0000 7f344affd640 10 client.17946 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f344affd640 1 -- 172.21.15.19:0/2071327919 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec31170850 con 0x55ec31220820 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f344affd640 10 client.17946 handle_command_reply: tid=0 2024-09-12T03:31:39.763 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 client.17946 shutdown 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 msgr2=0x55ec312108f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f345c003650 tx=0x7f345c009260 comp rx=0 tx=0).stop 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 shutdown_connections 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec31220820 0x55ec31222c10 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 --2- 172.21.15.19:0/2071327919 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec312108f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f344affd640 0 client.17946 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.759+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 >> 172.21.15.19:0/2071327919 conn(0x55ec311f9a30 msgr2=0x55ec311f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.760+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 shutdown_connections 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.760+0000 7f3474470740 1 -- 172.21.15.19:0/2071327919 wait complete. 2024-09-12T03:31:39.764 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.760+0000 7f3474470740 20 client.17946 trim_cache size 0 max 16384 2024-09-12T03:31:39.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 Processor -- start 2024-09-12T03:31:39.780 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 -- start start 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec31228750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ab3e0 con 0x55ec311fd3c0 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ad160 con 0x55ec311fdce0 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.779+0000 7f3474470740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1abab0 con 0x55ec31210e90 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49680/0 (socket says 172.21.15.19:49680) 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 -- 172.21.15.19:0/2734783614 learned_addr learned my addr 172.21.15.19:0/2734783614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346bfff640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346affd640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec31228750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.781 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 -- 172.21.15.19:0/2734783614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 msgr2=0x55ec3120ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 -- 172.21.15.19:0/2734783614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 msgr2=0x55ec31228750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec31228750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 -- 172.21.15.19:0/2734783614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec3120c130 con 0x55ec311fd3c0 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346bfff640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f346b7fe640 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f345c055040 tx=0x7f345c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=140f94644570a8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f34517fa640 1 -- 172.21.15.19:0/2734783614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c04c040 con 0x55ec311fd3c0 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f34517fa640 1 -- 172.21.15.19:0/2734783614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c004d60 con 0x55ec311fd3c0 2024-09-12T03:31:39.782 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.780+0000 7f34517fa640 1 -- 172.21.15.19:0/2734783614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c035040 con 0x55ec311fd3c0 2024-09-12T03:31:39.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 -- 172.21.15.19:0/2734783614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 msgr2=0x55ec31228210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f345c055040 tx=0x7f345c0046c0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 -- 172.21.15.19:0/2734783614 shutdown_connections 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec31228750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 --2- 172.21.15.19:0/2734783614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31228210 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 -- 172.21.15.19:0/2734783614 >> 172.21.15.19:0/2734783614 conn(0x55ec311f9a30 msgr2=0x55ec31209110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 -- 172.21.15.19:0/2734783614 shutdown_connections 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.781+0000 7f3474470740 1 -- 172.21.15.19:0/2734783614 wait complete. 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 -- start start 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 Processor -- start 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31219260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac9d0 con 0x55ec311fd3c0 2024-09-12T03:31:39.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1ac240 con 0x55ec311fdce0 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f346c1a9300 con 0x55ec31210e90 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f3474470740 10 client.0 _ll_register_callbacks cb 0x7f3464009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31219260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f346affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f346b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31219260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:49694/0 (socket says 172.21.15.19:49694) 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f346affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39896/0 (socket says 172.21.15.19:39896) 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.782+0000 7f346b7fe640 1 -- 172.21.15.19:0/3175073558 learned_addr learned my addr 172.21.15.19:0/3175073558 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346bfff640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 -- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 msgr2=0x55ec3120ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 -- 172.21.15.19:0/3175073558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 msgr2=0x55ec31219260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:31:39.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31219260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 -- 172.21.15.19:0/3175073558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec3120b140 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f346affd640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ec3120d430 tx=0x7f34600054d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54faa10bd0dd0f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f344affd640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34600b3070 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f34600ba1e0 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f345c004d60 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f346001c040 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f3474470740 10 client.27290 fetch_fsmap learned FSMap version 58 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.783+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ec3128af00 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f345c004d60 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 58) ==== 8442+0+0 (secure 0 0 0) 0x7f346009ba80 con 0x55ec311fdce0 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 10 client.27290 fetch_fsmap finished waiting for FSMap version 58 2024-09-12T03:31:39.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 10 client.27290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 20 client.27290 populate_metadata read hostname 'smithi019' 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec3128b740 0x55ec3128db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 4 client.27290 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ec3116ad60 con 0x55ec3128b740 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.784+0000 7f346b7fe640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec3128b740 0x55ec3128db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f346b7fe640 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec3128b740 0x55ec3128db60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b4c6c5f3f8df8d3 server_cookie=8578503825d62ca3 in_seq=0 out_seq=0 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f344affd640 10 client.27290 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f344affd640 1 -- 172.21.15.19:0/3175073558 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ec3116ad60 con 0x55ec3128b740 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f344affd640 10 client.27290 handle_command_reply: tid=0 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 client.27290 shutdown 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 msgr2=0x55ec3120a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55ec3120d430 tx=0x7f34600054d0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 shutdown_connections 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ec31210e90 0x55ec3120ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ec3128b740 0x55ec3128db60 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ec311fdce0 0x55ec3120a6c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f344affd640 0 client.27290 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 --2- 172.21.15.19:0/3175073558 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ec311fd3c0 0x55ec31219260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 >> 172.21.15.19:0/3175073558 conn(0x55ec311f9a30 msgr2=0x55ec3121bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 shutdown_connections 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 1 -- 172.21.15.19:0/3175073558 wait complete. 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.785+0000 7f3474470740 20 client.27290 trim_cache size 0 max 16384 2024-09-12T03:31:39.791 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "6e03c154-48e2-4034-8193-28561de9f0a1": { 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "6e03c154-48e2-4034-8193-28561de9f0a1", 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:31:39.792 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:31:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 msgr2=0x7f34540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 0x7f34540a5040 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3464009fd0 tx=0x7f3464009500 comp rx=0 tx=0).stop 2024-09-12T03:31:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 msgr2=0x7f346c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:31:39.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f346000e9f0 tx=0x7f346000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 shutdown_connections 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f34540a2b80 0x7f34540a5040 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f346c102a50 0x7f346c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f346c100c80 0x7f346c19abb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 --2- 172.21.15.19:0/738596721 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f346c100100 0x7f346c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 >> 172.21.15.19:0/738596721 conn(0x7f346c077230 msgr2=0x7f346c0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.786+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 shutdown_connections 2024-09-12T03:31:39.794 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:31:39.787+0000 7f3472097640 1 -- 172.21.15.19:0/738596721 wait complete. 2024-09-12T03:31:40.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:40 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:40.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:40 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:40.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:40 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:40.701 INFO:tasks.cephfs.filesystem:scrub status for tag:6e03c154-48e2-4034-8193-28561de9f0a1 - {'path': '/', 'tag': '6e03c154-48e2-4034-8193-28561de9f0a1', 'options': 'recursive,force'} 2024-09-12T03:31:41.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:41 smithi121 ceph-mon[34963]: pgmap v716: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 404 op/s 2024-09-12T03:31:41.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:41 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:41 smithi019 ceph-mon[30953]: pgmap v716: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 404 op/s 2024-09-12T03:31:41.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:41 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:41 smithi135 ceph-mon[35474]: pgmap v716: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 404 op/s 2024-09-12T03:31:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:41 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:42.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:42 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:42.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:42 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:42 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:31:43.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:43 smithi121 ceph-mon[34963]: pgmap v717: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 475 op/s 2024-09-12T03:31:43.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:43 smithi019 ceph-mon[30953]: pgmap v717: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 475 op/s 2024-09-12T03:31:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:43 smithi135 ceph-mon[35474]: pgmap v717: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 475 op/s 2024-09-12T03:31:44.399 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:44 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:44.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:44 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:44.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:44 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:31:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: pgmap v718: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 499 op/s 2024-09-12T03:31:45.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:45.415 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: pgmap v718: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 499 op/s 2024-09-12T03:31:45.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:45.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: pgmap v718: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 499 op/s 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:31:45.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:31:46.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:46 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:46.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:46 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:46 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:47.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:47 smithi135 ceph-mon[35474]: pgmap v719: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-12T03:31:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:47 smithi121 ceph-mon[34963]: pgmap v719: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-12T03:31:47.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:47 smithi019 ceph-mon[30953]: pgmap v719: 97 pgs: 97 active+clean; 1.5 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 487 op/s 2024-09-12T03:31:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:49 smithi135 ceph-mon[35474]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 518 op/s 2024-09-12T03:31:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:49 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:49.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:49 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:49.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:49 smithi121 ceph-mon[34963]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 518 op/s 2024-09-12T03:31:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:49 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:49 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:49 smithi019 ceph-mon[30953]: pgmap v720: 97 pgs: 97 active+clean; 1.4 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 518 op/s 2024-09-12T03:31:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:49 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:49 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:50.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:50 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:31:50.445 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:50 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:50 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:31:50.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:50 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:50.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:50 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:31:50.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:50 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:31:51.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:51 smithi135 ceph-mon[35474]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 458 op/s 2024-09-12T03:31:51.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:51 smithi121 ceph-mon[34963]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 458 op/s 2024-09-12T03:31:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:51 smithi019 ceph-mon[30953]: pgmap v721: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 458 op/s 2024-09-12T03:31:53.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:53 smithi135 ceph-mon[35474]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-12T03:31:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:53 smithi121 ceph-mon[34963]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-12T03:31:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:53 smithi019 ceph-mon[30953]: pgmap v722: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 489 op/s 2024-09-12T03:31:55.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:55 smithi135 ceph-mon[35474]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-09-12T03:31:55.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:55 smithi121 ceph-mon[34963]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-09-12T03:31:55.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:55 smithi019 ceph-mon[30953]: pgmap v723: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 438 op/s 2024-09-12T03:31:57.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:57 smithi135 ceph-mon[35474]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-12T03:31:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:57 smithi121 ceph-mon[34963]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-12T03:31:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:57 smithi019 ceph-mon[30953]: pgmap v724: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 397 op/s 2024-09-12T03:31:59.451 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:31:59 smithi121 ceph-mon[34963]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-09-12T03:31:59.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:31:59 smithi135 ceph-mon[35474]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-09-12T03:31:59.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:31:59 smithi019 ceph-mon[30953]: pgmap v725: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 436 op/s 2024-09-12T03:32:00.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:00 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:00.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:00 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:00.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:00 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:01.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:01 smithi135 ceph-mon[35474]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-09-12T03:32:01.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:01 smithi121 ceph-mon[34963]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-09-12T03:32:01.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:01 smithi019 ceph-mon[30953]: pgmap v726: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 400 op/s 2024-09-12T03:32:03.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:03 smithi135 ceph-mon[35474]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-12T03:32:03.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:03 smithi121 ceph-mon[34963]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-12T03:32:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:03 smithi019 ceph-mon[30953]: pgmap v727: 97 pgs: 97 active+clean; 1.4 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-09-12T03:32:05.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:05 smithi135 ceph-mon[35474]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-09-12T03:32:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:05 smithi121 ceph-mon[34963]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-09-12T03:32:05.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:05 smithi019 ceph-mon[30953]: pgmap v728: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 432 op/s 2024-09-12T03:32:07.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:07 smithi135 ceph-mon[35474]: pgmap v729: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2024-09-12T03:32:07.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:07 smithi121 ceph-mon[34963]: pgmap v729: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2024-09-12T03:32:07.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:07 smithi019 ceph-mon[30953]: pgmap v729: 97 pgs: 97 active+clean; 1.3 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 423 op/s 2024-09-12T03:32:09.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:09 smithi135 ceph-mon[35474]: pgmap v730: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2024-09-12T03:32:09.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:09 smithi121 ceph-mon[34963]: pgmap v730: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2024-09-12T03:32:09.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:09 smithi019 ceph-mon[30953]: pgmap v730: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 452 op/s 2024-09-12T03:32:10.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:32:10.970 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:11.408 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:11 smithi019 ceph-mon[30953]: pgmap v731: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-12T03:32:11.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:11 smithi135 ceph-mon[35474]: pgmap v731: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-12T03:32:11.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:11 smithi121 ceph-mon[34963]: pgmap v731: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 398 op/s 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 -- 172.21.15.19:0/339551506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408101de0 msgr2=0x7fc4081021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 --2- 172.21.15.19:0/339551506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408101de0 0x7fc4081021c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009900 tx=0x7fc3f802a410 comp rx=0 tx=0).stop 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 -- 172.21.15.19:0/339551506 shutdown_connections 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 --2- 172.21.15.19:0/339551506 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408106f10 0x7fc40810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 --2- 172.21.15.19:0/339551506 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408102790 0x7fc4081067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 --2- 172.21.15.19:0/339551506 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408101de0 0x7fc4081021c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.669 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 -- 172.21.15.19:0/339551506 >> 172.21.15.19:0/339551506 conn(0x7fc4080fd8c0 msgr2=0x7fc4080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 -- 172.21.15.19:0/339551506 shutdown_connections 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.645+0000 7fc40d543640 1 -- 172.21.15.19:0/339551506 wait complete. 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 Processor -- start 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 -- start start 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408101de0 0x7fc4081984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408106f10 0x7fc40819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc408111f70 con 0x7fc408102790 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc408111df0 con 0x7fc408106f10 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc40d543640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081120f0 con 0x7fc408101de0 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.670 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57394/0 (socket says 172.21.15.19:57394) 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc3fffff640 1 -- 172.21.15.19:0/2727453634 learned_addr learned my addr 172.21.15.19:0/2727453634 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc4077fe640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408106f10 0x7fc40819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.646+0000 7fc406ffd640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408101de0 0x7fc4081984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 -- 172.21.15.19:0/2727453634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408101de0 msgr2=0x7fc4081984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408101de0 0x7fc4081984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 -- 172.21.15.19:0/2727453634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408106f10 msgr2=0x7fc40819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408106f10 0x7fc40819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 -- 172.21.15.19:0/2727453634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40819d480 con 0x7fc408102790 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc3fffff640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9f0 tx=0x7fc3f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc404ff9640 1 -- 172.21.15.19:0/2727453634 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f000cd80 con 0x7fc408102790 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc404ff9640 1 -- 172.21.15.19:0/2727453634 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f0010910 con 0x7fc408102790 2024-09-12T03:32:11.671 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40819d770 con 0x7fc408102790 2024-09-12T03:32:11.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc404ff9640 1 -- 172.21.15.19:0/2727453634 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0018990 con 0x7fc408102790 2024-09-12T03:32:11.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.647+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4081a5050 con 0x7fc408102790 2024-09-12T03:32:11.672 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.649+0000 7fc404ff9640 1 -- 172.21.15.19:0/2727453634 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc3f0010ac0 con 0x7fc408102790 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 Processor -- start 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc404ff9640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 0x7fc3f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 -- start start 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc4077fe640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 0x7fc3f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc404ff9640 1 -- 172.21.15.19:0/2727453634 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc3f00cf500 con 0x7fc408102790 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d44a0 0x55a5896d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 0x55a5896df140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a58963a2e0 con 0x55a5896df680 2024-09-12T03:32:11.673 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a5895e1ce0 con 0x55a5896d44a0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.650+0000 7fc40f91c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5896362a0 con 0x55a5896d4dc0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc4077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 0x55a5896df140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc4077fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57400/0 (socket says 172.21.15.19:57400) 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc4077fe640 1 -- 172.21.15.19:0/3281530471 learned_addr learned my addr 172.21.15.19:0/3281530471 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc406ffd640 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d44a0 0x55a5896d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc4077fe640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 0x7fc3f40a5040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc400009370 tx=0x7fc4000092c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 -- 172.21.15.19:0/3281530471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d44a0 msgr2=0x55a5896d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d44a0 0x55a5896d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 -- 172.21.15.19:0/3281530471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 msgr2=0x55a5896e1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.674 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 -- 172.21.15.19:0/3281530471 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5895c6440 con 0x55a5896d4dc0 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc4077fe640 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 0x55a5896df140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9c0 tx=0x7fc3f009c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d6ac08a6164506d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.651+0000 7fc3fd7fa640 1 -- 172.21.15.19:0/3281530471 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00a4980 con 0x55a5896d4dc0 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc3fd7fa640 1 -- 172.21.15.19:0/3281530471 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f009a010 con 0x55a5896d4dc0 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc3fd7fa640 1 -- 172.21.15.19:0/3281530471 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f009b160 con 0x55a5896d4dc0 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 -- 172.21.15.19:0/3281530471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 msgr2=0x55a5896df140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 0x55a5896df140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9c0 tx=0x7fc3f009c4d0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 -- 172.21.15.19:0/3281530471 shutdown_connections 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896df680 0x55a5896e1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d4dc0 0x55a5896df140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3281530471 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d44a0 0x55a5896d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 -- 172.21.15.19:0/3281530471 >> 172.21.15.19:0/3281530471 conn(0x55a5896d0a30 msgr2=0x55a5896d0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 -- 172.21.15.19:0/3281530471 shutdown_connections 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.652+0000 7fc40f91c740 1 -- 172.21.15.19:0/3281530471 wait complete. 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 -- start start 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 Processor -- start 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 0x55a5896dc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 0x55a5896dc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a5896e2250 con 0x55a5896d4dc0 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a5895f5050 con 0x55a5896df680 2024-09-12T03:32:11.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a5896e23d0 con 0x55a5896d44a0 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc40f91c740 10 client.0 _ll_register_callbacks cb 0x55a589635160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 0x55a5896dc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 0x55a5896dc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55072/0 (socket says 172.21.15.19:55072) 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc406ffd640 1 -- 172.21.15.19:0/972788641 learned_addr learned my addr 172.21.15.19:0/972788641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 -- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 msgr2=0x55a5896df050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 -- 172.21.15.19:0/972788641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 msgr2=0x55a5896dc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 0x55a5896dc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.653+0000 7fc4077fe640 1 -- 172.21.15.19:0/972788641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5893ab3c0 con 0x55a5896df680 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc406ffd640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3fffff640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 0x55a5896dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc4077fe640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 0x55a5896dc8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc400005fd0 tx=0x7fc400004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979d00f92ae45baf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001eb10 con 0x55a5896df680 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f009a010 con 0x55a5896df680 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc40001ecb0 con 0x55a5896df680 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc40001f430 con 0x55a5896df680 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc40f91c740 10 client.27302 fetch_fsmap learned FSMap version 61 2024-09-12T03:32:11.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a5896edd30 con 0x55a5896df680 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f009a010 con 0x55a5896df680 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 61) ==== 8426+0+0 (secure 0 0 0) 0x7fc400027650 con 0x55a5896df680 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc40f91c740 10 client.27302 fetch_fsmap finished waiting for FSMap version 61 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.654+0000 7fc40f91c740 10 client.27302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc40f91c740 20 client.27302 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5896ef2c0 0x55a5896f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc40f91c740 4 client.27302 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a589647850 con 0x55a5896ef2c0 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc406ffd640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5896ef2c0 0x55a5896f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc406ffd640 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5896ef2c0 0x55a5896f1700 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef06c1620cecd3a5 server_cookie=8463edb56788671 in_seq=0 out_seq=0 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.655+0000 7fc3ee7fc640 10 client.27302 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/972788641 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a589647850 con 0x55a5896ef2c0 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc3ee7fc640 10 client.27302 handle_command_reply: tid=0 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 client.27302 shutdown 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 msgr2=0x55a5896dc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 0x55a5896dc8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc400005fd0 tx=0x7fc400004a40 comp rx=0 tx=0).stop 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 shutdown_connections 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a5896ef2c0 0x55a5896f1700 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896df680 0x55a5896dc8f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d4dc0 0x55a5896dc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 --2- 172.21.15.19:0/972788641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896d44a0 0x55a5896df050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc3ee7fc640 0 client.27302 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:11.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 >> 172.21.15.19:0/972788641 conn(0x55a5896d0a30 msgr2=0x55a5896d6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 shutdown_connections 2024-09-12T03:32:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 1 -- 172.21.15.19:0/972788641 wait complete. 2024-09-12T03:32:11.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.656+0000 7fc40f91c740 20 client.27302 trim_cache size 0 max 16384 2024-09-12T03:32:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.674+0000 7fc40f91c740 1 Processor -- start 2024-09-12T03:32:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.674+0000 7fc40f91c740 1 -- start start 2024-09-12T03:32:11.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896de270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081ad610 con 0x55a5896d44a0 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081af2d0 con 0x55a5896d4dc0 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc40f91c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081adce0 con 0x55a5896df680 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896de270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57418/0 (socket says 172.21.15.19:57418) 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc406ffd640 1 -- 172.21.15.19:0/3720401391 learned_addr learned my addr 172.21.15.19:0/3720401391 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc4077fe640 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 -- 172.21.15.19:0/3720401391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 msgr2=0x55a5896f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 -- 172.21.15.19:0/3720401391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 msgr2=0x55a5896de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 -- 172.21.15.19:0/3720401391 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5896df3c0 con 0x55a5896d4dc0 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fffff640 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896f6fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a5896d1890 tx=0x7fc3f001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5534839e80b03e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fcff9640 1 -- 172.21.15.19:0/3720401391 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00a4980 con 0x55a5896d4dc0 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.675+0000 7fc3fcff9640 1 -- 172.21.15.19:0/3720401391 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00b8070 con 0x55a5896d4dc0 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc3fcff9640 1 -- 172.21.15.19:0/3720401391 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0097cc0 con 0x55a5896d4dc0 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 -- 172.21.15.19:0/3720401391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 msgr2=0x55a5896f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896f6fd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a5896d1890 tx=0x7fc3f001a040 comp rx=0 tx=0).stop 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 -- 172.21.15.19:0/3720401391 shutdown_connections 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896f6fd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 --2- 172.21.15.19:0/3720401391 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896de270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 -- 172.21.15.19:0/3720401391 >> 172.21.15.19:0/3720401391 conn(0x55a5896d0a30 msgr2=0x55a5896ddb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 -- 172.21.15.19:0/3720401391 shutdown_connections 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.676+0000 7fc40f91c740 1 -- 172.21.15.19:0/3720401391 wait complete. 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 -- start start 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 Processor -- start 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081aeb40 con 0x55a5896d44a0 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081ae3b0 con 0x55a5896d4dc0 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc4081ab530 con 0x55a5896df680 2024-09-12T03:32:11.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc40f91c740 10 client.0 _ll_register_callbacks cb 0x7fc400011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc3fffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc406ffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57432/0 (socket says 172.21.15.19:57432) 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc406ffd640 1 -- 172.21.15.19:0/446188818 learned_addr learned my addr 172.21.15.19:0/446188818 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.677+0000 7fc4077fe640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 -- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 msgr2=0x55a5896e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 -- 172.21.15.19:0/446188818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 msgr2=0x55a5896e4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 -- 172.21.15.19:0/446188818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5896e7b20 con 0x55a5896d4dc0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc406ffd640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3fffff640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896e5310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a5896dd000 tx=0x7fc3f0018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e6b1ef33bcf04bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:11.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f00a4980 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc3f0098380 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f00b8070 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fc3f00a4bd0 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc40f91c740 10 client.27308 fetch_fsmap learned FSMap version 61 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55a58975df70 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.678+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00b8070 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 61) ==== 8426+0+0 (secure 0 0 0) 0x7fc3f00a1250 con 0x55a5896d4dc0 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 10 client.27308 fetch_fsmap finished waiting for FSMap version 61 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 10 client.27308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:11.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 20 client.27308 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a58975e290 0x55a5897606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 4 client.27308 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a589641d60 con 0x55a58975e290 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc406ffd640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a58975e290 0x55a5897606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc406ffd640 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a58975e290 0x55a5897606b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8e81195b417a238 server_cookie=4506dc19f69e3aeb in_seq=0 out_seq=0 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.679+0000 7fc3ee7fc640 10 client.27308 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc3ee7fc640 1 -- 172.21.15.19:0/446188818 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a589641d60 con 0x55a58975e290 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc3ee7fc640 10 client.27308 handle_command_reply: tid=0 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 client.27308 shutdown 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 msgr2=0x55a5896e5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896e5310 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a5896dd000 tx=0x7fc3f0018420 comp rx=0 tx=0).stop 2024-09-12T03:32:11.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 shutdown_connections 2024-09-12T03:32:11.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a5896df680 0x55a5896e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a58975e290 0x55a5897606b0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a5896d4dc0 0x55a5896e5310 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 --2- 172.21.15.19:0/446188818 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a5896d44a0 0x55a5896e4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc3ee7fc640 0 client.27308 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 >> 172.21.15.19:0/446188818 conn(0x55a5896d0a30 msgr2=0x55a5896cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 shutdown_connections 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 1 -- 172.21.15.19:0/446188818 wait complete. 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.681+0000 7fc40f91c740 20 client.27308 trim_cache size 0 max 16384 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:32:11.695 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 msgr2=0x7fc3f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 0x7fc3f40a5040 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc400009370 tx=0x7fc4000092c0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 msgr2=0x7fc408198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e9f0 tx=0x7fc3f000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 shutdown_connections 2024-09-12T03:32:11.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc408106f10 0x7fc40819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc3f40a2b80 0x7fc3f40a5040 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc408102790 0x7fc408198a10 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 --2- 172.21.15.19:0/2727453634 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc408101de0 0x7fc4081984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 >> 172.21.15.19:0/2727453634 conn(0x7fc4080fd8c0 msgr2=0x7fc4080fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 shutdown_connections 2024-09-12T03:32:11.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:11.682+0000 7fc40d543640 1 -- 172.21.15.19:0/2727453634 wait complete. 2024-09-12T03:32:12.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:32:12.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:32:12.770 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- 172.21.15.19:0/2661858270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 msgr2=0x7f208810d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 --2- 172.21.15.19:0/2661858270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810d110 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f2080009ca0 tx=0x7f208002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- 172.21.15.19:0/2661858270 shutdown_connections 2024-09-12T03:32:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 --2- 172.21.15.19:0/2661858270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810d110 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 --2- 172.21.15.19:0/2661858270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 0x7f2088104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 --2- 172.21.15.19:0/2661858270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f2088100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- 172.21.15.19:0/2661858270 >> 172.21.15.19:0/2661858270 conn(0x7f20880fbee0 msgr2=0x7f20880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- 172.21.15.19:0/2661858270 shutdown_connections 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- 172.21.15.19:0/2661858270 wait complete. 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 Processor -- start 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.280+0000 7f208f1d2640 1 -- start start 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 0x7f208810ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f2088111f70 con 0x7f2088105590 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2088111df0 con 0x7f2088100460 2024-09-12T03:32:13.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208f1d2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f20881120f0 con 0x7f2088100e10 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208d76f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:48564/0 (socket says 172.21.15.19:48564) 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 -- 172.21.15.19:0/3117344159 learned_addr learned my addr 172.21.15.19:0/3117344159 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208d76f640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57438/0 (socket says 172.21.15.19:57438) 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f2087fff640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 0x7f208810ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 -- 172.21.15.19:0/3117344159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 msgr2=0x7f208810ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 0x7f208810ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 -- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 msgr2=0x7f208810f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 -- 172.21.15.19:0/3117344159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20881a6870 con 0x7f2088100460 2024-09-12T03:32:13.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f208cf6e640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.281+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c017070 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.282+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20881a6b60 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.282+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f207c009c30 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.282+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207c00f6b0 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.282+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20881a70f0 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.283+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2088100840 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.284+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f207c0109d0 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.284+0000 7f2085ffb640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 0x7f20780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.284+0000 7f2087fff640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 0x7f20780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.284+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f207c0cebf0 con 0x7f2088100460 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.285+0000 7f2087fff640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 0x7f20780a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20881102c0 tx=0x7f2074005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:13.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.289+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f207c009de0 con 0x7f2088100460 2024-09-12T03:32:13.507 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.506+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f208802d7c0 con 0x7f2088100460 2024-09-12T03:32:13.507 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:13 smithi019 ceph-mon[30953]: pgmap v732: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-12T03:32:13.508 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.507+0000 7f2085ffb640 1 -- 172.21.15.19:0/3117344159 <== mon.2 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+12002 (secure 0 0 0) 0x7f207c09b4b0 con 0x7f2088100460 2024-09-12T03:32:13.508 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:32:13.508 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":61,"btime":"2024-09-12T03:31:59:200608+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:31:59.165632+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:32:13.508 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 61 2024-09-12T03:32:13.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 msgr2=0x7f20780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:13.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 0x7f20780a5040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f20881102c0 tx=0x7f2074005f90 comp rx=0 tx=0).stop 2024-09-12T03:32:13.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 msgr2=0x7f208810e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:13.510 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f207c00d950 tx=0x7f207c00de10 comp rx=0 tx=0).stop 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 shutdown_connections 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f20780a2b80 0x7f20780a5040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f2088105590 0x7f208810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f2088100e10 0x7f208810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 --2- 172.21.15.19:0/3117344159 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2088100460 0x7f208810e830 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 >> 172.21.15.19:0/3117344159 conn(0x7f20880fbee0 msgr2=0x7f20880fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 shutdown_connections 2024-09-12T03:32:13.511 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:13.510+0000 7f208f1d2640 1 -- 172.21.15.19:0/3117344159 wait complete. 2024-09-12T03:32:13.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:13 smithi135 ceph-mon[35474]: pgmap v732: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-12T03:32:13.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:13 smithi121 ceph-mon[34963]: pgmap v732: 97 pgs: 97 active+clean; 1.3 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 427 op/s 2024-09-12T03:32:13.971 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.2 2024-09-12T03:32:13.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:32:14.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:32:14.320 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:14.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:14 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3117344159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:14.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:14 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3117344159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:14.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:14 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3117344159' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 -- 172.21.15.19:0/915430148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 msgr2=0x7f6988101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 --2- 172.21.15.19:0/915430148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f6988101200 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f697802a480 comp rx=0 tx=0).stop 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 -- 172.21.15.19:0/915430148 shutdown_connections 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 --2- 172.21.15.19:0/915430148 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102ef0 0x7f698810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 --2- 172.21.15.19:0/915430148 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102570 0x7f69881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 --2- 172.21.15.19:0/915430148 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f6988101200 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 -- 172.21.15.19:0/915430148 >> 172.21.15.19:0/915430148 conn(0x7f69880fc7d0 msgr2=0x7f69880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 -- 172.21.15.19:0/915430148 shutdown_connections 2024-09-12T03:32:14.889 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.873+0000 7f698e861640 1 -- 172.21.15.19:0/915430148 wait complete. 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.874+0000 7f698e861640 1 Processor -- start 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.874+0000 7f698e861640 1 -- start start 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102570 0x7f6988198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 0x7f698819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6988110f50 con 0x7f6988100e20 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6988110dd0 con 0x7f6988102ef0 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698e861640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f69881110d0 con 0x7f6988102570 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57466/0 (socket says 172.21.15.19:57466) 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 -- 172.21.15.19:0/145597508 learned_addr learned my addr 172.21.15.19:0/145597508 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f69877fe640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102570 0x7f6988198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:14.890 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698cdfe640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 0x7f698819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 -- 172.21.15.19:0/145597508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102570 msgr2=0x7f6988198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102570 0x7f6988198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 -- 172.21.15.19:0/145597508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 msgr2=0x7f698819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 0x7f698819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 -- 172.21.15.19:0/145597508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698819d470 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f698cdfe640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 0x7f698819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.875+0000 7f6987fff640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6978006010 tx=0x7f6978003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.876+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697803f070 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.876+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6978031c30 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.876+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698819d700 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.876+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6978035030 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.876+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a5040 con 0x7f6988100e20 2024-09-12T03:32:14.891 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.877+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69881021b0 con 0x7f6988100e20 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.878+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6978042070 con 0x7f6988100e20 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.878+0000 7f69857fa640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 0x7f69700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.879+0000 7f69877fe640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 0x7f69700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.879+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f69780ef7a0 con 0x7f6988100e20 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.879+0000 7f69877fe640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 0x7f69700a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f69880ff060 tx=0x7f697c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:14.892 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:14.883+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f69780bc6b0 con 0x7f6988100e20 2024-09-12T03:32:14.931 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:15.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.123+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f698810a0a0 con 0x7f6988100e20 2024-09-12T03:32:15.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.123+0000 7f69857fa640 1 -- 172.21.15.19:0/145597508 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+12003 (secure 0 0 0) 0x7f69780bbfb0 con 0x7f6988100e20 2024-09-12T03:32:15.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:32:15.124 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":62,"btime":"2024-09-12T03:32:14:226820+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":62,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:32:14.168020+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:32:15.124 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 62 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.125+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 msgr2=0x7f69700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.125+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 0x7f69700a5040 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f69880ff060 tx=0x7f697c00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.125+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 msgr2=0x7f69881984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.125+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6978006010 tx=0x7f6978003c10 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 shutdown_connections 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6988102ef0 0x7f698819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6988102570 0x7f6988198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f69700a2b80 0x7f69700a5040 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 --2- 172.21.15.19:0/145597508 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6988100e20 0x7f69881984c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 >> 172.21.15.19:0/145597508 conn(0x7f69880fc7d0 msgr2=0x7f698806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:15.126 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 shutdown_connections 2024-09-12T03:32:15.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:15.126+0000 7f698e861640 1 -- 172.21.15.19:0/145597508 wait complete. 2024-09-12T03:32:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:15 smithi019 ceph-mon[30953]: pgmap v733: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 416 op/s 2024-09-12T03:32:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:15 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:15.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:15 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/145597508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:15 smithi135 ceph-mon[35474]: pgmap v733: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 416 op/s 2024-09-12T03:32:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:15 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:15.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:15 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/145597508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:15 smithi121 ceph-mon[34963]: pgmap v733: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 416 op/s 2024-09-12T03:32:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:15 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:15 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/145597508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:16.635 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0' 2024-09-12T03:32:16.635 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db / --timeout 90 --expiration 147.2 --await-for 45.0 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 -- 172.21.15.19:0/588429559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc106f10 msgr2=0x7f97dc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/588429559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc106f10 0x7f97dc10ea90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009ca0 tx=0x7f97d402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 -- 172.21.15.19:0/588429559 shutdown_connections 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/588429559 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc106f10 0x7f97dc10ea90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/588429559 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/588429559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc101de0 0x7f97dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 -- 172.21.15.19:0/588429559 >> 172.21.15.19:0/588429559 conn(0x7f97dc0fd8c0 msgr2=0x7f97dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 -- 172.21.15.19:0/588429559 shutdown_connections 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.890+0000 7f97e1cbb640 1 -- 172.21.15.19:0/588429559 wait complete. 2024-09-12T03:32:16.962 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 Processor -- start 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 -- start start 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 0x7f97dc19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111f70 con 0x7f97dc101de0 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc111df0 con 0x7f97dc106f10 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97e1cbb640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1120f0 con 0x7f97dc102790 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57474/0 (socket says 172.21.15.19:57474) 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.891+0000 7f97db7fe640 1 -- 172.21.15.19:0/4211694522 learned_addr learned my addr 172.21.15.19:0/4211694522 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97daffd640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97dbfff640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 0x7f97dc19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.963 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 -- 172.21.15.19:0/4211694522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 msgr2=0x7f97dc19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc19ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 -- 172.21.15.19:0/4211694522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 msgr2=0x7f97dc19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 0x7f97dc19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 -- 172.21.15.19:0/4211694522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc19f5a0 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97dbfff640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 0x7f97dc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97daffd640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97db7fe640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d950 tx=0x7f97cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97d8ff9640 1 -- 172.21.15.19:0/4211694522 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc017030 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97d8ff9640 1 -- 172.21.15.19:0/4211694522 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc004020 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc19f890 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97d8ff9640 1 -- 172.21.15.19:0/4211694522 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0047a0 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.892+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1a7170 con 0x7f97dc101de0 2024-09-12T03:32:16.964 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.894+0000 7f97d8ff9640 1 -- 172.21.15.19:0/4211694522 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97cc004940 con 0x7f97dc101de0 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 Processor -- start 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 -- start start 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a517090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a524760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55668a47d2f0 con 0x55668a5175d0 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55668a424cf0 con 0x55668a522370 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97e4094740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55668a4792b0 con 0x55668a518b40 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a517090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57486/0 (socket says 172.21.15.19:57486) 2024-09-12T03:32:16.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a517090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55142/0 (socket says 172.21.15.19:55142) 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 -- 172.21.15.19:0/2366143269 learned_addr learned my addr 172.21.15.19:0/2366143269 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97daffd640 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a524760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97d8ff9640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 0x7f97c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 -- 172.21.15.19:0/2366143269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 msgr2=0x55668a517090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a517090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 -- 172.21.15.19:0/2366143269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 msgr2=0x55668a524760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a524760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.895+0000 7f97db7fe640 1 -- 172.21.15.19:0/2366143269 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668a4093a0 con 0x55668a5175d0 2024-09-12T03:32:16.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97dbfff640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 0x7f97c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97d8ff9640 1 -- 172.21.15.19:0/4211694522 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97cc0ceb60 con 0x7f97dc101de0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97db7fe640 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0a01a0 tx=0x7f97cc097440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f23b9040834535 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97c1ffb640 1 -- 172.21.15.19:0/2366143269 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09aeb0 con 0x55668a5175d0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97c1ffb640 1 -- 172.21.15.19:0/2366143269 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc09fba0 con 0x55668a5175d0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.896+0000 7f97c1ffb640 1 -- 172.21.15.19:0/2366143269 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09acd0 con 0x55668a5175d0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 -- 172.21.15.19:0/2366143269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 msgr2=0x55668a517a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97dbfff640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 0x7f97c40a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005fe0 tx=0x7f97d40095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0a01a0 tx=0x7f97cc097440 comp rx=0 tx=0).stop 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 -- 172.21.15.19:0/2366143269 shutdown_connections 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a524760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a517a10 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 --2- 172.21.15.19:0/2366143269 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a517090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 -- 172.21.15.19:0/2366143269 >> 172.21.15.19:0/2366143269 conn(0x55668a4e0310 msgr2=0x55668a513c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 -- 172.21.15.19:0/2366143269 shutdown_connections 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.897+0000 7f97e4094740 1 -- 172.21.15.19:0/2366143269 wait complete. 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 -- start start 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 Processor -- start 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a51f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a51f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55668a525070 con 0x55668a5175d0 2024-09-12T03:32:16.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55668a438060 con 0x55668a522370 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55668a5251f0 con 0x55668a518b40 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97e4094740 10 client.0 _ll_register_callbacks cb 0x55668a18b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57492/0 (socket says 172.21.15.19:57492) 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97daffd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a51f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97db7fe640 1 -- 172.21.15.19:0/1762430459 learned_addr learned my addr 172.21.15.19:0/1762430459 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.898+0000 7f97dbfff640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a51f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 -- 172.21.15.19:0/1762430459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 msgr2=0x55668a51f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a51f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 -- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 msgr2=0x55668a51f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a51f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.969 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 -- 172.21.15.19:0/1762430459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668a1ee3c0 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97daffd640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a51f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97db7fe640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0975b0 tx=0x7f97cc0d2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59412089a07b680b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97baffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09acd0 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55668a48a860 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97cc099670 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f97cc09aeb0 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97e4094740 10 client.17994 fetch_fsmap learned FSMap version 62 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55668a530be0 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.899+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55668a48a860 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 62) ==== 8430+0+0 (secure 0 0 0) 0x7f97cc0979e0 con 0x55668a5175d0 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 10 client.17994 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T03:32:16.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 10 client.17994 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 20 client.17994 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a531f30 0x55668a534320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 4 client.17994 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55668a484d70 con 0x55668a531f30 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97daffd640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a531f30 0x55668a534320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97daffd640 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a531f30 0x55668a534320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3799182369cee93 server_cookie=33e1306bbadffbbc in_seq=0 out_seq=0 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.900+0000 7f97baffd640 10 client.17994 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97baffd640 1 -- 172.21.15.19:0/1762430459 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55668a484d70 con 0x55668a531f30 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97baffd640 10 client.17994 handle_command_reply: tid=0 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 client.17994 shutdown 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 msgr2=0x55668a521e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0975b0 tx=0x7f97cc0d2180 comp rx=0 tx=0).stop 2024-09-12T03:32:16.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 shutdown_connections 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a522370 0x55668a51f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a531f30 0x55668a534320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a518b40 0x55668a51f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 --2- 172.21.15.19:0/1762430459 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a521e20 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97baffd640 0 client.17994 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 >> 172.21.15.19:0/1762430459 conn(0x55668a4e0310 msgr2=0x55668a519960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 shutdown_connections 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 1 -- 172.21.15.19:0/1762430459 wait complete. 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.901+0000 7f97e4094740 20 client.17994 trim_cache size 0 max 16384 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 Processor -- start 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 -- start start 2024-09-12T03:32:16.972 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a518b40 0x55668a513700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a513c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1af840 con 0x55668a5175d0 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1b1530 con 0x55668a518b40 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97e4094740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1aff10 con 0x55668a522370 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57494/0 (socket says 172.21.15.19:57494) 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.915+0000 7f97db7fe640 1 -- 172.21.15.19:0/3237066933 learned_addr learned my addr 172.21.15.19:0/3237066933 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97daffd640 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a518b40 0x55668a513700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97dbfff640 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a513c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 -- 172.21.15.19:0/3237066933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 msgr2=0x55668a513c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a513c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 -- 172.21.15.19:0/3237066933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a518b40 msgr2=0x55668a513700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a518b40 0x55668a513700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 -- 172.21.15.19:0/3237066933 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55668a51fbd0 con 0x55668a5175d0 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f97cc097d40 tx=0x7f97cc09f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba96ae3d0aba6137 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97c17fa640 1 -- 172.21.15.19:0/3237066933 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc09acd0 con 0x55668a5175d0 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97c17fa640 1 -- 172.21.15.19:0/3237066933 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc09ba70 con 0x55668a5175d0 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.916+0000 7f97c17fa640 1 -- 172.21.15.19:0/3237066933 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0d2370 con 0x55668a5175d0 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- 172.21.15.19:0/3237066933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 msgr2=0x55668a51e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f97cc097d40 tx=0x7f97cc09f860 comp rx=0 tx=0).stop 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- 172.21.15.19:0/3237066933 shutdown_connections 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a513c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a518b40 0x55668a513700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 --2- 172.21.15.19:0/3237066933 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a5175d0 0x55668a51e990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- 172.21.15.19:0/3237066933 >> 172.21.15.19:0/3237066933 conn(0x55668a4e0310 msgr2=0x55668a51a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- 172.21.15.19:0/3237066933 shutdown_connections 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- 172.21.15.19:0/3237066933 wait complete. 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 -- start start 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.917+0000 7f97e4094740 1 Processor -- start 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a534810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1b0da0 con 0x55668a518b40 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1b0610 con 0x55668a5175d0 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc1ad760 con 0x55668a522370 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97e4094740 10 client.0 _ll_register_callbacks cb 0x7f97cc0cef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.975 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97dbfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a534810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57502/0 (socket says 172.21.15.19:57502) 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 -- 172.21.15.19:0/3981288114 learned_addr learned my addr 172.21.15.19:0/3981288114 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97db7fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:48660/0 (socket says 172.21.15.19:48660) 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 -- 172.21.15.19:0/3981288114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 msgr2=0x55668a534810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a534810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 -- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 msgr2=0x55668a533d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 -- 172.21.15.19:0/3981288114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55668a59a910 con 0x55668a518b40 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97daffd640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x55668a518070 tx=0x7f97d0014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c76ee1256350a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.918+0000 7f97baffd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:16.976 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0005020 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f97d00051c0 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97cc09ba70 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f97d00053c0 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 10 client.18000 fetch_fsmap learned FSMap version 62 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55668a59bc80 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97cc09ba70 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 62) ==== 8430+0+0 (secure 0 0 0) 0x7f97d001e3f0 con 0x55668a518b40 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 10 client.18000 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 10 client.18000 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 20 client.18000 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a59c4f0 0x55668a59e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 4 client.18000 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:32:16.977 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.919+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55668a401170 con 0x55668a59c4f0 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a59c4f0 0x55668a59e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97baffd640 10 client.18000 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97db7fe640 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a59c4f0 0x55668a59e910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf45e3542d8b1761 server_cookie=eb899eb18200778c in_seq=0 out_seq=0 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97baffd640 1 -- 172.21.15.19:0/3981288114 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55668a401170 con 0x55668a59c4f0 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97baffd640 10 client.18000 handle_command_reply: tid=0 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 client.18000 shutdown 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 msgr2=0x55668a5342d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x55668a518070 tx=0x7f97d0014840 comp rx=0 tx=0).stop 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 shutdown_connections 2024-09-12T03:32:16.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55668a59c4f0 0x55668a59e910 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55668a522370 0x55668a534810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55668a518b40 0x55668a5342d0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97baffd640 0 client.18000 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 --2- 172.21.15.19:0/3981288114 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55668a5175d0 0x55668a533d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 >> 172.21.15.19:0/3981288114 conn(0x55668a4e0310 msgr2=0x55668a5136a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 shutdown_connections 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.920+0000 7f97e4094740 1 -- 172.21.15.19:0/3981288114 wait complete. 2024-09-12T03:32:16.979 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e4094740 20 client.18000 trim_cache size 0 max 16384 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 msgr2=0x7f97c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 0x7f97c40a5040 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f97d4005fe0 tx=0x7f97d40095c0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 msgr2=0x7f97dc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00d950 tx=0x7f97cc00de10 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 shutdown_connections 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f97dc106f10 0x7f97dc19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f97dc102790 0x7f97dc19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f97c40a2b80 0x7f97c40a5040 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 --2- 172.21.15.19:0/4211694522 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f97dc101de0 0x7f97dc19a5f0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:16.980 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 >> 172.21.15.19:0/4211694522 conn(0x7f97dc0fd8c0 msgr2=0x7f97dc0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 shutdown_connections 2024-09-12T03:32:16.981 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:16.921+0000 7f97e1cbb640 1 -- 172.21.15.19:0/4211694522 wait complete. 2024-09-12T03:32:17.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:17 smithi135 ceph-mon[35474]: pgmap v734: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-12T03:32:17.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:17 smithi121 ceph-mon[34963]: pgmap v734: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-12T03:32:17.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:17 smithi019 ceph-mon[30953]: pgmap v734: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 405 op/s 2024-09-12T03:32:17.754 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:32:17.757 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:18.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 -- 172.21.15.19:0/3934588234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 msgr2=0x7f37f4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 --2- 172.21.15.19:0/3934588234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f4108830 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f37e4009be0 tx=0x7f37e402a6c0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 -- 172.21.15.19:0/3934588234 shutdown_connections 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 --2- 172.21.15.19:0/3934588234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f410d530 0x7f37f41150b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 --2- 172.21.15.19:0/3934588234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f4108e00 0x7f37f410ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 --2- 172.21.15.19:0/3934588234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f4108830 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 -- 172.21.15.19:0/3934588234 >> 172.21.15.19:0/3934588234 conn(0x7f37f4077ba0 msgr2=0x7f37f4079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 -- 172.21.15.19:0/3934588234 shutdown_connections 2024-09-12T03:32:18.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.307+0000 7f37fa465640 1 -- 172.21.15.19:0/3934588234 wait complete. 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.308+0000 7f37fa465640 1 Processor -- start 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.308+0000 7f37fa465640 1 -- start start 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f41a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 0x7f37f41a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 0x7f37f419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37f4118530 con 0x7f37f4108450 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37f8a02640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 0x7f37f419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f41a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 0x7f37f41a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37f8a02640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 0x7f37f419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55186/0 (socket says 172.21.15.19:55186) 2024-09-12T03:32:18.310 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37f8a02640 1 -- 172.21.15.19:0/1859779547 learned_addr learned my addr 172.21.15.19:0/1859779547 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41183b0 con 0x7f37f4108e00 2024-09-12T03:32:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.309+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41186b0 con 0x7f37f410d530 2024-09-12T03:32:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.310+0000 7f37f3fff640 1 -- 172.21.15.19:0/1859779547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 msgr2=0x7f37f419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.310+0000 7f37f3fff640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 0x7f37f419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.311 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.310+0000 7f37f3fff640 1 -- 172.21.15.19:0/1859779547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 msgr2=0x7f37f41a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.310+0000 7f37f3fff640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 0x7f37f41a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f3fff640 1 -- 172.21.15.19:0/1859779547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f419efd0 con 0x7f37f4108450 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f37fe640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 0x7f37f41a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f3fff640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f41a46e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f37e402abe0 tx=0x7f37e40305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f17fa640 1 -- 172.21.15.19:0/1859779547 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4041020 con 0x7f37f4108450 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f17fa640 1 -- 172.21.15.19:0/1859779547 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e4003e50 con 0x7f37f4108450 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.311+0000 7f37f17fa640 1 -- 172.21.15.19:0/1859779547 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4036040 con 0x7f37f4108450 2024-09-12T03:32:18.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.312+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f419f260 con 0x7f37f4108450 2024-09-12T03:32:18.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.312+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f419f6c0 con 0x7f37f4108450 2024-09-12T03:32:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.315+0000 7f37fc83e740 1 Processor -- start 2024-09-12T03:32:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.316+0000 7f37f17fa640 1 -- 172.21.15.19:0/1859779547 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37e4044020 con 0x7f37f4108450 2024-09-12T03:32:18.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.316+0000 7f37fc83e740 1 -- start start 2024-09-12T03:32:18.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a91b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870a9c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870a9ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37f8a02640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a91b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37f8a02640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a91b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55192/0 (socket says 172.21.15.19:55192) 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37f17fa640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 0x7f37e00a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.317+0000 7f37f8a02640 1 -- 172.21.15.19:0/3578981518 learned_addr learned my addr 172.21.15.19:0/3578981518 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f8709f72a0 con 0x55f870a9c940 2024-09-12T03:32:18.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37f17fa640 1 -- 172.21.15.19:0/1859779547 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f37e40ef4d0 con 0x7f37f4108450 2024-09-12T03:32:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f87099eb10 con 0x55f870a92080 2024-09-12T03:32:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f8709f3260 con 0x55f870a91760 2024-09-12T03:32:18.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37f3fff640 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870a9c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.318+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870a9ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 -- 172.21.15.19:0/3578981518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 msgr2=0x55f870a91b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a91b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 -- 172.21.15.19:0/3578981518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 msgr2=0x55f870a9c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870a9c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 -- 172.21.15.19:0/3578981518 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f870768b90 con 0x55f870a9c940 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.319+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870a9ed30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f37f41a0090 tx=0x7f37dc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99646cf7baccd093 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.320+0000 7f37f8a02640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 0x7f37e00a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.320+0000 7f37ea7fc640 1 -- 172.21.15.19:0/3578981518 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc004030 con 0x55f870a9c940 2024-09-12T03:32:18.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.320+0000 7f37ea7fc640 1 -- 172.21.15.19:0/3578981518 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37dc0041d0 con 0x55f870a9c940 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.321+0000 7f37ea7fc640 1 -- 172.21.15.19:0/3578981518 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00f9a0 con 0x55f870a9c940 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.321+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 msgr2=0x55f870a9ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.321+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870a9ed30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f37f41a0090 tx=0x7f37dc0118d0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37f8a02640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 0x7f37e00a5190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009640 tx=0x7f37ec004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 shutdown_connections 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870a9ed30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870a9c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3578981518 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a91b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 >> 172.21.15.19:0/3578981518 conn(0x55f870a8dcf0 msgr2=0x55f870a8e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 shutdown_connections 2024-09-12T03:32:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 -- 172.21.15.19:0/3578981518 wait complete. 2024-09-12T03:32:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.322+0000 7f37fc83e740 1 -- start start 2024-09-12T03:32:18.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.323+0000 7f37fc83e740 1 Processor -- start 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.323+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.323+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a92080 0x55f870a99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55202/0 (socket says 172.21.15.19:55202) 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 -- 172.21.15.19:0/3135293062 learned_addr learned my addr 172.21.15.19:0/3135293062 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870a9a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f870a9f5b0 con 0x55f870a92080 2024-09-12T03:32:18.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f87097f6e0 con 0x55f870a9c940 2024-09-12T03:32:18.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f8a02640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870a9a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f870a9f730 con 0x55f870a91760 2024-09-12T03:32:18.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 -- 172.21.15.19:0/3135293062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 msgr2=0x55f870a9a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870a9a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 -- 172.21.15.19:0/3135293062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a92080 msgr2=0x55f870a99d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a92080 0x55f870a99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.324+0000 7f37f3fff640 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f870aa8af0 con 0x55f870a91760 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37f3fff640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e40ed030 tx=0x7f37e40bb760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8801f96e5337bad9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37fc83e740 10 client.0 _ll_register_callbacks cb 0x55f8709a3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37d37fe640 10 client.27322 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37d37fe640 1 -- 172.21.15.19:0/3135293062 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e40c49b0 con 0x55f870a91760 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37dc0041d0 con 0x55f870a91760 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37d37fe640 1 -- 172.21.15.19:0/3135293062 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f37e40ef0d0 con 0x55f870a91760 2024-09-12T03:32:18.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37d37fe640 1 -- 172.21.15.19:0/3135293062 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f37e40ef2d0 con 0x55f870a91760 2024-09-12T03:32:18.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37fc83e740 10 client.27322 fetch_fsmap learned FSMap version 62 2024-09-12T03:32:18.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.325+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({fsmap=62}) -- 0x55f870a9f5b0 con 0x55f870a91760 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37d37fe640 1 -- 172.21.15.19:0/3135293062 <== mon.1 v2:172.21.15.135:3300/0 4 ==== fsmap(e 62) ==== 8430+0+0 (secure 0 0 0) 0x7f37e40c4520 con 0x55f870a91760 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 10 client.27322 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 10 client.27322 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 20 client.27322 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870aaec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 4 client.27322 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.326+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f37dc0041d0 con 0x55f870aac790 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.327+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870aaec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.328+0000 7f37d37fe640 10 client.27322 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:18.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.328+0000 7f37f37fe640 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870aaec50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0949a0b15e1fee server_cookie=6adf95d34e4dfddc in_seq=0 out_seq=0 2024-09-12T03:32:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.330+0000 7f37d37fe640 1 -- 172.21.15.19:0/3135293062 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f37dc0041d0 con 0x55f870aac790 2024-09-12T03:32:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.330+0000 7f37d37fe640 10 client.27322 handle_command_reply: tid=0 2024-09-12T03:32:18.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.330+0000 7f37fc83e740 1 client.27322 shutdown 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 msgr2=0x55f870a99810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f37e40ed030 tx=0x7f37e40bb760 comp rx=0 tx=0).stop 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 shutdown_connections 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870aaec50 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870a9a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37d37fe640 0 client.27322 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a92080 0x55f870a99d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 --2- 172.21.15.19:0/3135293062 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a99810 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 >> 172.21.15.19:0/3135293062 conn(0x55f870a8dcf0 msgr2=0x55f870a939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 shutdown_connections 2024-09-12T03:32:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.331+0000 7f37fc83e740 1 -- 172.21.15.19:0/3135293062 wait complete. 2024-09-12T03:32:18.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.332+0000 7f37fc83e740 20 client.27322 trim_cache size 0 max 16384 2024-09-12T03:32:18.349 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:18.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.362+0000 7f37fc83e740 1 Processor -- start 2024-09-12T03:32:18.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.363+0000 7f37fc83e740 1 -- start start 2024-09-12T03:32:18.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.363+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870aa7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870aa84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55218/0 (socket says 172.21.15.19:55218) 2024-09-12T03:32:18.365 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37f3fff640 1 -- 172.21.15.19:0/4293088320 learned_addr learned my addr 172.21.15.19:0/4293088320 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b3ae0 con 0x55f870a9c940 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.364+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b57d0 con 0x55f870a92080 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b41b0 con 0x55f870a91760 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f37fe640 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870aa7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f8a02640 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870aa84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 -- 172.21.15.19:0/4293088320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 msgr2=0x55f870aa7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870aa7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 -- 172.21.15.19:0/4293088320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 msgr2=0x55f870aa84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870aa84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.366 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 -- 172.21.15.19:0/4293088320 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f870a8db20 con 0x55f870a91760 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.365+0000 7f37f3fff640 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b8710 tx=0x7f37e40d2000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0663ec60fb0dd7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.366+0000 7f37e9ffb640 1 -- 172.21.15.19:0/4293088320 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e40c4520 con 0x55f870a91760 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.366+0000 7f37e9ffb640 1 -- 172.21.15.19:0/4293088320 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e40315d0 con 0x55f870a91760 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.366+0000 7f37e9ffb640 1 -- 172.21.15.19:0/4293088320 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4004e30 con 0x55f870a91760 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 msgr2=0x55f870a9a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f37e40b8710 tx=0x7f37e40d2000 comp rx=0 tx=0).stop 2024-09-12T03:32:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 shutdown_connections 2024-09-12T03:32:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a9c940 0x55f870aa84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a92080 0x55f870aa7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 --2- 172.21.15.19:0/4293088320 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a91760 0x55f870a9a0e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.367+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 >> 172.21.15.19:0/4293088320 conn(0x55f870a8dcf0 msgr2=0x55f870aa7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.368+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 shutdown_connections 2024-09-12T03:32:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.368+0000 7f37fc83e740 1 -- 172.21.15.19:0/4293088320 wait complete. 2024-09-12T03:32:18.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.368+0000 7f37fc83e740 1 -- start start 2024-09-12T03:32:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.369+0000 7f37fc83e740 1 Processor -- start 2024-09-12T03:32:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.370+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 0x55f870aa1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.370+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 0x55f870aa1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.370+0000 7f37fc83e740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870b0f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.370+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 0x55f870aa1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 0x55f870aa1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55224/0 (socket says 172.21.15.19:55224) 2024-09-12T03:32:18.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 0x55f870aa1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37f3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 0x55f870aa1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57548/0 (socket says 172.21.15.19:57548) 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37f37fe640 1 -- 172.21.15.19:0/2626184862 learned_addr learned my addr 172.21.15.19:0/2626184862 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37f8a02640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870b0f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b5040 con 0x55f870a91760 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b48b0 con 0x55f870a9c940 2024-09-12T03:32:18.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.371+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41b1a00 con 0x55f870a92080 2024-09-12T03:32:18.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37fc83e740 10 client.0 _ll_register_callbacks cb 0x7f37e40bb880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 -- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 msgr2=0x55f870aa1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 0x55f870aa1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 -- 172.21.15.19:0/2626184862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 msgr2=0x55f870aa1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 0x55f870aa1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f870a8e980 con 0x55f870a9c940 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37f8a02640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870b0f4f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37ec014fd0 tx=0x7f37ec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73e124e8b4f9a576 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec021850 con 0x55f870a9c940 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f37ec0219f0 con 0x55f870a9c940 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.372+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e40315d0 con 0x55f870a9c940 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f37ec021bf0 con 0x55f870a9c940 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 10 client.27350 fetch_fsmap learned FSMap version 62 2024-09-12T03:32:18.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55f870b17df0 con 0x55f870a9c940 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37ec021d90 con 0x55f870a9c940 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 62) ==== 8430+0+0 (secure 0 0 0) 0x7f37ec029070 con 0x55f870a9c940 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 10 client.27350 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 10 client.27350 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 20 client.27350 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870b1a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 4 client.27350 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}] 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.2, "await_for": 45.0}) -- 0x7f37e40315d0 con 0x55f870aac790 2024-09-12T03:32:18.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.373+0000 7f37f3fff640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870b1a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.374+0000 7f37f3fff640 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870b1a640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbd7351b6c70c48 server_cookie=cc91004d5515213a in_seq=0 out_seq=0 2024-09-12T03:32:18.376 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.374+0000 7f37d37fe640 10 client.27350 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 -- 172.21.15.19:0/3367771175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 msgr2=0x7f17d410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 --2- 172.21.15.19:0/3367771175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d410ec40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009cd0 tx=0x7f17cc02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 -- 172.21.15.19:0/3367771175 shutdown_connections 2024-09-12T03:32:18.858 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 --2- 172.21.15.19:0/3367771175 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d410ec40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 --2- 172.21.15.19:0/3367771175 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 0x7f17d4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 --2- 172.21.15.19:0/3367771175 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 0x7f17d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 -- 172.21.15.19:0/3367771175 >> 172.21.15.19:0/3367771175 conn(0x7f17d40fc7d0 msgr2=0x7f17d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 -- 172.21.15.19:0/3367771175 shutdown_connections 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 -- 172.21.15.19:0/3367771175 wait complete. 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.841+0000 7f17da9f8640 1 Processor -- start 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 -- start start 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 0x7f17d4198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 0x7f17d4198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41120a0 con 0x7f17d41082a0 2024-09-12T03:32:18.859 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4111f20 con 0x7f17d41036e0 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17da9f8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17d4112220 con 0x7f17d4102d30 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57564/0 (socket says 172.21.15.19:57564) 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 -- 172.21.15.19:0/1546570599 learned_addr learned my addr 172.21.15.19:0/1546570599 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 0x7f17d4198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 0x7f17d4198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 -- 172.21.15.19:0/1546570599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 msgr2=0x7f17d4198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 0x7f17d4198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 -- 172.21.15.19:0/1546570599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 msgr2=0x7f17d4198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 0x7f17d4198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 -- 172.21.15.19:0/1546570599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d419d4e0 con 0x7f17d41082a0 2024-09-12T03:32:18.860 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.842+0000 7f17d8f95640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005cc0 tx=0x7f17cc032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.843+0000 7f17d1ffb640 1 -- 172.21.15.19:0/1546570599 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc004040 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.843+0000 7f17d1ffb640 1 -- 172.21.15.19:0/1546570599 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17cc0046c0 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.843+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d419d770 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.843+0000 7f17d1ffb640 1 -- 172.21.15.19:0/1546570599 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17cc030e00 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.843+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d41a50b0 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.845+0000 7f17d1ffb640 1 -- 172.21.15.19:0/1546570599 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17cc0305a0 con 0x7f17d41082a0 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.845+0000 7f17dcdd1740 1 Processor -- start 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.845+0000 7f17dcdd1740 1 -- start start 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225a5d0 0x55d32225aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d3221c02f0 con 0x55d32225bb40 2024-09-12T03:32:18.861 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d322167cf0 con 0x55d322269780 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d3221bc2b0 con 0x55d32225a5d0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17d1ffb640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 0x7f17c00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57568/0 (socket says 172.21.15.19:57568) 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 -- 172.21.15.19:0/3832208021 learned_addr learned my addr 172.21.15.19:0/3832208021 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17d3fff640 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225a5d0 0x55d32225aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17d8f95640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 0x7f17c00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17d1ffb640 1 -- 172.21.15.19:0/1546570599 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17cc02e030 con 0x7f17d41082a0 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 -- 172.21.15.19:0/3832208021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225a5d0 msgr2=0x55d32225aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.862 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225a5d0 0x55d32225aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 -- 172.21.15.19:0/3832208021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 msgr2=0x55d32226bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 -- 172.21.15.19:0/3832208021 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d32214c3a0 con 0x55d32225bb40 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.846+0000 7f17cbfff640 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0058a0 tx=0x7f17bc00b740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b855f86146f67af5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17d8f95640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 0x7f17c00a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0bcd40 tx=0x7f17cc043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17ca7fc640 1 -- 172.21.15.19:0/3832208021 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004030 con 0x55d32225bb40 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17ca7fc640 1 -- 172.21.15.19:0/3832208021 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc005500 con 0x55d32225bb40 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17ca7fc640 1 -- 172.21.15.19:0/3832208021 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004d10 con 0x55d32225bb40 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 -- 172.21.15.19:0/3832208021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 msgr2=0x55d32225a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0058a0 tx=0x7f17bc00b740 comp rx=0 tx=0).stop 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 -- 172.21.15.19:0/3832208021 shutdown_connections 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.863 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225a5d0 0x55d32225aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/3832208021 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d32225a090 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 -- 172.21.15.19:0/3832208021 >> 172.21.15.19:0/3832208021 conn(0x55d322223310 msgr2=0x55d322256c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 -- 172.21.15.19:0/3832208021 shutdown_connections 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.847+0000 7f17dcdd1740 1 -- 172.21.15.19:0/3832208021 wait complete. 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 -- start start 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 Processor -- start 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322269140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d3222669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d32226c480 con 0x55d32225bb40 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d32217b060 con 0x55d32225a5d0 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d32226c600 con 0x55d322269780 2024-09-12T03:32:18.864 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17dcdd1740 10 client.0 _ll_register_callbacks cb 0x55d321ece8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57576/0 (socket says 172.21.15.19:57576) 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17d3fff640 1 -- 172.21.15.19:0/2393235484 learned_addr learned my addr 172.21.15.19:0/2393235484 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.848+0000 7f17cbfff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d3222669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17cbfff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322269140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 -- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 msgr2=0x55d3222669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d3222669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 -- 172.21.15.19:0/2393235484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 msgr2=0x55d322269140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322269140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 -- 172.21.15.19:0/2393235484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d321f313c0 con 0x55d32225bb40 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17d3fff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17c4012740 tx=0x7f17c4012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9704433b55a54826 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:18.865 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c400f920 con 0x55d32225bb40 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17bc005500 con 0x55d32225bb40 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17c400fac0 con 0x55d32225bb40 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f17c400fcc0 con 0x55d32225bb40 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17dcdd1740 10 client.18027 fetch_fsmap learned FSMap version 62 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55d322277f40 con 0x55d32225bb40 2024-09-12T03:32:18.866 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.849+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17bc005500 con 0x55d32225bb40 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 62) ==== 8430+0+0 (secure 0 0 0) 0x7f17c401c420 con 0x55d32225bb40 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 10 client.18027 fetch_fsmap finished waiting for FSMap version 62 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 10 client.18027 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 20 client.18027 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d32227b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 4 client.18027 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3221cd860 con 0x55d322279410 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17cbfff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d32227b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17cbfff640 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d32227b850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6f9b3e28b3b06bee server_cookie=a6b90d186333c924 in_seq=0 out_seq=0 2024-09-12T03:32:18.867 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:18.850+0000 7f17b37fe640 10 client.18027 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:32:19.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:19 smithi135 ceph-mon[35474]: pgmap v735: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T03:32:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:19 smithi121 ceph-mon[34963]: pgmap v735: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T03:32:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:19 smithi019 ceph-mon[30953]: pgmap v735: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 426 op/s 2024-09-12T03:32:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.395+0000 7f17b37fe640 1 -- 172.21.15.19:0/2393235484 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3221cd860 con 0x55d322279410 2024-09-12T03:32:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.395+0000 7f17b37fe640 10 client.18027 handle_command_reply: tid=0 2024-09-12T03:32:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.395+0000 7f17dcdd1740 1 client.18027 shutdown 2024-09-12T03:32:20.397 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 msgr2=0x55d3222663f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17c4012740 tx=0x7f17c4012c00 comp rx=0 tx=0).stop 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 shutdown_connections 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d32227b850 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d3222669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222663f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.398 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17b37fe640 0 client.18027 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:32:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/2393235484 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322269140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 >> 172.21.15.19:0/2393235484 conn(0x55d322223310 msgr2=0x55d32225c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 shutdown_connections 2024-09-12T03:32:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 1 -- 172.21.15.19:0/2393235484 wait complete. 2024-09-12T03:32:20.399 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.396+0000 7f17dcdd1740 20 client.18027 trim_cache size 0 max 16384 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 Processor -- start 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 -- start start 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322275720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d322281160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ab350 con 0x55d32225bb40 2024-09-12T03:32:20.412 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ad040 con 0x55d32225a5d0 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41aba20 con 0x55d322269780 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322275720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57590/0 (socket says 172.21.15.19:57590) 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.411+0000 7f17d3fff640 1 -- 172.21.15.19:0/1523833626 learned_addr learned my addr 172.21.15.19:0/1523833626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d322281160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 -- 172.21.15.19:0/1523833626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 msgr2=0x55d322281160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d322281160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 -- 172.21.15.19:0/1523833626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 msgr2=0x55d322275720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322275720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 -- 172.21.15.19:0/1523833626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d322264940 con 0x55d32225bb40 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322275720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:20.413 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f17c400f840 tx=0x7f17c4009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5233438dfea1110 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17c9ffb640 1 -- 172.21.15.19:0/1523833626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c402b070 con 0x55d32225bb40 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17c9ffb640 1 -- 172.21.15.19:0/1523833626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4022030 con 0x55d32225bb40 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.412+0000 7f17c9ffb640 1 -- 172.21.15.19:0/1523833626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c401cab0 con 0x55d32225bb40 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1523833626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 msgr2=0x55d3222653d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f17c400f840 tx=0x7f17c4009f00 comp rx=0 tx=0).stop 2024-09-12T03:32:20.416 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1523833626 shutdown_connections 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d322269780 0x55d322281160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225bb40 0x55d3222653d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1523833626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d32225a5d0 0x55d322275720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1523833626 >> 172.21.15.19:0/1523833626 conn(0x55d322223310 msgr2=0x55d322264cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1523833626 shutdown_connections 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.413+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1523833626 wait complete. 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.414+0000 7f17dcdd1740 1 -- start start 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.414+0000 7f17dcdd1740 1 Processor -- start 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.414+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225a5d0 0x55d322272090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.414+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17dcdd1740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ac8b0 con 0x55d32225a5d0 2024-09-12T03:32:20.417 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41ac120 con 0x55d322269780 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17dcdd1740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f17d41a9290 con 0x55d32225bb40 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17dcdd1740 10 client.0 _ll_register_callbacks cb 0x7f17c4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:55292/0 (socket says 172.21.15.19:55292) 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17cbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225a5d0 0x55d322272090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 -- 172.21.15.19:0/1312705067 learned_addr learned my addr 172.21.15.19:0/1312705067 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 -- 172.21.15.19:0/1312705067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 msgr2=0x55d32226d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 -- 172.21.15.19:0/1312705067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225a5d0 msgr2=0x55d322272090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225a5d0 0x55d322272090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 -- 172.21.15.19:0/1312705067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d32226e170 con 0x55d32225bb40 2024-09-12T03:32:20.418 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17d3fff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c4009240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9716d3b7daf2c1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.415+0000 7f17b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c4034060 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f17c401ca30 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17c4022030 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f17c402b070 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 10 client.27346 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d3222e41d0 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17c4022030 con 0x55d32225bb40 2024-09-12T03:32:20.419 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f17c4003b80 con 0x55d32225bb40 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 10 client.27346 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 10 client.27346 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 20 client.27346 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d3222e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 4 client.27346 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.416+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3221c7d70 con 0x55d322279410 2024-09-12T03:32:20.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.417+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d3222e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.417+0000 7f17cbfff640 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d3222e6940 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3efa6d25da3ec2e2 server_cookie=c4a8f43fcdd67311 in_seq=0 out_seq=0 2024-09-12T03:32:20.421 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.417+0000 7f17b37fe640 10 client.27346 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17b37fe640 1 -- 172.21.15.19:0/1312705067 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3221c7d70 con 0x55d322279410 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17b37fe640 10 client.27346 handle_command_reply: tid=0 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 client.27346 shutdown 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 msgr2=0x55d32226d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000c00 tx=0x7f17c4009240 comp rx=0 tx=0).stop 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 shutdown_connections 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55d322279410 0x55d3222e6940 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d322269780 0x55d32226d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d32225bb40 0x55d32226d3c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 --2- 172.21.15.19:0/1312705067 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d32225a5d0 0x55d322272090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17b37fe640 0 client.27346 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:32:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 >> 172.21.15.19:0/1312705067 conn(0x55d322223310 msgr2=0x55d322267c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 shutdown_connections 2024-09-12T03:32:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.467+0000 7f17dcdd1740 1 -- 172.21.15.19:0/1312705067 wait complete. 2024-09-12T03:32:20.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17dcdd1740 20 client.27346 trim_cache size 0 max 16384 2024-09-12T03:32:20.469 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 msgr2=0x7f17c00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 0x7f17c00a5040 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0bcd40 tx=0x7f17cc043000 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 msgr2=0x7f17d419ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f17cc005cc0 tx=0x7f17cc032f40 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 shutdown_connections 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f17c00a2b80 0x7f17c00a5040 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f17d41082a0 0x7f17d419ce00 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f17d41036e0 0x7f17d4198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.468+0000 7f17da9f8640 1 --2- 172.21.15.19:0/1546570599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f17d4102d30 0x7f17d4198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:20.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.469+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 >> 172.21.15.19:0/1546570599 conn(0x7f17d40fc7d0 msgr2=0x7f17d4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.469+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 shutdown_connections 2024-09-12T03:32:20.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:20.469+0000 7f17da9f8640 1 -- 172.21.15.19:0/1546570599 wait complete. 2024-09-12T03:32:20.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:20 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:20.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:20 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:20.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:20 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:32:21.523 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:21 smithi019 ceph-mon[30953]: pgmap v736: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:32:21.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:21 smithi135 ceph-mon[35474]: pgmap v736: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:32:21.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:32:21.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:21 smithi121 ceph-mon[34963]: pgmap v736: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 380 op/s 2024-09-12T03:32:21.825 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:22.460 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 -- 172.21.15.19:0/1507345081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 msgr2=0x7ff7501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 --2- 172.21.15.19:0/1507345081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff7501067e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7ff744009900 tx=0x7ff74402a410 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 -- 172.21.15.19:0/1507345081 shutdown_connections 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 --2- 172.21.15.19:0/1507345081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750106f10 0x7ff75010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 --2- 172.21.15.19:0/1507345081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff7501067e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 --2- 172.21.15.19:0/1507345081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750101de0 0x7ff7501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 -- 172.21.15.19:0/1507345081 >> 172.21.15.19:0/1507345081 conn(0x7ff7500fd8c0 msgr2=0x7ff7500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.444+0000 7ff754fe5640 1 -- 172.21.15.19:0/1507345081 shutdown_connections 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.445+0000 7ff754fe5640 1 -- 172.21.15.19:0/1507345081 wait complete. 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.445+0000 7ff754fe5640 1 Processor -- start 2024-09-12T03:32:22.461 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 -- start start 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750101de0 0x7ff75019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750106f10 0x7ff75019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff750111f70 con 0x7ff750102790 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff750111df0 con 0x7ff750101de0 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff754fe5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501120f0 con 0x7ff750106f10 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34238/0 (socket says 172.21.15.19:34238) 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 -- 172.21.15.19:0/790257536 learned_addr learned my addr 172.21.15.19:0/790257536 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750106f10 0x7ff75019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74e59c640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750101de0 0x7ff75019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.462 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 -- 172.21.15.19:0/790257536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750106f10 msgr2=0x7ff75019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750106f10 0x7ff75019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 -- 172.21.15.19:0/790257536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750101de0 msgr2=0x7ff75019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750101de0 0x7ff75019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 -- 172.21.15.19:0/790257536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75019f5f0 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.446+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff7440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.447+0000 7ff73f7fe640 1 -- 172.21.15.19:0/790257536 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74403f030 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.447+0000 7ff73f7fe640 1 -- 172.21.15.19:0/790257536 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff744003ce0 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.447+0000 7ff73f7fe640 1 -- 172.21.15.19:0/790257536 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744035070 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.447+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75019f880 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.447+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7501a71c0 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.449+0000 7ff73f7fe640 1 -- 172.21.15.19:0/790257536 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff744042030 con 0x7ff750102790 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.449+0000 7ff7573be740 1 Processor -- start 2024-09-12T03:32:22.463 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.449+0000 7ff7573be740 1 -- start start 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 0x565340f79a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565340edf2f0 con 0x565340f84370 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x565340e86cf0 con 0x565340f7ab40 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff7573be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565340edb2b0 con 0x565340f795d0 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34372/0 (socket says 172.21.15.19:34372) 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 -- 172.21.15.19:0/2207593372 learned_addr learned my addr 172.21.15.19:0/2207593372 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74e59c640 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 0x565340f79a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.464 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff73f7fe640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 0x7ff7380a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 -- 172.21.15.19:0/2207593372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 msgr2=0x565340f79a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 0x565340f79a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 -- 172.21.15.19:0/2207593372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 msgr2=0x565340f86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 -- 172.21.15.19:0/2207593372 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565340e6b3a0 con 0x565340f7ab40 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74e59c640 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 0x565340f79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff73f7fe640 1 -- 172.21.15.19:0/790257536 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff7440ef5c0 con 0x7ff750102790 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.450+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 0x7ff7380a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff748009fc0 tx=0x7ff748010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421fba26596e8b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff73cff9640 1 -- 172.21.15.19:0/2207593372 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74801c900 con 0x565340f7ab40 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 0x7ff7380a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff748020000 tx=0x7ff74801d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.465 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff73cff9640 1 -- 172.21.15.19:0/2207593372 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74801cb60 con 0x565340f7ab40 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff73cff9640 1 -- 172.21.15.19:0/2207593372 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7480113e0 con 0x565340f7ab40 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff7573be740 1 -- 172.21.15.19:0/2207593372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 msgr2=0x565340f79090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.451+0000 7ff7573be740 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff748009fc0 tx=0x7ff748010800 comp rx=0 tx=0).stop 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- 172.21.15.19:0/2207593372 shutdown_connections 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f795d0 0x565340f79a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- 172.21.15.19:0/2207593372 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f79090 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- 172.21.15.19:0/2207593372 >> 172.21.15.19:0/2207593372 conn(0x565340f42310 msgr2=0x565340f75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- 172.21.15.19:0/2207593372 shutdown_connections 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- 172.21.15.19:0/2207593372 wait complete. 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- start start 2024-09-12T03:32:22.466 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 Processor -- start 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f83d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x565340f87070 con 0x565340f7ab40 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x565340e9a060 con 0x565340f795d0 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x565340f871f0 con 0x565340f84370 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff7573be740 10 client.0 _ll_register_callbacks cb 0x565340bed8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff74dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.452+0000 7ff74dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34256/0 (socket says 172.21.15.19:34256) 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 -- 172.21.15.19:0/3895581090 learned_addr learned my addr 172.21.15.19:0/3895581090 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:22.467 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74e59c640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f83d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 -- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 msgr2=0x565340f81650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f81650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 -- 172.21.15.19:0/3895581090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 msgr2=0x565340f83d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f83d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 -- 172.21.15.19:0/3895581090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565340c503c0 con 0x565340f7ab40 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74e59c640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f83d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff744000c00 tx=0x7ff744005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e378fd5eaabdf92 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff744049070 con 0x565340f7ab40 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff7440bc150 con 0x565340f7ab40 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74801cb60 con 0x565340f7ab40 2024-09-12T03:32:22.468 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff7440bbcc0 con 0x565340f7ab40 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff7573be740 10 client.18045 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.453+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x565340f92b40 con 0x565340f7ab40 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff74801cb60 con 0x565340f7ab40 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7ff7440c46d0 con 0x565340f7ab40 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 10 client.18045 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 10 client.18045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 20 client.18045 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340f93ee0 0x565340f96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 4 client.18045 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565340eec860 con 0x565340f93ee0 2024-09-12T03:32:22.469 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.454+0000 7ff74e59c640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340f93ee0 0x565340f96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.455+0000 7ff74e59c640 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340f93ee0 0x565340f96340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5354d20ffe0444b3 server_cookie=22946b1371b25342 in_seq=0 out_seq=0 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.455+0000 7ff735ffb640 10 client.18045 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.455+0000 7ff735ffb640 1 -- 172.21.15.19:0/3895581090 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565340eec860 con 0x565340f93ee0 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.455+0000 7ff735ffb640 10 client.18045 handle_command_reply: tid=0 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 client.18045 shutdown 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 msgr2=0x565340f81040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7ff744000c00 tx=0x7ff744005b50 comp rx=0 tx=0).stop 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 shutdown_connections 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340f93ee0 0x565340f96340 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f81650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f7ab40 0x565340f81040 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.470 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff735ffb640 0 client.18045 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:32:22.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 --2- 172.21.15.19:0/3895581090 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f83d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 >> 172.21.15.19:0/3895581090 conn(0x565340f42310 msgr2=0x565340f7b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 shutdown_connections 2024-09-12T03:32:22.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 1 -- 172.21.15.19:0/3895581090 wait complete. 2024-09-12T03:32:22.471 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.456+0000 7ff7573be740 20 client.18045 trim_cache size 0 max 16384 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 Processor -- start 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 -- start start 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f90310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f7ab40 0x565340f82fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.476 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501af7e0 con 0x565340f84370 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501b1560 con 0x565340f795d0 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.469+0000 7ff7573be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501afeb0 con 0x565340f7ab40 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34270/0 (socket says 172.21.15.19:34270) 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 -- 172.21.15.19:0/145715836 learned_addr learned my addr 172.21.15.19:0/145715836 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f7ab40 0x565340f82fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74e59c640 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f90310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 -- 172.21.15.19:0/145715836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f7ab40 msgr2=0x565340f82fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f7ab40 0x565340f82fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.477 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 -- 172.21.15.19:0/145715836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 msgr2=0x565340f90310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f90310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 -- 172.21.15.19:0/145715836 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565340f76960 con 0x565340f84370 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff74801bbe0 tx=0x7ff74801bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b25e022baa4ba739 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff737fff640 1 -- 172.21.15.19:0/145715836 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748009b10 con 0x565340f84370 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff737fff640 1 -- 172.21.15.19:0/145715836 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff748035070 con 0x565340f84370 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.470+0000 7ff737fff640 1 -- 172.21.15.19:0/145715836 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74801cb60 con 0x565340f84370 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- 172.21.15.19:0/145715836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 msgr2=0x565340f9bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff74801bbe0 tx=0x7ff74801bc10 comp rx=0 tx=0).stop 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- 172.21.15.19:0/145715836 shutdown_connections 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f84370 0x565340f9bc10 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.478 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f7ab40 0x565340f82fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 --2- 172.21.15.19:0/145715836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f795d0 0x565340f90310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- 172.21.15.19:0/145715836 >> 172.21.15.19:0/145715836 conn(0x565340f42310 msgr2=0x565340f828a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- 172.21.15.19:0/145715836 shutdown_connections 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- 172.21.15.19:0/145715836 wait complete. 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.471+0000 7ff7573be740 1 -- start start 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 Processor -- start 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f88930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f88e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501b0dd0 con 0x565340f795d0 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501b0640 con 0x565340f7ab40 2024-09-12T03:32:22.479 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7ff7501ad700 con 0x565340f84370 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff7573be740 10 client.0 _ll_register_callbacks cb 0x7ff744005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34284/0 (socket says 172.21.15.19:34284) 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 -- 172.21.15.19:0/1451842164 learned_addr learned my addr 172.21.15.19:0/1451842164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f88930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f88e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 -- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 msgr2=0x565340f88e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f88e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 -- 172.21.15.19:0/1451842164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 msgr2=0x565340f88930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:22.480 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f88930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74e59c640 1 -- 172.21.15.19:0/1451842164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565340ffdde0 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74ed9d640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f88e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.472+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f88930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff74e59c640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff740004700 tx=0x7ff740012db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7597a39097d5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff740023030 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7ff74000fbc0 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff748035070 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7ff74000fdc0 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 10 client.18051 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x565340fff120 con 0x565340f795d0 2024-09-12T03:32:22.481 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff748035070 con 0x565340f795d0 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7ff74001f390 con 0x565340f795d0 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 10 client.18051 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 10 client.18051 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 20 client.18051 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340fff980 0x565341001da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 4 client.18051 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.473+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565340ee6d70 con 0x565340fff980 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340fff980 0x565341001da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff74dd9b640 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340fff980 0x565341001da0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=949ca7ed9f397722 server_cookie=c929f3689db2348a in_seq=0 out_seq=0 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff735ffb640 10 client.18051 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:32:22.482 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff735ffb640 1 -- 172.21.15.19:0/1451842164 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565340ee6d70 con 0x565340fff980 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff735ffb640 10 client.18051 handle_command_reply: tid=0 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 client.18051 shutdown 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 msgr2=0x565340f883f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff740004700 tx=0x7ff740012db0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 shutdown_connections 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x565340fff980 0x565341001da0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x565340f84370 0x565340f88e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x565340f7ab40 0x565340f88930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 --2- 172.21.15.19:0/1451842164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x565340f795d0 0x565340f883f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.474+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 >> 172.21.15.19:0/1451842164 conn(0x565340f42310 msgr2=0x565340f830c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff735ffb640 0 client.18051 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:32:22.483 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 shutdown_connections 2024-09-12T03:32:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff7573be740 1 -- 172.21.15.19:0/1451842164 wait complete. 2024-09-12T03:32:22.484 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff7573be740 20 client.18051 trim_cache size 0 max 16384 2024-09-12T03:32:22.484 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 msgr2=0x7ff7380a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 0x7ff7380a50c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff748020000 tx=0x7ff74801d000 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 msgr2=0x7ff75019ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff7440042f0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 shutdown_connections 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7ff750106f10 0x7ff75019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7ff7380a2c00 0x7ff7380a50c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7ff750102790 0x7ff75019ab80 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 --2- 172.21.15.19:0/790257536 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff750101de0 0x7ff75019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 >> 172.21.15.19:0/790257536 conn(0x7ff7500fd8c0 msgr2=0x7ff7500feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:22.485 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.475+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 shutdown_connections 2024-09-12T03:32:22.486 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:22.476+0000 7ff754fe5640 1 -- 172.21.15.19:0/790257536 wait complete. 2024-09-12T03:32:23.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:32:23.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:23 smithi019 ceph-mon[30953]: pgmap v737: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-09-12T03:32:23.441 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:23.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:23 smithi135 ceph-mon[35474]: pgmap v737: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-09-12T03:32:23.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37d37fe640 1 -- 172.21.15.19:0/2626184862 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f37e40315d0 con 0x55f870aac790 2024-09-12T03:32:23.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37d37fe640 10 client.27350 handle_command_reply: tid=0 2024-09-12T03:32:23.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 client.27350 shutdown 2024-09-12T03:32:23.619 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 msgr2=0x55f870b0f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870b0f4f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f37ec014fd0 tx=0x7f37ec0046c0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 shutdown_connections 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f870aac790 0x55f870b1a640 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f870a9c940 0x55f870b0f4f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f870a92080 0x55f870aa1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 --2- 172.21.15.19:0/2626184862 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f870a91760 0x55f870aa1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37d37fe640 0 client.27350 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.618+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 >> 172.21.15.19:0/2626184862 conn(0x55f870a8dcf0 msgr2=0x55f870aa8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 shutdown_connections 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fc83e740 1 -- 172.21.15.19:0/2626184862 wait complete. 2024-09-12T03:32:23.620 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fc83e740 20 client.27350 trim_cache size 0 max 16384 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 63, 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 46, 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "81c320fe": { 2024-09-12T03:32:23.621 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 46, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.2, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "QUIESCED", 2024-09-12T03:32:23.622 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:32:23.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 msgr2=0x7f37e00a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 0x7f37e00a5190 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009640 tx=0x7f37ec004730 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 msgr2=0x7f37f41a46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.619+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f41a46e0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f37e402abe0 tx=0x7f37e40305d0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 shutdown_connections 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f37f410d530 0x7f37f419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37f4108e00 0x7f37f41a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f37e00a2cd0 0x7f37e00a5190 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 --2- 172.21.15.19:0/1859779547 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f37f4108450 0x7f37f41a46e0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 >> 172.21.15.19:0/1859779547 conn(0x7f37f4077ba0 msgr2=0x7f37f4078ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:23.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 shutdown_connections 2024-09-12T03:32:23.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:23.620+0000 7f37fa465640 1 -- 172.21.15.19:0/1859779547 wait complete. 2024-09-12T03:32:23.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:23 smithi121 ceph-mon[34963]: pgmap v737: 97 pgs: 97 active+clean; 1.2 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-09-12T03:32:24.469 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '81c320fe', quiesce took 10.5 seconds. Will release after: 16.7 2024-09-12T03:32:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- 172.21.15.19:0/476912173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28102ef0 msgr2=0x7fed2810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 --2- 172.21.15.19:0/476912173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28102ef0 0x7fed2810da40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fed20009ca0 tx=0x7fed2002a950 comp rx=0 tx=0).stop 2024-09-12T03:32:24.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- 172.21.15.19:0/476912173 shutdown_connections 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 --2- 172.21.15.19:0/476912173 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28102ef0 0x7fed2810da40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 --2- 172.21.15.19:0/476912173 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102570 0x7fed281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 --2- 172.21.15.19:0/476912173 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28100e20 0x7fed28101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- 172.21.15.19:0/476912173 >> 172.21.15.19:0/476912173 conn(0x7fed280fc7d0 msgr2=0x7fed280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- 172.21.15.19:0/476912173 shutdown_connections 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- 172.21.15.19:0/476912173 wait complete. 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 Processor -- start 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.661+0000 7fed2e80a640 1 -- start start 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28102570 0x7fed28198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 0x7fed2819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.677 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed28111020 con 0x7fed28100e20 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed28110ea0 con 0x7fed28102570 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed281111a0 con 0x7fed28102ef0 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34306/0 (socket says 172.21.15.19:34306) 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 -- 172.21.15.19:0/1871709094 learned_addr learned my addr 172.21.15.19:0/1871709094 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2cda7640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 0x7fed2819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed1ffff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28102570 0x7fed28198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 -- 172.21.15.19:0/1871709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 msgr2=0x7fed2819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 0x7fed2819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 -- 172.21.15.19:0/1871709094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28102570 msgr2=0x7fed28198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28102570 0x7fed28198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 -- 172.21.15.19:0/1871709094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2819d590 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2cda7640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 0x7fed2819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed27fff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b4a0 tx=0x7fed1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed25ffb640 1 -- 172.21.15.19:0/1871709094 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18014070 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed25ffb640 1 -- 172.21.15.19:0/1871709094 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed18010af0 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2819d880 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.662+0000 7fed25ffb640 1 -- 172.21.15.19:0/1871709094 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed18017d10 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.663+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed281a5160 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.665+0000 7fed25ffb640 1 -- 172.21.15.19:0/1871709094 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fed18017070 con 0x7fed28100e20 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.665+0000 7fed30be3740 1 Processor -- start 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.665+0000 7fed30be3740 1 -- start start 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.665+0000 7fed30be3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c88090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.665+0000 7fed30be3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed30be3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed30be3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e27bee2f0 con 0x563e27c885d0 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed30be3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e27b95cf0 con 0x563e27c89b40 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed30be3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e27bea2b0 con 0x563e27c93370 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34318/0 (socket says 172.21.15.19:34318) 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 -- 172.21.15.19:0/411958846 learned_addr learned my addr 172.21.15.19:0/411958846 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed2cda7640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c88090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed1ffff640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 -- 172.21.15.19:0/411958846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 msgr2=0x563e27c95760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c95760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 -- 172.21.15.19:0/411958846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 msgr2=0x563e27c88090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c88090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed25ffb640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 0x7fed140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 -- 172.21.15.19:0/411958846 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e27b7a3a0 con 0x563e27c885d0 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed2cda7640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed1ffff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 0x7fed140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed25ffb640 1 -- 172.21.15.19:0/1871709094 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed180ce4b0 con 0x7fed28100e20 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed27fff640 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fed1809aa20 tx=0x7fed180d49f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfa9a49dd684d05 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.666+0000 7fed1e7fc640 1 -- 172.21.15.19:0/411958846 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1801e070 con 0x563e27c885d0 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed1e7fc640 1 -- 172.21.15.19:0/411958846 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1809b420 con 0x563e27c885d0 2024-09-12T03:32:24.681 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed1e7fc640 1 -- 172.21.15.19:0/411958846 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1809a860 con 0x563e27c885d0 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 -- 172.21.15.19:0/411958846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 msgr2=0x563e27c88a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed1ffff640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 0x7fed140a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x563e27c89070 tx=0x7fed10008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fed1809aa20 tx=0x7fed180d49f0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 -- 172.21.15.19:0/411958846 shutdown_connections 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c95760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c885d0 0x563e27c88a10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 --2- 172.21.15.19:0/411958846 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c88090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 -- 172.21.15.19:0/411958846 >> 172.21.15.19:0/411958846 conn(0x563e27c51310 msgr2=0x563e27c84c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 -- 172.21.15.19:0/411958846 shutdown_connections 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.667+0000 7fed30be3740 1 -- 172.21.15.19:0/411958846 wait complete. 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 -- start start 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 Processor -- start 2024-09-12T03:32:24.682 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c92dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c93370 0x563e27c90810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x563e27c96070 con 0x563e27c89b40 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x563e27ba9060 con 0x563e27c93370 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x563e27c961f0 con 0x563e27c885d0 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed30be3740 10 client.0 _ll_register_callbacks cb 0x563e278fc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34334/0 (socket says 172.21.15.19:34334) 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed1ffff640 1 -- 172.21.15.19:0/111053758 learned_addr learned my addr 172.21.15.19:0/111053758 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.668+0000 7fed27fff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c92dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed2cda7640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c93370 0x563e27c90810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 -- 172.21.15.19:0/111053758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 msgr2=0x563e27c92dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c92dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 -- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c93370 msgr2=0x563e27c90810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c93370 0x563e27c90810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 -- 172.21.15.19:0/111053758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e2795f3c0 con 0x563e27c89b40 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed1ffff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fed10014710 tx=0x7fed100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54494e71e089a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed077fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1001b050 con 0x563e27c89b40 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed10015820 con 0x563e27c89b40 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1809b420 con 0x563e27c89b40 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fed10015a20 con 0x563e27c89b40 2024-09-12T03:32:24.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed30be3740 10 client.18063 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.669+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563e27ca1c20 con 0x563e27c89b40 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1809b420 con 0x563e27c89b40 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fed10020440 con 0x563e27c89b40 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 10 client.18063 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 10 client.18063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 20 client.18063 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27ca31a0 0x563e27ca55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 4 client.18063 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e27bfb860 con 0x563e27ca31a0 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.670+0000 7fed27fff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27ca31a0 0x563e27ca55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.671+0000 7fed077fe640 10 client.18063 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.671+0000 7fed27fff640 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27ca31a0 0x563e27ca55e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=12719f02793692cb server_cookie=6afdbbdec47cdfce in_seq=0 out_seq=0 2024-09-12T03:32:24.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.671+0000 7fed077fe640 1 -- 172.21.15.19:0/111053758 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563e27bfb860 con 0x563e27ca31a0 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.671+0000 7fed077fe640 10 client.18063 handle_command_reply: tid=0 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.671+0000 7fed30be3740 1 client.18063 shutdown 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 msgr2=0x563e27c90080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fed10014710 tx=0x7fed100046c0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 shutdown_connections 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27ca31a0 0x563e27ca55e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c93370 0x563e27c90810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c90080 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed077fe640 0 client.18063 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 --2- 172.21.15.19:0/111053758 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c92dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 >> 172.21.15.19:0/111053758 conn(0x563e27c51310 msgr2=0x563e27c8a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 shutdown_connections 2024-09-12T03:32:24.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 1 -- 172.21.15.19:0/111053758 wait complete. 2024-09-12T03:32:24.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.672+0000 7fed30be3740 20 client.18063 trim_cache size 0 max 16384 2024-09-12T03:32:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 Processor -- start 2024-09-12T03:32:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 -- start start 2024-09-12T03:32:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c89300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed281ad6f0 con 0x563e27c93370 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed281af470 con 0x563e27c89b40 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed30be3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed281addc0 con 0x563e27c885d0 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34340/0 (socket says 172.21.15.19:34340) 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 -- 172.21.15.19:0/2755008291 learned_addr learned my addr 172.21.15.19:0/2755008291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed1ffff640 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c89300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 -- 172.21.15.19:0/2755008291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 msgr2=0x563e27c84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 -- 172.21.15.19:0/2755008291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 msgr2=0x563e27c89300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c89300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.685+0000 7fed2cda7640 1 -- 172.21.15.19:0/2755008291 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e27c8b120 con 0x563e27c93370 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed2cda7640 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fed2002f9e0 tx=0x7fed20005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1dbc6c1003f8e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed1dffb640 1 -- 172.21.15.19:0/2755008291 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20034040 con 0x563e27c93370 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed1dffb640 1 -- 172.21.15.19:0/2755008291 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2004a070 con 0x563e27c93370 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed1dffb640 1 -- 172.21.15.19:0/2755008291 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed20004500 con 0x563e27c93370 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 -- 172.21.15.19:0/2755008291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 msgr2=0x563e27c91ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fed2002f9e0 tx=0x7fed20005c30 comp rx=0 tx=0).stop 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 -- 172.21.15.19:0/2755008291 shutdown_connections 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c93370 0x563e27c91ff0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c89b40 0x563e27c89300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 --2- 172.21.15.19:0/2755008291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c885d0 0x563e27c84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 -- 172.21.15.19:0/2755008291 >> 172.21.15.19:0/2755008291 conn(0x563e27c51310 msgr2=0x563e27c918e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 -- 172.21.15.19:0/2755008291 shutdown_connections 2024-09-12T03:32:24.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.686+0000 7fed30be3740 1 -- 172.21.15.19:0/2755008291 wait complete. 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 -- start start 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 Processor -- start 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 0x563e27c9b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c8cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fed281aece0 con 0x563e27c89b40 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed281ae550 con 0x563e27c885d0 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fed281ab630 con 0x563e27c93370 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed30be3740 10 client.0 _ll_register_callbacks cb 0x7fed1000b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed1ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34346/0 (socket says 172.21.15.19:34346) 2024-09-12T03:32:24.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.687+0000 7fed1ffff640 1 -- 172.21.15.19:0/175676204 learned_addr learned my addr 172.21.15.19:0/175676204 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed2cda7640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c8cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed27fff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 0x563e27c9b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 -- 172.21.15.19:0/175676204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 msgr2=0x563e27c8cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c8cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 -- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 msgr2=0x563e27c9b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 0x563e27c9b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 -- 172.21.15.19:0/175676204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e27c8d4d0 con 0x563e27c89b40 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed27fff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 0x563e27c9b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed1ffff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fed10000c00 tx=0x7fed10015320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1ee154783d2883 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed077fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1001b040 con 0x563e27c89b40 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fed10015620 con 0x563e27c89b40 2024-09-12T03:32:24.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed2004a070 con 0x563e27c89b40 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fed10020650 con 0x563e27c89b40 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed30be3740 10 client.18069 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.688+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x563e27d0db10 con 0x563e27c89b40 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed2004a070 con 0x563e27c89b40 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fed10005cc0 con 0x563e27c89b40 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 10 client.18069 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 10 client.18069 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 20 client.18069 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27d0e340 0x563e27d10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 4 client.18069 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e27bf5d70 con 0x563e27d0e340 2024-09-12T03:32:24.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.689+0000 7fed27fff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27d0e340 0x563e27d10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed27fff640 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27d0e340 0x563e27d10760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e4e8c1f146a175a server_cookie=a9fe41da3f345293 in_seq=0 out_seq=0 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed077fe640 10 client.18069 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed077fe640 1 -- 172.21.15.19:0/175676204 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e27bf5d70 con 0x563e27d0e340 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed077fe640 10 client.18069 handle_command_reply: tid=0 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 client.18069 shutdown 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 msgr2=0x563e27c9bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fed10000c00 tx=0x7fed10015320 comp rx=0 tx=0).stop 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 shutdown_connections 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x563e27d0e340 0x563e27d10760 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x563e27c93370 0x563e27c8cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x563e27c89b40 0x563e27c9bd60 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 --2- 172.21.15.19:0/175676204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x563e27c885d0 0x563e27c9b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 >> 172.21.15.19:0/175676204 conn(0x563e27c51310 msgr2=0x563e27c8a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed077fe640 0 client.18069 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:32:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 shutdown_connections 2024-09-12T03:32:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 1 -- 172.21.15.19:0/175676204 wait complete. 2024-09-12T03:32:24.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.690+0000 7fed30be3740 20 client.18069 trim_cache size 0 max 16384 2024-09-12T03:32:24.698 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 msgr2=0x7fed140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 0x7fed140a5040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x563e27c89070 tx=0x7fed10008040 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 msgr2=0x7fed281985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b4a0 tx=0x7fed1800b960 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 shutdown_connections 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fed28102ef0 0x7fed2819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed28102570 0x7fed28198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fed140a2b80 0x7fed140a5040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 --2- 172.21.15.19:0/1871709094 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fed28100e20 0x7fed281985e0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.691+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 >> 172.21.15.19:0/1871709094 conn(0x7fed280fc7d0 msgr2=0x7fed2806a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.692+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 shutdown_connections 2024-09-12T03:32:24.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:24.692+0000 7fed2e80a640 1 -- 172.21.15.19:0/1871709094 wait complete. 2024-09-12T03:32:25.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:25 smithi019 ceph-mon[30953]: pgmap v738: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2024-09-12T03:32:25.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:32:25.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:25 smithi135 ceph-mon[35474]: pgmap v738: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2024-09-12T03:32:25.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:25 smithi121 ceph-mon[34963]: pgmap v738: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 344 op/s 2024-09-12T03:32:25.767 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:26.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 -- 172.21.15.19:0/1720817599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 msgr2=0x7f5aac1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 --2- 172.21.15.19:0/1720817599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac1067e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009900 tx=0x7f5aa002a410 comp rx=0 tx=0).stop 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 -- 172.21.15.19:0/1720817599 shutdown_connections 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 --2- 172.21.15.19:0/1720817599 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 0x7f5aac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 --2- 172.21.15.19:0/1720817599 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac1067e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 --2- 172.21.15.19:0/1720817599 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 0x7f5aac1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 -- 172.21.15.19:0/1720817599 >> 172.21.15.19:0/1720817599 conn(0x7f5aac0fd8c0 msgr2=0x7f5aac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 -- 172.21.15.19:0/1720817599 shutdown_connections 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.270+0000 7f5ab327d640 1 -- 172.21.15.19:0/1720817599 wait complete. 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 Processor -- start 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 -- start start 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 0x7f5aac1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.297 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 0x7f5aac19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112010 con 0x7f5aac102790 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac111e90 con 0x7f5aac106f10 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab327d640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac112190 con 0x7f5aac101de0 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.271+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34378/0 (socket says 172.21.15.19:34378) 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 -- 172.21.15.19:0/3393241815 learned_addr learned my addr 172.21.15.19:0/3393241815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab1019640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 0x7f5aac1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab181a640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 0x7f5aac19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 -- 172.21.15.19:0/3393241815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 msgr2=0x7f5aac1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 0x7f5aac1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 -- 172.21.15.19:0/3393241815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 msgr2=0x7f5aac19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 0x7f5aac19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 -- 172.21.15.19:0/3393241815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac19d560 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab0818640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005ab0 tx=0x7f5aa00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5a9e7fc640 1 -- 172.21.15.19:0/3393241815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa003f030 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5a9e7fc640 1 -- 172.21.15.19:0/3393241815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa0003ce0 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac19d7f0 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5a9e7fc640 1 -- 172.21.15.19:0/3393241815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0035070 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.272+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1a5130 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5a9e7fc640 1 -- 172.21.15.19:0/3393241815 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5aa0042030 con 0x7f5aac102790 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 Processor -- start 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 -- start start 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 0x55669e09e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55669e0042f0 con 0x55669e09e5d0 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55669dfabcf0 con 0x55669e0ad780 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab5656740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55669e0002b0 con 0x55669e09fb40 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab1019640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab1019640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34382/0 (socket says 172.21.15.19:34382) 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.275+0000 7f5ab1019640 1 -- 172.21.15.19:0/3629184400 learned_addr learned my addr 172.21.15.19:0/3629184400 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5a9e7fc640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 0x7f5a980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab0818640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab181a640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 0x55669e09e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.300 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 -- 172.21.15.19:0/3629184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 msgr2=0x55669e09e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 0x55669e09e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 -- 172.21.15.19:0/3629184400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 msgr2=0x55669e0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 -- 172.21.15.19:0/3629184400 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669df903a0 con 0x55669e09e5d0 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab181a640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 0x55669e09e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab181a640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 0x7f5a980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5a9e7fc640 1 -- 172.21.15.19:0/3393241815 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5aa00ef580 con 0x7f5aac102790 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5ab1019640 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b820 tx=0x7f5aa800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7976f19afa0734f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5a977fe640 1 -- 172.21.15.19:0/3629184400 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800d950 con 0x55669e09e5d0 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5a977fe640 1 -- 172.21.15.19:0/3629184400 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8019070 con 0x55669e09e5d0 2024-09-12T03:32:26.301 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.276+0000 7f5a977fe640 1 -- 172.21.15.19:0/3629184400 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80123e0 con 0x55669e09e5d0 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- 172.21.15.19:0/3629184400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 msgr2=0x55669e09ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab181a640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 0x7f5a980a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006a50 tx=0x7f5aa4004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800b820 tx=0x7f5aa800bce0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- 172.21.15.19:0/3629184400 shutdown_connections 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09e5d0 0x55669e09ea10 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 --2- 172.21.15.19:0/3629184400 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09fb40 0x55669e09e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- 172.21.15.19:0/3629184400 >> 172.21.15.19:0/3629184400 conn(0x55669e067310 msgr2=0x55669e09ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- 172.21.15.19:0/3629184400 shutdown_connections 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- 172.21.15.19:0/3629184400 wait complete. 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 -- start start 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.277+0000 7f5ab5656740 1 Processor -- start 2024-09-12T03:32:26.302 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09e5d0 0x55669e0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e0ad780 0x55669e0aa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55669e0b0480 con 0x55669e09fb40 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55669dfbf060 con 0x55669e09e5d0 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55669e0b0600 con 0x55669e0ad780 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab5656740 10 client.0 _ll_register_callbacks cb 0x55669dd128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34396/0 (socket says 172.21.15.19:34396) 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 -- 172.21.15.19:0/2119354397 learned_addr learned my addr 172.21.15.19:0/2119354397 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab1019640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09e5d0 0x55669e0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab181a640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e0ad780 0x55669e0aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 -- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e0ad780 msgr2=0x55669e0aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.303 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e0ad780 0x55669e0aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 -- 172.21.15.19:0/2119354397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09e5d0 msgr2=0x55669e0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09e5d0 0x55669e0ad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.278+0000 7f5ab0818640 1 -- 172.21.15.19:0/2119354397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55669dd753c0 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab0818640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006010 tx=0x7f5aa002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517eef450322d323 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00bf060 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5aa003bd70 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa8019070 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f5aa0049040 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 10 client.18081 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55669e0bbe20 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa8019070 con 0x55669e09fb40 2024-09-12T03:32:26.304 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f5aa00b8460 con 0x55669e09fb40 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 10 client.18081 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 20 client.18081 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e0bd280 0x55669e0bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 4 client.18081 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.279+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55669e011860 con 0x55669e0bd280 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.280+0000 7f5ab1019640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e0bd280 0x55669e0bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.280+0000 7f5a94ff9640 10 client.18081 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.280+0000 7f5ab1019640 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e0bd280 0x55669e0bf6a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9179ee077621b4 server_cookie=afa4205541ffda92 in_seq=0 out_seq=0 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5a94ff9640 1 -- 172.21.15.19:0/2119354397 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55669e011860 con 0x55669e0bd280 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5a94ff9640 10 client.18081 handle_command_reply: tid=0 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 client.18081 shutdown 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 msgr2=0x55669e0aa3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.305 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0006010 tx=0x7f5aa002a410 comp rx=0 tx=0).stop 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 shutdown_connections 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e0ad780 0x55669e0aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0aa3c0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e0bd280 0x55669e0bf6a0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 --2- 172.21.15.19:0/2119354397 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09e5d0 0x55669e0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5a94ff9640 0 client.18081 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 >> 172.21.15.19:0/2119354397 conn(0x55669e067310 msgr2=0x55669e0a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 shutdown_connections 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 1 -- 172.21.15.19:0/2119354397 wait complete. 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.281+0000 7f5ab5656740 20 client.18081 trim_cache size 0 max 16384 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 Processor -- start 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 -- start start 2024-09-12T03:32:26.306 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09fb40 0x55669e0a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1ab470 con 0x55669e0ad780 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1ad1f0 con 0x55669e09fb40 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.294+0000 7f5ab5656740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1abb40 con 0x55669e09e5d0 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34408/0 (socket says 172.21.15.19:34408) 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 -- 172.21.15.19:0/982802514 learned_addr learned my addr 172.21.15.19:0/982802514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab1019640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab0818640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09fb40 0x55669e0a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 -- 172.21.15.19:0/982802514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 msgr2=0x55669e0b9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 -- 172.21.15.19:0/982802514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09fb40 msgr2=0x55669e0a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.307 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09fb40 0x55669e0a9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 -- 172.21.15.19:0/982802514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669e0a8910 con 0x55669e0ad780 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab1019640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5ab181a640 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006490 tx=0x7f5aa4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a1adfc46c7ed23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5a96ffd640 1 -- 172.21.15.19:0/982802514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4015110 con 0x55669e0ad780 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5a96ffd640 1 -- 172.21.15.19:0/982802514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa40152b0 con 0x55669e0ad780 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.295+0000 7f5a96ffd640 1 -- 172.21.15.19:0/982802514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa40218d0 con 0x55669e0ad780 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- 172.21.15.19:0/982802514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 msgr2=0x55669e0aeb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006490 tx=0x7f5aa4012000 comp rx=0 tx=0).stop 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- 172.21.15.19:0/982802514 shutdown_connections 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e0ad780 0x55669e0aeb70 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e09fb40 0x55669e0a9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 --2- 172.21.15.19:0/982802514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.308 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- 172.21.15.19:0/982802514 >> 172.21.15.19:0/982802514 conn(0x55669e067310 msgr2=0x55669e0a5720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- 172.21.15.19:0/982802514 shutdown_connections 2024-09-12T03:32:26.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- 172.21.15.19:0/982802514 wait complete. 2024-09-12T03:32:26.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.296+0000 7f5ab5656740 1 -- start start 2024-09-12T03:32:26.309 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 Processor -- start 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1aca60 con 0x55669e09fb40 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1ac2d0 con 0x55669e0ad780 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac1a9390 con 0x55669e09e5d0 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab5656740 10 client.0 _ll_register_callbacks cb 0x7f5aa004bfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34410/0 (socket says 172.21.15.19:34410) 2024-09-12T03:32:26.312 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 -- 172.21.15.19:0/1956312127 learned_addr learned my addr 172.21.15.19:0/1956312127 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab1019640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab181a640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 -- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 msgr2=0x55669e0b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 -- 172.21.15.19:0/1956312127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 msgr2=0x55669e0c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab0818640 1 -- 172.21.15.19:0/1956312127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55669e0c4d70 con 0x55669e09fb40 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.297+0000 7f5ab181a640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab0818640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00ba0a0 tx=0x7f5aa0005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df2a353f1b7e4b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa00bf050 con 0x55669e09fb40 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5aa00c45d0 con 0x55669e09fb40 2024-09-12T03:32:26.313 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa40152b0 con 0x55669e09fb40 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f5aa00bb750 con 0x55669e09fb40 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 10 client.18087 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55669e0b5370 con 0x55669e09fb40 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5aa40152b0 con 0x55669e09fb40 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f5aa0031ba0 con 0x55669e09fb40 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 10 client.18087 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 10 client.18087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 20 client.18087 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e128050 0x55669e12a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 4 client.18087 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.298+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55669e00bd70 con 0x55669e128050 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab1019640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e128050 0x55669e12a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:26.314 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab1019640 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e128050 0x55669e12a470 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f506b704439c0246 server_cookie=1c8d2ff5805adc67 in_seq=0 out_seq=0 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5a94ff9640 10 client.18087 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5a94ff9640 1 -- 172.21.15.19:0/1956312127 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55669e00bd70 con 0x55669e128050 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5a94ff9640 10 client.18087 handle_command_reply: tid=0 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 client.18087 shutdown 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 msgr2=0x55669e0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00ba0a0 tx=0x7f5aa0005be0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 shutdown_connections 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55669e128050 0x55669e12a470 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55669e0ad780 0x55669e0c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55669e09fb40 0x55669e0b6070 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.315 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5a94ff9640 0 client.18087 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 --2- 172.21.15.19:0/1956312127 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55669e09e5d0 0x55669e0b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.299+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 >> 172.21.15.19:0/1956312127 conn(0x55669e067310 msgr2=0x55669e0a94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 shutdown_connections 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab5656740 1 -- 172.21.15.19:0/1956312127 wait complete. 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab5656740 20 client.18087 trim_cache size 0 max 16384 2024-09-12T03:32:26.316 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 msgr2=0x7f5a980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 0x7f5a980a50c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4006a50 tx=0x7f5aa4004b50 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 msgr2=0x7f5aac198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0005ab0 tx=0x7f5aa00042f0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 shutdown_connections 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5aac106f10 0x7f5aac19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f5a980a2c00 0x7f5a980a50c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f5aac102790 0x7f5aac198af0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 --2- 172.21.15.19:0/3393241815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f5aac101de0 0x7f5aac1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:26.317 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 >> 172.21.15.19:0/3393241815 conn(0x7f5aac0fd8c0 msgr2=0x7f5aac0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:26.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 shutdown_connections 2024-09-12T03:32:26.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:26.300+0000 7f5ab327d640 1 -- 172.21.15.19:0/3393241815 wait complete. 2024-09-12T03:32:27.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:27 smithi019 ceph-mon[30953]: pgmap v739: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 924 KiB/s wr, 285 op/s 2024-09-12T03:32:27.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:27 smithi135 ceph-mon[35474]: pgmap v739: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 924 KiB/s wr, 285 op/s 2024-09-12T03:32:27.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:27 smithi121 ceph-mon[34963]: pgmap v739: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 923 KiB/s rd, 924 KiB/s wr, 285 op/s 2024-09-12T03:32:28.111 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:32:28.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:32:28.384 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 -- 172.21.15.19:0/2300437497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c102570 msgr2=0x7f6d7c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 --2- 172.21.15.19:0/2300437497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c102570 0x7f6d7c1029b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f6d70009ca0 tx=0x7f6d7002a950 comp rx=0 tx=0).stop 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 -- 172.21.15.19:0/2300437497 shutdown_connections 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 --2- 172.21.15.19:0/2300437497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 --2- 172.21.15.19:0/2300437497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c102570 0x7f6d7c1029b0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 --2- 172.21.15.19:0/2300437497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c100e20 0x7f6d7c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 -- 172.21.15.19:0/2300437497 >> 172.21.15.19:0/2300437497 conn(0x7f6d7c0fc7d0 msgr2=0x7f6d7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 -- 172.21.15.19:0/2300437497 shutdown_connections 2024-09-12T03:32:28.898 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.884+0000 7f6d8357a640 1 -- 172.21.15.19:0/2300437497 wait complete. 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 Processor -- start 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 -- start start 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102570 0x7f6d7c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1120a0 con 0x7f6d7c100e20 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c111f20 con 0x7f6d7c102ef0 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d8357a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c112220 con 0x7f6d7c102570 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34428/0 (socket says 172.21.15.19:34428) 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d81316640 1 -- 172.21.15.19:0/3433848128 learned_addr learned my addr 172.21.15.19:0/3433848128 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d80b15640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102570 0x7f6d7c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.899 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.885+0000 7f6d81b17640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 -- 172.21.15.19:0/3433848128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102570 msgr2=0x7f6d7c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102570 0x7f6d7c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 -- 172.21.15.19:0/3433848128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 msgr2=0x7f6d7c19cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c19cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 -- 172.21.15.19:0/3433848128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7c19d3f0 con 0x7f6d7c100e20 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81b17640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d81316640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d950 tx=0x7f6d6c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d7a7fc640 1 -- 172.21.15.19:0/3433848128 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c017070 con 0x7f6d7c100e20 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d7a7fc640 1 -- 172.21.15.19:0/3433848128 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c009c30 con 0x7f6d7c100e20 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7c19d6e0 con 0x7f6d7c100e20 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d7a7fc640 1 -- 172.21.15.19:0/3433848128 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c00f6b0 con 0x7f6d7c100e20 2024-09-12T03:32:28.900 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.886+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d7c1a4fc0 con 0x7f6d7c100e20 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d7a7fc640 1 -- 172.21.15.19:0/3433848128 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d6c0109d0 con 0x7f6d7c100e20 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 Processor -- start 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 -- start start 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 0x55f602203f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f602211130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f602169280 con 0x55f602204480 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f602110b10 con 0x55f60220ed40 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d85953740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f602165240 con 0x55f602203b60 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34442/0 (socket says 172.21.15.19:34442) 2024-09-12T03:32:28.901 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d81316640 1 -- 172.21.15.19:0/1393745690 learned_addr learned my addr 172.21.15.19:0/1393745690 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d81b17640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 0x55f602203f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d80b15640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f602211130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.889+0000 7f6d7a7fc640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 0x7f6d640a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 0x7f6d640a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 -- 172.21.15.19:0/1393745690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 msgr2=0x55f602211130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f602211130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d7a7fc640 1 -- 172.21.15.19:0/3433848128 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d6c0ced30 con 0x7f6d7c100e20 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 -- 172.21.15.19:0/1393745690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 msgr2=0x55f60220e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 -- 172.21.15.19:0/1393745690 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f60211d1f0 con 0x55f602203b60 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81316640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:28.902 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d80b15640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f602211130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 0x55f602203f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d740127b0 tx=0x7f6d74012c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a988132923263ff0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d81b17640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 0x7f6d640a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6d7401a000 tx=0x7f6d7401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d637fe640 1 -- 172.21.15.19:0/1393745690 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74005230 con 0x55f602203b60 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d637fe640 1 -- 172.21.15.19:0/1393745690 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74004c80 con 0x55f602203b60 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.890+0000 7f6d637fe640 1 -- 172.21.15.19:0/1393745690 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7401c960 con 0x55f602203b60 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- 172.21.15.19:0/1393745690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 msgr2=0x55f602203f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 0x55f602203f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6d740127b0 tx=0x7f6d74012c70 comp rx=0 tx=0).stop 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- 172.21.15.19:0/1393745690 shutdown_connections 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f602211130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602204480 0x55f60220e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 --2- 172.21.15.19:0/1393745690 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602203b60 0x55f602203f40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.903 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- 172.21.15.19:0/1393745690 >> 172.21.15.19:0/1393745690 conn(0x55f6022000f0 msgr2=0x55f6022004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- 172.21.15.19:0/1393745690 shutdown_connections 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- 172.21.15.19:0/1393745690 wait complete. 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.891+0000 7f6d85953740 1 -- start start 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 Processor -- start 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602204480 0x55f60220c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f60220c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6022118b0 con 0x55f602203b60 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f602123f10 con 0x55f60220ed40 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f602211a30 con 0x55f602204480 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d85953740 10 client.0 _ll_register_callbacks cb 0x55f6020eca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:28.904 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34450/0 (socket says 172.21.15.19:34450) 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 -- 172.21.15.19:0/2995274932 learned_addr learned my addr 172.21.15.19:0/2995274932 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602204480 0x55f60220c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81b17640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f60220c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 -- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602204480 msgr2=0x55f60220c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602204480 0x55f60220c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 -- 172.21.15.19:0/2995274932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 msgr2=0x55f60220c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.892+0000 7f6d81316640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f60220c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d81316640 1 -- 172.21.15.19:0/2995274932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f601eda6a0 con 0x55f602203b60 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d81b17640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f60220c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:28.905 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d81316640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009610 tx=0x7f6d6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b66fb13fe422a51 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c020040 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d6c0998a0 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d74004c80 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c009de0 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d85953740 10 client.18099 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f60221d530 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d74004c80 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f6d6c084b30 con 0x55f602203b60 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.893+0000 7f6d85953740 10 client.18099 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d85953740 10 client.18099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d85953740 20 client.18099 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:28.906 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f602220ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d85953740 4 client.18099 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6021767f0 con 0x55f60221eab0 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f602220ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f602220ec0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bceab704b787e32 server_cookie=6e53cbe445905551 in_seq=0 out_seq=0 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.894+0000 7f6d60ff9640 10 client.18099 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2995274932 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6021767f0 con 0x55f60221eab0 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d60ff9640 10 client.18099 handle_command_reply: tid=0 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 client.18099 shutdown 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 msgr2=0x55f60220bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009610 tx=0x7f6d6c000bc0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 shutdown_connections 2024-09-12T03:32:28.907 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f602220ec0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f60220ed40 0x55f60220c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d60ff9640 0 client.18099 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f602204480 0x55f60220c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 --2- 172.21.15.19:0/2995274932 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220bc50 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 >> 172.21.15.19:0/2995274932 conn(0x55f6022000f0 msgr2=0x55f6021ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 shutdown_connections 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 1 -- 172.21.15.19:0/2995274932 wait complete. 2024-09-12T03:32:28.908 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.895+0000 7f6d85953740 20 client.18099 trim_cache size 0 max 16384 2024-09-12T03:32:28.914 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.912+0000 7f6d85953740 1 Processor -- start 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 -- start start 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f602206610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f602226780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1ad490 con 0x55f602203b60 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1af210 con 0x55f602204480 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d85953740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1adb60 con 0x55f60220ed40 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34460/0 (socket says 172.21.15.19:34460) 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 -- 172.21.15.19:0/25176383 learned_addr learned my addr 172.21.15.19:0/25176383 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:28.915 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81b17640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f602226780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d80b15640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f602206610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 -- 172.21.15.19:0/25176383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 msgr2=0x55f602226780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f602226780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 -- 172.21.15.19:0/25176383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 msgr2=0x55f602206610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f602206610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 -- 172.21.15.19:0/25176383 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f602201260 con 0x55f602203b60 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81b17640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f602226780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d80b15640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f602206610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.913+0000 7f6d81316640 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0058c0 tx=0x7f6d6c005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba728b6e8278a5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d62ffd640 1 -- 172.21.15.19:0/25176383 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c020050 con 0x55f602203b60 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d62ffd640 1 -- 172.21.15.19:0/25176383 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0b8020 con 0x55f602203b60 2024-09-12T03:32:28.916 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d62ffd640 1 -- 172.21.15.19:0/25176383 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c097ca0 con 0x55f602203b60 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 -- 172.21.15.19:0/25176383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 msgr2=0x55f60220ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0058c0 tx=0x7f6d6c005bf0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 -- 172.21.15.19:0/25176383 shutdown_connections 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f602226780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f602206610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 --2- 172.21.15.19:0/25176383 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f60220ab90 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 -- 172.21.15.19:0/25176383 >> 172.21.15.19:0/25176383 conn(0x55f6022000f0 msgr2=0x55f60220dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 -- 172.21.15.19:0/25176383 shutdown_connections 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.914+0000 7f6d85953740 1 -- 172.21.15.19:0/25176383 wait complete. 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 -- start start 2024-09-12T03:32:28.917 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 Processor -- start 2024-09-12T03:32:28.920 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f6022130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f6022135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1aea80 con 0x55f602203b60 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1ae2f0 con 0x55f602204480 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6d7c1ab3b0 con 0x55f60220ed40 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d85953740 10 client.0 _ll_register_callbacks cb 0x7f6d6c09b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.921 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d81316640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:34474/0 (socket says 172.21.15.19:34474) 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.915+0000 7f6d81316640 1 -- 172.21.15.19:0/2628596368 learned_addr learned my addr 172.21.15.19:0/2628596368 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81b17640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f6022135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f6022130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 -- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 msgr2=0x55f6022135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f6022135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 -- 172.21.15.19:0/2628596368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 msgr2=0x55f6022130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f6022130b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 -- 172.21.15.19:0/2628596368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6022888f0 con 0x55f602203b60 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f6022130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d81316640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c09b530 tx=0x7f6d6c09be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67be7cfef708515f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:28.922 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c065060 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6d6c097c10 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c0b8020 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c020070 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d85953740 10 client.18105 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f602289c10 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.916+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d6c0b8020 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f6d6c0973f0 con 0x55f602203b60 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 10 client.18105 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 10 client.18105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 20 client.18105 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f60228bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:28.923 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 4 client.18105 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f602170d00 con 0x55f60221eab0 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f60228bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d80b15640 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f60228bf10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b029265e7a8a34b4 server_cookie=17295e185d1c15fe in_seq=0 out_seq=0 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.917+0000 7f6d60ff9640 10 client.18105 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.918+0000 7f6d60ff9640 1 -- 172.21.15.19:0/2628596368 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f602170d00 con 0x55f60221eab0 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.918+0000 7f6d60ff9640 10 client.18105 handle_command_reply: tid=0 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.918+0000 7f6d85953740 1 client.18105 shutdown 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 msgr2=0x55f602212b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c09b530 tx=0x7f6d6c09be10 comp rx=0 tx=0).stop 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 shutdown_connections 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f60220ed40 0x55f6022135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.924 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f60221eab0 0x55f60228bf10 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f602204480 0x55f6022130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 --2- 172.21.15.19:0/2628596368 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f602203b60 0x55f602212b70 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 >> 172.21.15.19:0/2628596368 conn(0x55f6022000f0 msgr2=0x55f60220d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d60ff9640 0 client.18105 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 shutdown_connections 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 1 -- 172.21.15.19:0/2628596368 wait complete. 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d85953740 20 client.18105 trim_cache size 0 max 16384 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:32:28.925 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "9c70597d-9066-420e-840d-98beca7bc49f", 2024-09-12T03:32:28.926 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:32:28.926 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:32:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.919+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 msgr2=0x7f6d640a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 0x7f6d640a5040 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6d7401a000 tx=0x7f6d7401d000 comp rx=0 tx=0).stop 2024-09-12T03:32:28.926 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 msgr2=0x7f6d7c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00d950 tx=0x7f6d6c00de10 comp rx=0 tx=0).stop 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 shutdown_connections 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6d7c102ef0 0x7f6d7c19cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6d7c102570 0x7f6d7c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6d640a2b80 0x7f6d640a5040 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 --2- 172.21.15.19:0/3433848128 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6d7c100e20 0x7f6d7c198440 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 >> 172.21.15.19:0/3433848128 conn(0x7f6d7c0fc7d0 msgr2=0x7f6d7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 shutdown_connections 2024-09-12T03:32:28.927 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:28.920+0000 7f6d8357a640 1 -- 172.21.15.19:0/3433848128 wait complete. 2024-09-12T03:32:29.531 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:29 smithi019 ceph-mon[30953]: pgmap v740: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 599 KiB/s wr, 237 op/s 2024-09-12T03:32:29.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:29 smithi135 ceph-mon[35474]: pgmap v740: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 599 KiB/s wr, 237 op/s 2024-09-12T03:32:29.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:29 smithi121 ceph-mon[34963]: pgmap v740: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 599 KiB/s wr, 237 op/s 2024-09-12T03:32:29.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:32:30.066 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:30.492 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:32:30.493 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:30 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:32:30.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:32:30.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:32:30.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:32:30.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:30 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 -- 172.21.15.19:0/3012468870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102ef0 msgr2=0x7f51f410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.592 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 --2- 172.21.15.19:0/3012468870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102ef0 0x7f51f410da40 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009ca0 tx=0x7f51ec02a950 comp rx=0 tx=0).stop 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 -- 172.21.15.19:0/3012468870 shutdown_connections 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 --2- 172.21.15.19:0/3012468870 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102ef0 0x7f51f410da40 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 --2- 172.21.15.19:0/3012468870 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4102570 0x7f51f41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 --2- 172.21.15.19:0/3012468870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4100e20 0x7f51f4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 -- 172.21.15.19:0/3012468870 >> 172.21.15.19:0/3012468870 conn(0x7f51f40fc7d0 msgr2=0x7f51f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 -- 172.21.15.19:0/3012468870 shutdown_connections 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.575+0000 7f51fc715640 1 -- 172.21.15.19:0/3012468870 wait complete. 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 Processor -- start 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 -- start start 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4100e20 0x7f51f4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.593 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 0x7f51f419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4110f80 con 0x7f51f4102570 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4110e00 con 0x7f51f4102ef0 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fc715640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51f4111100 con 0x7f51f4100e20 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41076/0 (socket says 172.21.15.19:41076) 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 -- 172.21.15.19:0/1781596270 learned_addr learned my addr 172.21.15.19:0/1781596270 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4100e20 0x7f51f4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 -- 172.21.15.19:0/1781596270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4100e20 msgr2=0x7f51f4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4100e20 0x7f51f4198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 -- 172.21.15.19:0/1781596270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 msgr2=0x7f51f419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51facb2640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 0x7f51f419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 0x7f51f419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51f9cb0640 1 -- 172.21.15.19:0/1781596270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f419d4b0 con 0x7f51f4102570 2024-09-12T03:32:30.594 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.576+0000 7f51facb2640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 0x7f51f419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9f0 tx=0x7f51e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51eb7fe640 1 -- 172.21.15.19:0/1781596270 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e400cd80 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f419d7a0 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51eb7fe640 1 -- 172.21.15.19:0/1781596270 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51e4005500 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51eb7fe640 1 -- 172.21.15.19:0/1781596270 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e4010690 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.577+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f41a5080 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.579+0000 7f51eb7fe640 1 -- 172.21.15.19:0/1781596270 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f51e40046c0 con 0x7f51f4102570 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.579+0000 7f51feaee740 1 Processor -- start 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 -- start start 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e88740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e99e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.595 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8dee2e0 con 0x55a2b8e97a90 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8d95ce0 con 0x55a2b8e88360 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51feaee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8dea2a0 con 0x55a2b8e88c80 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51f9cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e99e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:33418/0 (socket says 172.21.15.19:33418) 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 -- 172.21.15.19:0/1246733344 learned_addr learned my addr 172.21.15.19:0/1246733344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51facb2640 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e88740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51eb7fe640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 0x7f51e00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 -- 172.21.15.19:0/1246733344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 msgr2=0x55a2b8e88740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e88740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 -- 172.21.15.19:0/1246733344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 msgr2=0x55a2b8e99e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e99e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.596 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.580+0000 7f51fa4b1640 1 -- 172.21.15.19:0/1246733344 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b8d7a440 con 0x55a2b8e88c80 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51facb2640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 0x7f51e00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51eb7fe640 1 -- 172.21.15.19:0/1781596270 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51e40cf590 con 0x7f51f4102570 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009be0 tx=0x7f51f000e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44a1efb1b4786ace server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51e8ff9640 1 -- 172.21.15.19:0/1246733344 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0016070 con 0x55a2b8e88c80 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51facb2640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 0x7f51e00a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f51ec004af0 tx=0x7f51ec03d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.597 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51e8ff9640 1 -- 172.21.15.19:0/1246733344 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f0005500 con 0x55a2b8e88c80 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.581+0000 7f51e8ff9640 1 -- 172.21.15.19:0/1246733344 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0011470 con 0x55a2b8e88c80 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- 172.21.15.19:0/1246733344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 msgr2=0x55a2b8e97550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009be0 tx=0x7f51f000e6c0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- 172.21.15.19:0/1246733344 shutdown_connections 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e99e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e97550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 --2- 172.21.15.19:0/1246733344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e88740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- 172.21.15.19:0/1246733344 >> 172.21.15.19:0/1246733344 conn(0x55a2b8e84a30 msgr2=0x55a2b8e84e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- 172.21.15.19:0/1246733344 shutdown_connections 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- 172.21.15.19:0/1246733344 wait complete. 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 -- start start 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.582+0000 7f51feaee740 1 Processor -- start 2024-09-12T03:32:30.598 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e97380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e94630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e94b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8e9a660 con 0x55a2b8e97a90 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8da9050 con 0x55a2b8e88360 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55a2b8e9a7e0 con 0x55a2b8e88c80 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51feaee740 10 client.0 _ll_register_callbacks cb 0x55a2b8de9160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51fa4b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e97380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e94b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51fa4b1640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e97380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:39868/0 (socket says 172.21.15.19:39868) 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51fa4b1640 1 -- 172.21.15.19:0/3233974415 learned_addr learned my addr 172.21.15.19:0/3233974415 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e94630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 -- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 msgr2=0x55a2b8e94630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.599 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e94630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 -- 172.21.15.19:0/3233974415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 msgr2=0x55a2b8e97380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e97380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 -- 172.21.15.19:0/3233974415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b8b5f3c0 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51facb2640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e94b70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005e30 tx=0x7f51ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4328c91349a8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.583+0000 7f51ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec034040 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51ec003d80 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f0005500 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f51ec004040 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 10 client.18117 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55a2b8ea6010 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51f0005500 con 0x55a2b8e97a90 2024-09-12T03:32:30.600 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f51ec04e020 con 0x55a2b8e97a90 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 10 client.18117 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 10 client.18117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 20 client.18117 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8ea7520 0x55a2b8ea9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 4 client.18117 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.584+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2b8dfb850 con 0x55a2b8ea7520 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.585+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8ea7520 0x55a2b8ea9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.585+0000 7f51ddffb640 10 client.18117 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.585+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8ea7520 0x55a2b8ea9980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53de388ef52ee2f6 server_cookie=e2a8d228685af231 in_seq=0 out_seq=0 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51ddffb640 1 -- 172.21.15.19:0/3233974415 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2b8dfb850 con 0x55a2b8ea7520 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51ddffb640 10 client.18117 handle_command_reply: tid=0 2024-09-12T03:32:30.601 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 client.18117 shutdown 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 msgr2=0x55a2b8e94b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e94b70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005e30 tx=0x7f51ec004a40 comp rx=0 tx=0).stop 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 shutdown_connections 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e94b70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88c80 0x55a2b8e94630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8ea7520 0x55a2b8ea9980 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 --2- 172.21.15.19:0/3233974415 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88360 0x55a2b8e97380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51ddffb640 0 client.18117 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 >> 172.21.15.19:0/3233974415 conn(0x55a2b8e84a30 msgr2=0x55a2b8e8a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 shutdown_connections 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 1 -- 172.21.15.19:0/3233974415 wait complete. 2024-09-12T03:32:30.602 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.586+0000 7f51feaee740 20 client.18117 trim_cache size 0 max 16384 2024-09-12T03:32:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 Processor -- start 2024-09-12T03:32:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 -- start start 2024-09-12T03:32:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8ea3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.604 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88c80 0x55a2b8e8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ab3a0 con 0x55a2b8e97a90 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ad120 con 0x55a2b8e88c80 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.603+0000 7f51feaee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41aba70 con 0x55a2b8e88360 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41112/0 (socket says 172.21.15.19:41112) 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 -- 172.21.15.19:0/1084287157 learned_addr learned my addr 172.21.15.19:0/1084287157 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88c80 0x55a2b8e8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8ea3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 -- 172.21.15.19:0/1084287157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 msgr2=0x55a2b8ea3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8ea3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 -- 172.21.15.19:0/1084287157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88c80 msgr2=0x55a2b8e8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.605 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88c80 0x55a2b8e8fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 -- 172.21.15.19:0/1084287157 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2b8e84860 con 0x55a2b8e97a90 2024-09-12T03:32:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8ea3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:30.606 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51facb2640 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f51ec045b80 tx=0x7f51ec045f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755d47f928ccf99b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51dffff640 1 -- 172.21.15.19:0/1084287157 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec034070 con 0x55a2b8e97a90 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51dffff640 1 -- 172.21.15.19:0/1084287157 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51ec003d80 con 0x55a2b8e97a90 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.604+0000 7f51dffff640 1 -- 172.21.15.19:0/1084287157 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ec050440 con 0x55a2b8e97a90 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- 172.21.15.19:0/1084287157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 msgr2=0x55a2b8e965a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f51ec045b80 tx=0x7f51ec045f80 comp rx=0 tx=0).stop 2024-09-12T03:32:30.609 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- 172.21.15.19:0/1084287157 shutdown_connections 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e97a90 0x55a2b8e965a0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e88c80 0x55a2b8e8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 --2- 172.21.15.19:0/1084287157 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8ea3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- 172.21.15.19:0/1084287157 >> 172.21.15.19:0/1084287157 conn(0x55a2b8e84a30 msgr2=0x55a2b8e95e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- 172.21.15.19:0/1084287157 shutdown_connections 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- 172.21.15.19:0/1084287157 wait complete. 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 -- start start 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.605+0000 7f51feaee740 1 Processor -- start 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8e9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e97a90 0x55a2b8f11220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ac990 con 0x55a2b8e88c80 2024-09-12T03:32:30.610 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41ac200 con 0x55a2b8e97a90 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f51f41a92c0 con 0x55a2b8e88360 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51feaee740 10 client.0 _ll_register_callbacks cb 0x7f51e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41116/0 (socket says 172.21.15.19:41116) 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 -- 172.21.15.19:0/2374496154 learned_addr learned my addr 172.21.15.19:0/2374496154 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51facb2640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e97a90 0x55a2b8f11220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8e9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 -- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 msgr2=0x55a2b8e9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8e9bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 -- 172.21.15.19:0/2374496154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e97a90 msgr2=0x55a2b8f11220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e97a90 0x55a2b8f11220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 -- 172.21.15.19:0/2374496154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2b8f11760 con 0x55a2b8e88c80 2024-09-12T03:32:30.611 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51f9cb0640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c6a0 tx=0x7f51e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f92e08ae95dc3df server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.606+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e409f030 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f51e40cd150 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51ec003d80 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f51e409c4a0 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 10 client.18123 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55a2b8f12870 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51ec003d80 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f51e40cf0b0 con 0x55a2b8e88c80 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 10 client.18123 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 10 client.18123 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:32:30.612 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 20 client.18123 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8f12fd0 0x55a2b8f153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 4 client.18123 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a2b8df5d60 con 0x55a2b8f12fd0 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.607+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8f12fd0 0x55a2b8f153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51fa4b1640 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8f12fd0 0x55a2b8f153f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d98fc83d7bfa5 server_cookie=81b17a434ee0b5f in_seq=0 out_seq=0 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51ddffb640 10 client.18123 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51ddffb640 1 -- 172.21.15.19:0/2374496154 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a2b8df5d60 con 0x55a2b8f12fd0 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51ddffb640 10 client.18123 handle_command_reply: tid=0 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 client.18123 shutdown 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 msgr2=0x55a2b8e9c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f51e400c6a0 tx=0x7f51e4004a40 comp rx=0 tx=0).stop 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 shutdown_connections 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55a2b8f12fd0 0x55a2b8f153f0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.613 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55a2b8e97a90 0x55a2b8f11220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55a2b8e88c80 0x55a2b8e9c0f0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 --2- 172.21.15.19:0/2374496154 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55a2b8e88360 0x55a2b8e9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 >> 172.21.15.19:0/2374496154 conn(0x55a2b8e84a30 msgr2=0x55a2b8e8fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51ddffb640 0 client.18123 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 shutdown_connections 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 1 -- 172.21.15.19:0/2374496154 wait complete. 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.608+0000 7f51feaee740 20 client.18123 trim_cache size 0 max 16384 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:32:30.614 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: "9c70597d-9066-420e-840d-98beca7bc49f": { 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "9c70597d-9066-420e-840d-98beca7bc49f", 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:30.615 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 msgr2=0x7f51e00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 0x7f51e00a5040 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f51ec004af0 tx=0x7f51ec03d040 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 msgr2=0x7f51f4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f51e400e9f0 tx=0x7f51e400edf0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 shutdown_connections 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f51f4102ef0 0x7f51f419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f51e00a2b80 0x7f51e00a5040 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f51f4102570 0x7f51f4198a40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 --2- 172.21.15.19:0/1781596270 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f51f4100e20 0x7f51f4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 >> 172.21.15.19:0/1781596270 conn(0x7f51f40fc7d0 msgr2=0x7f51f406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 shutdown_connections 2024-09-12T03:32:30.616 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:30.609+0000 7f51fc715640 1 -- 172.21.15.19:0/1781596270 wait complete. 2024-09-12T03:32:30.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:32:30.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:30 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:31.475 INFO:tasks.cephfs.filesystem:scrub status for tag:9c70597d-9066-420e-840d-98beca7bc49f - {'path': '/', 'tag': '9c70597d-9066-420e-840d-98beca7bc49f', 'options': 'recursive,force'} 2024-09-12T03:32:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:31 smithi135 ceph-mon[35474]: pgmap v741: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 540 KiB/s wr, 166 op/s 2024-09-12T03:32:31.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:31 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:31 smithi121 ceph-mon[34963]: pgmap v741: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 540 KiB/s wr, 166 op/s 2024-09-12T03:32:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:31 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:31.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:31 smithi019 ceph-mon[30953]: pgmap v741: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 540 KiB/s wr, 166 op/s 2024-09-12T03:32:31.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:31 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:32:33.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:33 smithi135 ceph-mon[35474]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 381 KiB/s wr, 129 op/s 2024-09-12T03:32:33.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:33 smithi121 ceph-mon[34963]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 381 KiB/s wr, 129 op/s 2024-09-12T03:32:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:33 smithi019 ceph-mon[30953]: pgmap v742: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 381 KiB/s wr, 129 op/s 2024-09-12T03:32:35.545 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:35 smithi135 ceph-mon[35474]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 72 op/s 2024-09-12T03:32:35.655 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:35 smithi121 ceph-mon[34963]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 72 op/s 2024-09-12T03:32:35.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:35 smithi019 ceph-mon[30953]: pgmap v743: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 75 KiB/s wr, 72 op/s 2024-09-12T03:32:37.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:37 smithi135 ceph-mon[35474]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 45 op/s 2024-09-12T03:32:37.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:37 smithi121 ceph-mon[34963]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 45 op/s 2024-09-12T03:32:37.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:37 smithi019 ceph-mon[30953]: pgmap v744: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 70 KiB/s wr, 45 op/s 2024-09-12T03:32:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:39 smithi135 ceph-mon[35474]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 KiB/s wr, 37 op/s 2024-09-12T03:32:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:39 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:32:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:39 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:32:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:39 smithi121 ceph-mon[34963]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 KiB/s wr, 37 op/s 2024-09-12T03:32:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:39 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:32:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:39 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:32:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:39 smithi019 ceph-mon[30953]: pgmap v745: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 KiB/s wr, 37 op/s 2024-09-12T03:32:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:39 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:32:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:39 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:32:40.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:40 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:32:40.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:40 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:32:40.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:40 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:32:41.170 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:32:41.446 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:41.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:41 smithi135 ceph-mon[35474]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-12T03:32:41.568 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:41 smithi019 ceph-mon[30953]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-12T03:32:41.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:41 smithi121 ceph-mon[34963]: pgmap v746: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 12 KiB/s wr, 26 op/s 2024-09-12T03:32:42.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 -- 172.21.15.19:0/1867215814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0101de0 msgr2=0x7fceb01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 --2- 172.21.15.19:0/1867215814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0101de0 0x7fceb01021c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fcea0009900 tx=0x7fcea002a410 comp rx=0 tx=0).stop 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 -- 172.21.15.19:0/1867215814 shutdown_connections 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 --2- 172.21.15.19:0/1867215814 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 0x7fceb010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 --2- 172.21.15.19:0/1867215814 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0102790 0x7fceb01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 --2- 172.21.15.19:0/1867215814 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0101de0 0x7fceb01021c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 -- 172.21.15.19:0/1867215814 >> 172.21.15.19:0/1867215814 conn(0x7fceb00fd8c0 msgr2=0x7fceb00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 -- 172.21.15.19:0/1867215814 shutdown_connections 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.045+0000 7fceb6611640 1 -- 172.21.15.19:0/1867215814 wait complete. 2024-09-12T03:32:42.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.046+0000 7fceb6611640 1 Processor -- start 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.046+0000 7fceb6611640 1 -- start start 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0101de0 0x7fceb01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 0x7fceb019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0112010 con 0x7fceb0102790 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0111e90 con 0x7fceb0106f10 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fceb0112190 con 0x7fceb0101de0 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36800/0 (socket says 172.21.15.19:36800) 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 -- 172.21.15.19:0/1027174216 learned_addr learned my addr 172.21.15.19:0/1027174216 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaffff640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0101de0 0x7fceb01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:42.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb4bae640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 0x7fceb019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 -- 172.21.15.19:0/1027174216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0101de0 msgr2=0x7fceb01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0101de0 0x7fceb01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 -- 172.21.15.19:0/1027174216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 msgr2=0x7fceb019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 0x7fceb019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 -- 172.21.15.19:0/1027174216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceb019d560 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceaf7fe640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcea400b4a0 tx=0x7fcea400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4014070 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcea4010af0 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceb019d850 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.047+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea4017d10 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.048+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceb01a5130 con 0x7fceb0102790 2024-09-12T03:32:42.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.048+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceb0102240 con 0x7fceb0102790 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.050+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcea4017070 con 0x7fceb0102790 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.050+0000 7fcead7fa640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 0x7fce980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.050+0000 7fceaffff640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 0x7fce980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.050+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fcea40ce760 con 0x7fceb0102790 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.051+0000 7fceaffff640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 0x7fce980a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:42.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.054+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcea409b080 con 0x7fceb0102790 2024-09-12T03:32:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.268+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fceb0199910 con 0x7fceb0102790 2024-09-12T03:32:42.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.269+0000 7fcead7fa640 1 -- 172.21.15.19:0/1027174216 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+12009 (secure 0 0 0) 0x7fceb0199910 con 0x7fceb0102790 2024-09-12T03:32:42.269 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:32:42.270 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"btime":"2024-09-12T03:32:19:237768+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:32:19.167870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:32:42.270 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 63 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.271+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 msgr2=0x7fce980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.271+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 0x7fce980a5040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcea0000c00 tx=0x7fcea00058d0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.271+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 msgr2=0x7fceb0198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.271+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fcea400b4a0 tx=0x7fcea400b960 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 shutdown_connections 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fceb0106f10 0x7fceb019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fce980a2b80 0x7fce980a5040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fceb0102790 0x7fceb0198af0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 --2- 172.21.15.19:0/1027174216 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fceb0101de0 0x7fceb01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:42.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 >> 172.21.15.19:0/1027174216 conn(0x7fceb00fd8c0 msgr2=0x7fceb00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:42.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 shutdown_connections 2024-09-12T03:32:42.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:42.272+0000 7fceb6611640 1 -- 172.21.15.19:0/1027174216 wait complete. 2024-09-12T03:32:42.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:42 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1027174216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:42.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:42 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1027174216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:42.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:42 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1027174216' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:32:43.213 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --set-id 81c320fe --release --await' 2024-09-12T03:32:43.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --set-id 81c320fe --release --await 2024-09-12T03:32:43.480 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:32:43.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:43 smithi135 ceph-mon[35474]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-09-12T03:32:43.602 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:43 smithi019 ceph-mon[30953]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-09-12T03:32:43.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:43 smithi121 ceph-mon[34963]: pgmap v747: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 1.1 KiB/s wr, 22 op/s 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 -- 172.21.15.19:0/2389482018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 msgr2=0x7fe4701021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 --2- 172.21.15.19:0/2389482018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4701021c0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe460009900 tx=0x7fe46002a410 comp rx=0 tx=0).stop 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 -- 172.21.15.19:0/2389482018 shutdown_connections 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 --2- 172.21.15.19:0/2389482018 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470106f10 0x7fe47010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 --2- 172.21.15.19:0/2389482018 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470102790 0x7fe4701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 --2- 172.21.15.19:0/2389482018 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4701021c0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 -- 172.21.15.19:0/2389482018 >> 172.21.15.19:0/2389482018 conn(0x7fe4700fd8c0 msgr2=0x7fe4700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:43.999 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 -- 172.21.15.19:0/2389482018 shutdown_connections 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.973+0000 7fe476606640 1 -- 172.21.15.19:0/2389482018 wait complete. 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 Processor -- start 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 -- start start 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470102790 0x7fe470074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470106f10 0x7fe470074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe470112010 con 0x7fe470101de0 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe470111e90 con 0x7fe470102790 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe476606640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe470112190 con 0x7fe470106f10 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.000 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36814/0 (socket says 172.21.15.19:36814) 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe46ffff640 1 -- 172.21.15.19:0/3294400905 learned_addr learned my addr 172.21.15.19:0/3294400905 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470102790 0x7fe470074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.974+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470106f10 0x7fe470074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 -- 172.21.15.19:0/3294400905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470106f10 msgr2=0x7fe470074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470106f10 0x7fe470074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 -- 172.21.15.19:0/3294400905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470102790 msgr2=0x7fe470074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470102790 0x7fe470074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 -- 172.21.15.19:0/3294400905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4701a6af0 con 0x7fe470101de0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46ffff640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe460005ab0 tx=0x7fe4600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46d7fa640 1 -- 172.21.15.19:0/3294400905 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46003f030 con 0x7fe470101de0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46d7fa640 1 -- 172.21.15.19:0/3294400905 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe460003ce0 con 0x7fe470101de0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe46d7fa640 1 -- 172.21.15.19:0/3294400905 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460035070 con 0x7fe470101de0 2024-09-12T03:32:44.001 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4701a6d80 con 0x7fe470101de0 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.975+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4701a7200 con 0x7fe470101de0 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.978+0000 7fe46d7fa640 1 -- 172.21.15.19:0/3294400905 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe460042030 con 0x7fe470101de0 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.978+0000 7fe4789df740 1 Processor -- start 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.978+0000 7fe4789df740 1 -- start start 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4060 0x564eccbe4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbeed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564eccb49620 con 0x564eccbef240 2024-09-12T03:32:44.002 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564eccaf0e50 con 0x564eccbe4980 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe4789df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564eccb455e0 con 0x564eccbe4060 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36826/0 (socket says 172.21.15.19:36826) 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 -- 172.21.15.19:0/2381840244 learned_addr learned my addr 172.21.15.19:0/2381840244 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe474ba3640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4060 0x564eccbe4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46ffff640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbeed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 -- 172.21.15.19:0/2381840244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4060 msgr2=0x564eccbe4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4060 0x564eccbe4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 -- 172.21.15.19:0/2381840244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 msgr2=0x564eccbeed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbeed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.003 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 -- 172.21.15.19:0/2381840244 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eccae1d40 con 0x564eccbef240 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46ffff640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbeed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46d7fa640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 0x7fe4580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe464009a40 tx=0x7fe464009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c626904782fda6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe46d7fa640 1 -- 172.21.15.19:0/3294400905 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe4600ef9a0 con 0x7fe470101de0 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.979+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 0x7fe4580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4567fc640 1 -- 172.21.15.19:0/2381840244 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464017070 con 0x564eccbef240 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4567fc640 1 -- 172.21.15.19:0/2381840244 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe464013470 con 0x564eccbef240 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4567fc640 1 -- 172.21.15.19:0/2381840244 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe464012610 con 0x564eccbef240 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 -- 172.21.15.19:0/2381840244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 msgr2=0x564eccbf1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 0x7fe4580a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe46800a400 tx=0x7fe46800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fe464009a40 tx=0x7fe464009f00 comp rx=0 tx=0).stop 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 -- 172.21.15.19:0/2381840244 shutdown_connections 2024-09-12T03:32:44.004 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbf1630 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbeed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 --2- 172.21.15.19:0/2381840244 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4060 0x564eccbe4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 -- 172.21.15.19:0/2381840244 >> 172.21.15.19:0/2381840244 conn(0x564eccbe05f0 msgr2=0x564eccbe09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 -- 172.21.15.19:0/2381840244 shutdown_connections 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.980+0000 7fe4789df740 1 -- 172.21.15.19:0/2381840244 wait complete. 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 -- start start 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 Processor -- start 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccbec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x564eccbf1e10 con 0x564eccbef240 2024-09-12T03:32:44.005 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x564eccad1f10 con 0x564eccbe4060 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x564eccbf1f90 con 0x564eccbe4980 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.981+0000 7fe4789df740 10 client.0 _ll_register_callbacks cb 0x564eccaf5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36830/0 (socket says 172.21.15.19:36830) 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 -- 172.21.15.19:0/4064257660 learned_addr learned my addr 172.21.15.19:0/4064257660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccbec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 -- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 msgr2=0x564eccbec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccbec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 -- 172.21.15.19:0/4064257660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 msgr2=0x564eccbec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 -- 172.21.15.19:0/4064257660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ecc858580 con 0x564eccbef240 2024-09-12T03:32:44.006 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe46ffff640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:32:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe474ba3640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe468018e60 tx=0x7fe468004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c47f3a97e7e9758 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe44f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468027060 con 0x564eccbef240 2024-09-12T03:32:44.007 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe464013470 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.982+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe468018050 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe468018250 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 10 client.18141 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564ecc922960 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe464013470 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fe468016070 con 0x564eccbef240 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 10 client.18141 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 10 client.18141 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 20 client.18141 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:44.008 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccbfedd0 0x564eccc01250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 4 client.18141 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564eccb56b90 con 0x564eccbfedd0 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.983+0000 7fe46ffff640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccbfedd0 0x564eccc01250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.984+0000 7fe46ffff640 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccbfedd0 0x564eccc01250 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b291e3da480de server_cookie=6f9103d674e39a30 in_seq=0 out_seq=0 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.984+0000 7fe44f7fe640 10 client.18141 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe44f7fe640 1 -- 172.21.15.19:0/4064257660 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564eccb56b90 con 0x564eccbfedd0 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe44f7fe640 10 client.18141 handle_command_reply: tid=0 2024-09-12T03:32:44.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 client.18141 shutdown 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 msgr2=0x564eccbecb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fe468018e60 tx=0x7fe468004a40 comp rx=0 tx=0).stop 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 shutdown_connections 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccbecb30 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccbec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccbfedd0 0x564eccc01250 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 --2- 172.21.15.19:0/4064257660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 >> 172.21.15.19:0/4064257660 conn(0x564eccbe05f0 msgr2=0x564eccbe62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe44f7fe640 0 client.18141 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 shutdown_connections 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 1 -- 172.21.15.19:0/4064257660 wait complete. 2024-09-12T03:32:44.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:43.985+0000 7fe4789df740 20 client.18141 trim_cache size 0 max 16384 2024-09-12T03:32:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.002+0000 7fe4789df740 1 Processor -- start 2024-09-12T03:32:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.002+0000 7fe4789df740 1 -- start start 2024-09-12T03:32:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.002+0000 7fe4789df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbdf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe4789df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccc06b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe4789df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe4789df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701af840 con 0x564eccbef240 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe4789df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701b15c0 con 0x564eccbe4060 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe4789df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701aff10 con 0x564eccbe4980 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36840/0 (socket says 172.21.15.19:36840) 2024-09-12T03:32:44.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 -- 172.21.15.19:0/3008873359 learned_addr learned my addr 172.21.15.19:0/3008873359 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccc06b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe46ffff640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbdf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 -- 172.21.15.19:0/3008873359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 msgr2=0x564eccc06b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccc06b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 -- 172.21.15.19:0/3008873359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 msgr2=0x564eccbdf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.019 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbdf750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 -- 172.21.15.19:0/3008873359 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564eccbe0420 con 0x564eccbef240 2024-09-12T03:32:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe46ffff640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbdf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe474ba3640 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe468018460 tx=0x7fe468009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98cdd74780ac3973 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe455ffb640 1 -- 172.21.15.19:0/3008873359 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468032070 con 0x564eccbef240 2024-09-12T03:32:44.020 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe455ffb640 1 -- 172.21.15.19:0/3008873359 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe468017400 con 0x564eccbef240 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.003+0000 7fe455ffb640 1 -- 172.21.15.19:0/3008873359 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4680229a0 con 0x564eccbef240 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- 172.21.15.19:0/3008873359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 msgr2=0x564eccc070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe468018460 tx=0x7fe468009dd0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- 172.21.15.19:0/3008873359 shutdown_connections 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbef240 0x564eccc070a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.021 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbe4980 0x564eccc06b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 --2- 172.21.15.19:0/3008873359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4060 0x564eccbdf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- 172.21.15.19:0/3008873359 >> 172.21.15.19:0/3008873359 conn(0x564eccbe05f0 msgr2=0x564eccbdb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- 172.21.15.19:0/3008873359 shutdown_connections 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- 172.21.15.19:0/3008873359 wait complete. 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.004+0000 7fe4789df740 1 -- start start 2024-09-12T03:32:44.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 Processor -- start 2024-09-12T03:32:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbf3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 0x564eccbf3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701b0e30 con 0x564eccbe4060 2024-09-12T03:32:44.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701b06a0 con 0x564eccbe4980 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe4701ad760 con 0x564eccbef240 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe4789df740 10 client.0 _ll_register_callbacks cb 0x7fe468018b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36852/0 (socket says 172.21.15.19:36852) 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 -- 172.21.15.19:0/1298005503 learned_addr learned my addr 172.21.15.19:0/1298005503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:32:44.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbf3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe474ba3640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 0x564eccbf3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 -- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 msgr2=0x564eccbf3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 0x564eccbf3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 -- 172.21.15.19:0/1298005503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 msgr2=0x564eccbf3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:32:44.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbf3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe46ffff640 1 -- 172.21.15.19:0/1298005503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564eccc68d30 con 0x564eccbe4060 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.005+0000 7fe474ba3640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 0x564eccbf3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe46ffff640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe4600bc1a0 tx=0x7fe4600f2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7e2bd560e74d25 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe460049040 con 0x564eccbe4060 2024-09-12T03:32:44.026 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe46003bd00 con 0x564eccbe4060 2024-09-12T03:32:44.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe468017400 con 0x564eccbe4060 2024-09-12T03:32:44.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe4600bc210 con 0x564eccbe4060 2024-09-12T03:32:44.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 10 client.18147 fetch_fsmap learned FSMap version 63 2024-09-12T03:32:44.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x564eccc6a070 con 0x564eccbe4060 2024-09-12T03:32:44.027 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe468017400 con 0x564eccbe4060 2024-09-12T03:32:44.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fe4600b8880 con 0x564eccbe4060 2024-09-12T03:32:44.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 10 client.18147 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 10 client.18147 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 20 client.18147 populate_metadata read hostname 'smithi019' 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.006+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccc6a820 0x564eccc6cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.007+0000 7fe4789df740 4 client.18147 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "81c320fe", "release": true, "await": true}] 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.007+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "set_id": "81c320fe", "release": true, "await": true}) -- 0x564eccb510a0 con 0x564eccc6a820 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.007+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccc6a820 0x564eccc6cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.007+0000 7fe44f7fe640 10 client.18147 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:44.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.007+0000 7fe46f7fe640 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccc6a820 0x564eccc6cc40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999a0c8b3af56bed server_cookie=ed5d72ac6eb530fa in_seq=0 out_seq=0 2024-09-12T03:32:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe44f7fe640 1 -- 172.21.15.19:0/1298005503 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564eccb510a0 con 0x564eccc6a820 2024-09-12T03:32:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe44f7fe640 10 client.18147 handle_command_reply: tid=0 2024-09-12T03:32:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe4789df740 1 client.18147 shutdown 2024-09-12T03:32:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 msgr2=0x564eccbf3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe4600bc1a0 tx=0x7fe4600f2f90 comp rx=0 tx=0).stop 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.447+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 shutdown_connections 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x564eccbef240 0x564eccbf3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x564eccc6a820 0x564eccc6cc40 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x564eccbe4980 0x564eccbf3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 --2- 172.21.15.19:0/1298005503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x564eccbe4060 0x564eccbf3140 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe44f7fe640 0 client.18147 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 >> 172.21.15.19:0/1298005503 conn(0x564eccbe05f0 msgr2=0x564eccbee130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 shutdown_connections 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 1 -- 172.21.15.19:0/1298005503 wait complete. 2024-09-12T03:32:44.449 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.448+0000 7fe4789df740 20 client.18147 trim_cache size 0 max 16384 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 63, 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 48, 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": { 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "81c320fe": { 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "version": 48, 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "age_ref": 0.0, 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:32:44.450 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "timeout": 90.0, 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "expiration": 147.2, 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "members": { 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "file:/": { 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "excluded": false, 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "state": { 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RELEASED", 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 0.0 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:44.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:32:44.452 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:32:44.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 msgr2=0x7fe4580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 0x7fe4580a5040 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe46800a400 tx=0x7fe46800e040 comp rx=0 tx=0).stop 2024-09-12T03:32:44.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 msgr2=0x7fe4700744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:32:44.452 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe460005ab0 tx=0x7fe4600042f0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 shutdown_connections 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe470106f10 0x7fe470074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe470102790 0x7fe470074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe4580a2b80 0x7fe4580a5040 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 --2- 172.21.15.19:0/3294400905 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe470101de0 0x7fe4700744f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 >> 172.21.15.19:0/3294400905 conn(0x7fe4700fd8c0 msgr2=0x7fe4700fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 shutdown_connections 2024-09-12T03:32:44.453 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:32:44.449+0000 7fe476606640 1 -- 172.21.15.19:0/3294400905 wait complete. 2024-09-12T03:32:45.350 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '81c320fe', total seconds elapsed: 31.4 2024-09-12T03:32:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:45 smithi121 ceph-mon[34963]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 938 B/s wr, 17 op/s 2024-09-12T03:32:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:45 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:45 smithi019 ceph-mon[30953]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 938 B/s wr, 17 op/s 2024-09-12T03:32:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:45 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:45 smithi135 ceph-mon[35474]: pgmap v748: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 938 B/s wr, 17 op/s 2024-09-12T03:32:45.839 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:45 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:32:46.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:32:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:32:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:32:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:32:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:32:47.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:47 smithi121 ceph-mon[34963]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-12T03:32:47.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:47 smithi019 ceph-mon[30953]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-12T03:32:47.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:47 smithi135 ceph-mon[35474]: pgmap v749: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2024-09-12T03:32:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:49 smithi121 ceph-mon[34963]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 33 op/s 2024-09-12T03:32:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:49 smithi019 ceph-mon[30953]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 33 op/s 2024-09-12T03:32:49.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:49 smithi135 ceph-mon[35474]: pgmap v750: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 33 op/s 2024-09-12T03:32:51.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:51 smithi135 ceph-mon[35474]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 546 KiB/s wr, 63 op/s 2024-09-12T03:32:51.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:51 smithi121 ceph-mon[34963]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 546 KiB/s wr, 63 op/s 2024-09-12T03:32:51.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:51 smithi019 ceph-mon[30953]: pgmap v751: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 546 KiB/s wr, 63 op/s 2024-09-12T03:32:53.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:53 smithi121 ceph-mon[34963]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-09-12T03:32:53.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:53 smithi019 ceph-mon[30953]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-09-12T03:32:53.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:53 smithi135 ceph-mon[35474]: pgmap v752: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 586 KiB/s wr, 140 op/s 2024-09-12T03:32:55.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:55 smithi121 ceph-mon[34963]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 796 KiB/s wr, 204 op/s 2024-09-12T03:32:55.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:55 smithi019 ceph-mon[30953]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 796 KiB/s wr, 204 op/s 2024-09-12T03:32:55.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:55 smithi135 ceph-mon[35474]: pgmap v753: 97 pgs: 97 active+clean; 1.2 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 796 KiB/s wr, 204 op/s 2024-09-12T03:32:57.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:57 smithi121 ceph-mon[34963]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 912 KiB/s wr, 265 op/s 2024-09-12T03:32:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:57 smithi019 ceph-mon[30953]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 912 KiB/s wr, 265 op/s 2024-09-12T03:32:57.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:57 smithi135 ceph-mon[35474]: pgmap v754: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 912 KiB/s wr, 265 op/s 2024-09-12T03:32:59.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:32:59 smithi121 ceph-mon[34963]: pgmap v755: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 370 op/s 2024-09-12T03:32:59.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:32:59 smithi019 ceph-mon[30953]: pgmap v755: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 370 op/s 2024-09-12T03:32:59.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:32:59 smithi135 ceph-mon[35474]: pgmap v755: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 370 op/s 2024-09-12T03:33:01.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:33:01.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:01 smithi121 ceph-mon[34963]: pgmap v756: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 388 op/s 2024-09-12T03:33:01.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:01 smithi019 ceph-mon[30953]: pgmap v756: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 388 op/s 2024-09-12T03:33:01.740 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:01.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:01 smithi135 ceph-mon[35474]: pgmap v756: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 388 op/s 2024-09-12T03:33:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 -- 172.21.15.19:0/3019186984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6810c180 msgr2=0x7efc6810e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 --2- 172.21.15.19:0/3019186984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6810c180 0x7efc6810e620 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7efc60009ca0 tx=0x7efc6002a950 comp rx=0 tx=0).stop 2024-09-12T03:33:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 -- 172.21.15.19:0/3019186984 shutdown_connections 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 --2- 172.21.15.19:0/3019186984 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc6810c180 0x7efc6810e620 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 --2- 172.21.15.19:0/3019186984 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc68103670 0x7efc6810bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 --2- 172.21.15.19:0/3019186984 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 0x7efc68103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 -- 172.21.15.19:0/3019186984 >> 172.21.15.19:0/3019186984 conn(0x7efc680fc7d0 msgr2=0x7efc680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 -- 172.21.15.19:0/3019186984 shutdown_connections 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.227+0000 7efc6f179640 1 -- 172.21.15.19:0/3019186984 wait complete. 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 Processor -- start 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 -- start start 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 0x7efc681009b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 0x7efc68101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc68111f80 con 0x7efc68103670 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc68111e00 con 0x7efc6810c180 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6f179640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc68112100 con 0x7efc68102d50 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39516/0 (socket says 172.21.15.19:39516) 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 -- 172.21.15.19:0/1075271017 learned_addr learned my addr 172.21.15.19:0/1075271017 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6d716640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 0x7efc68101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6cf15640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 0x7efc681009b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 -- 172.21.15.19:0/1075271017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 msgr2=0x7efc681009b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 0x7efc681009b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 -- 172.21.15.19:0/1075271017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 msgr2=0x7efc68101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 0x7efc68101430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc67fff640 1 -- 172.21.15.19:0/1075271017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc681a6970 con 0x7efc68103670 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.228+0000 7efc6d716640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 0x7efc68101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc67fff640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d740 tx=0x7efc5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc65ffb640 1 -- 172.21.15.19:0/1075271017 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58017070 con 0x7efc68103670 2024-09-12T03:33:02.255 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc681a6c60 con 0x7efc68103670 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc65ffb640 1 -- 172.21.15.19:0/1075271017 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5800de50 con 0x7efc68103670 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc65ffb640 1 -- 172.21.15.19:0/1075271017 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5800f620 con 0x7efc68103670 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.229+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc681a71a0 con 0x7efc68103670 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.231+0000 7efc65ffb640 1 -- 172.21.15.19:0/1075271017 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc58009c30 con 0x7efc68103670 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 Processor -- start 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 -- start start 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc65ffb640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 0x7efc540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 0x557a67b25140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b25680 0x557a67b27a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a67a802e0 con 0x557a67b25680 2024-09-12T03:33:02.256 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a67a27ce0 con 0x557a67b1a4a0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc71552740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a67a7c2a0 con 0x557a67b1adc0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6d716640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 0x7efc540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc65ffb640 1 -- 172.21.15.19:0/1075271017 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efc580cec60 con 0x7efc68103670 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc67fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 0x557a67b25140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:57048/0 (socket says 172.21.15.19:57048) 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc67fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 0x557a67b25140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:44928/0 (socket says 172.21.15.19:44928) 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6cf15640 1 -- 172.21.15.19:0/984374665 learned_addr learned my addr 172.21.15.19:0/984374665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6d716640 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b25680 0x557a67b27a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6cf15640 1 -- 172.21.15.19:0/984374665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 msgr2=0x557a67b25140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.232+0000 7efc6cf15640 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 0x557a67b25140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.257 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc6cf15640 1 -- 172.21.15.19:0/984374665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b25680 msgr2=0x557a67b27a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc6cf15640 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b25680 0x557a67b27a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc6cf15640 1 -- 172.21.15.19:0/984374665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a67a0c440 con 0x557a67b1a4a0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc6d716640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 0x7efc540a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efc60005e30 tx=0x7efc60009800 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc6cf15640 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bcf0 tx=0x7efc5000f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7a956455e1b9c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc5effd640 1 -- 172.21.15.19:0/984374665 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50015070 con 0x557a67b1a4a0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc5effd640 1 -- 172.21.15.19:0/984374665 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50011920 con 0x557a67b1a4a0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.233+0000 7efc5effd640 1 -- 172.21.15.19:0/984374665 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50010910 con 0x557a67b1a4a0 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- 172.21.15.19:0/984374665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 msgr2=0x557a67b1a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7efc5000bcf0 tx=0x7efc5000f730 comp rx=0 tx=0).stop 2024-09-12T03:33:02.258 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- 172.21.15.19:0/984374665 shutdown_connections 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b25680 0x557a67b27a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b1adc0 0x557a67b25140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 --2- 172.21.15.19:0/984374665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b1a880 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- 172.21.15.19:0/984374665 >> 172.21.15.19:0/984374665 conn(0x557a67b16a30 msgr2=0x557a67b16e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- 172.21.15.19:0/984374665 shutdown_connections 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- 172.21.15.19:0/984374665 wait complete. 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 -- start start 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.234+0000 7efc71552740 1 Processor -- start 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.259 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557a67b28250 con 0x557a67b1a4a0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557a67a3b050 con 0x557a67b1adc0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557a67b283d0 con 0x557a67b25680 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 10 client.0 _ll_register_callbacks cb 0x557a67a7b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39534/0 (socket says 172.21.15.19:39534) 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 -- 172.21.15.19:0/3992324549 learned_addr learned my addr 172.21.15.19:0/3992324549 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc67fff640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6d716640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 -- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 msgr2=0x557a67b227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.260 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 -- 172.21.15.19:0/3992324549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 msgr2=0x557a67b22260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b22260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 -- 172.21.15.19:0/3992324549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a677f13c0 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc6cf15640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7efc50010410 tx=0x7efc5000ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393a29e34e404443 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc47fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50028060 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efc50011710 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.235+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc50011920 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7efc50015070 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 10 client.18159 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557a67b33c00 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc50011920 con 0x557a67b1a4a0 2024-09-12T03:33:02.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7efc500047e0 con 0x557a67b1a4a0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 10 client.18159 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 10 client.18159 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 20 client.18159 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b350c0 0x557a67b374b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 4 client.18159 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a67a8d850 con 0x557a67b350c0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.236+0000 7efc67fff640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b350c0 0x557a67b374b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.237+0000 7efc67fff640 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b350c0 0x557a67b374b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b133a07aa76d45 server_cookie=2dde83099f606f98 in_seq=0 out_seq=0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.237+0000 7efc47fff640 10 client.18159 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.237+0000 7efc47fff640 1 -- 172.21.15.19:0/3992324549 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557a67a8d850 con 0x557a67b350c0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.237+0000 7efc47fff640 10 client.18159 handle_command_reply: tid=0 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.237+0000 7efc71552740 1 client.18159 shutdown 2024-09-12T03:33:02.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 msgr2=0x557a67b24fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7efc50010410 tx=0x7efc5000ba30 comp rx=0 tx=0).stop 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 shutdown_connections 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b350c0 0x557a67b374b0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b22260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 --2- 172.21.15.19:0/3992324549 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b24fb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc47fff640 0 client.18159 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 >> 172.21.15.19:0/3992324549 conn(0x557a67b16a30 msgr2=0x557a67b1c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 shutdown_connections 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 1 -- 172.21.15.19:0/3992324549 wait complete. 2024-09-12T03:33:02.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.238+0000 7efc71552740 20 client.18159 trim_cache size 0 max 16384 2024-09-12T03:33:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 Processor -- start 2024-09-12T03:33:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 -- start start 2024-09-12T03:33:02.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b314b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc681af830 con 0x557a67b1adc0 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b15b0 con 0x557a67b1a4a0 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc71552740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc681aff00 con 0x557a67b25680 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39550/0 (socket says 172.21.15.19:39550) 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 -- 172.21.15.19:0/138728664 learned_addr learned my addr 172.21.15.19:0/138728664 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc6cf15640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b314b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc6d716640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 -- 172.21.15.19:0/138728664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 msgr2=0x557a67b3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 -- 172.21.15.19:0/138728664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 msgr2=0x557a67b314b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b314b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc67fff640 1 -- 172.21.15.19:0/138728664 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a67b164d0 con 0x557a67b1adc0 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc6d716640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.256+0000 7efc6cf15640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc67fff640 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efc580cc7c0 tx=0x7efc58005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b475d89e77ec7f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc5e7fc640 1 -- 172.21.15.19:0/138728664 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58020040 con 0x557a67b1adc0 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc5e7fc640 1 -- 172.21.15.19:0/138728664 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc580a4140 con 0x557a67b1adc0 2024-09-12T03:33:02.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc5e7fc640 1 -- 172.21.15.19:0/138728664 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc580ad0a0 con 0x557a67b1adc0 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc71552740 1 -- 172.21.15.19:0/138728664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 msgr2=0x557a67b21b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.257+0000 7efc71552740 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7efc580cc7c0 tx=0x7efc58005f90 comp rx=0 tx=0).stop 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 -- 172.21.15.19:0/138728664 shutdown_connections 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1adc0 0x557a67b21b40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- 172.21.15.19:0/138728664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1a4a0 0x557a67b314b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 -- 172.21.15.19:0/138728664 >> 172.21.15.19:0/138728664 conn(0x557a67b16a30 msgr2=0x557a67b1c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 -- 172.21.15.19:0/138728664 shutdown_connections 2024-09-12T03:33:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 -- 172.21.15.19:0/138728664 wait complete. 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 -- start start 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 Processor -- start 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b2de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.258+0000 7efc71552740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b1ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc71552740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b0e20 con 0x557a67b1a4a0 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc71552740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc681b0690 con 0x557a67b1adc0 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc71552740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc681ad750 con 0x557a67b25680 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc71552740 10 client.0 _ll_register_callbacks cb 0x7efc60009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39564/0 (socket says 172.21.15.19:39564) 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 -- 172.21.15.19:0/1592334819 learned_addr learned my addr 172.21.15.19:0/1592334819 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:02.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc67fff640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b2de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6d716640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b1ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 -- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 msgr2=0x557a67b1ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b1ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 -- 172.21.15.19:0/1592334819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 msgr2=0x557a67b2de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b2de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 -- 172.21.15.19:0/1592334819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a67b1f410 con 0x557a67b1a4a0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6d716640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc67fff640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b2de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc6cf15640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efc50004d20 tx=0x7efc5000b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b533d853e8bd93e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc47fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc50033070 con 0x557a67b1a4a0 2024-09-12T03:33:02.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efc50005a70 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.259+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc580a4140 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7efc50004640 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 10 client.18165 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557a67b9f6e0 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc580a4140 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7efc50011710 con 0x557a67b1a4a0 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 10 client.18165 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 20 client.18165 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b9ff10 0x557a67ba2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 4 client.18165 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a67a87d60 con 0x557a67b9ff10 2024-09-12T03:33:02.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.260+0000 7efc67fff640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b9ff10 0x557a67ba2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc47fff640 10 client.18165 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc67fff640 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b9ff10 0x557a67ba2330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9f6a65df87203b5 server_cookie=60d2563b9eef0b42 in_seq=0 out_seq=0 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc47fff640 1 -- 172.21.15.19:0/1592334819 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557a67a87d60 con 0x557a67b9ff10 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc47fff640 10 client.18165 handle_command_reply: tid=0 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 client.18165 shutdown 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 msgr2=0x557a67b2d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7efc50004d20 tx=0x7efc5000b3a0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 shutdown_connections 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557a67b25680 0x557a67b1ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x557a67b9ff10 0x557a67ba2330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557a67b1adc0 0x557a67b2de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 --2- 172.21.15.19:0/1592334819 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557a67b1a4a0 0x557a67b2d920 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 >> 172.21.15.19:0/1592334819 conn(0x557a67b16a30 msgr2=0x557a67b1bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.261+0000 7efc47fff640 0 client.18165 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 shutdown_connections 2024-09-12T03:33:02.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc71552740 1 -- 172.21.15.19:0/1592334819 wait complete. 2024-09-12T03:33:02.277 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc71552740 20 client.18165 trim_cache size 0 max 16384 2024-09-12T03:33:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:33:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:33:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:33:02.277 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 msgr2=0x7efc540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 0x7efc540a5040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efc60005e30 tx=0x7efc60009800 comp rx=0 tx=0).stop 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 msgr2=0x7efc68100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7efc5800d740 tx=0x7efc5800dc00 comp rx=0 tx=0).stop 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 shutdown_connections 2024-09-12T03:33:02.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc6810c180 0x7efc68101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc540a2b80 0x7efc540a5040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc68103670 0x7efc68100ef0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 --2- 172.21.15.19:0/1075271017 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc68102d50 0x7efc681009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.262+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 >> 172.21.15.19:0/1075271017 conn(0x7efc680fc7d0 msgr2=0x7efc680fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.263+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 shutdown_connections 2024-09-12T03:33:02.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:02.263+0000 7efc6f179640 1 -- 172.21.15.19:0/1075271017 wait complete. 2024-09-12T03:33:03.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:33:03.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:33:03.367 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:03.592 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:03 smithi019 ceph-mon[30953]: pgmap v757: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 444 op/s 2024-09-12T03:33:03.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:03 smithi121 ceph-mon[34963]: pgmap v757: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 444 op/s 2024-09-12T03:33:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:03 smithi135 ceph-mon[35474]: pgmap v757: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 444 op/s 2024-09-12T03:33:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 -- 172.21.15.19:0/1665223497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 msgr2=0x7f7438103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 --2- 172.21.15.19:0/1665223497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438103120 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f7434009900 tx=0x7f743402a480 comp rx=0 tx=0).stop 2024-09-12T03:33:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 -- 172.21.15.19:0/1665223497 shutdown_connections 2024-09-12T03:33:03.940 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 --2- 172.21.15.19:0/1665223497 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f743810c170 0x7f743810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 --2- 172.21.15.19:0/1665223497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7438103660 0x7f743810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 --2- 172.21.15.19:0/1665223497 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438103120 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 -- 172.21.15.19:0/1665223497 >> 172.21.15.19:0/1665223497 conn(0x7f74380fc7a0 msgr2=0x7f74380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 -- 172.21.15.19:0/1665223497 shutdown_connections 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.910+0000 7f743f2c2640 1 -- 172.21.15.19:0/1665223497 wait complete. 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 Processor -- start 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 -- start start 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 0x7f74381989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f743810c170 0x7f743819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.911+0000 7f743f2c2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7438111fb0 con 0x7f7438102d40 2024-09-12T03:33:03.941 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743f2c2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7438111e30 con 0x7f743810c170 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743f2c2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7438112130 con 0x7f7438103660 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39584/0 (socket says 172.21.15.19:39584) 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 -- 172.21.15.19:0/4238957143 learned_addr learned my addr 172.21.15.19:0/4238957143 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d85f640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f743810c170 0x7f743819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743c85d640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 0x7f74381989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 -- 172.21.15.19:0/4238957143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 msgr2=0x7f74381989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 0x7f74381989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 -- 172.21.15.19:0/4238957143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f743810c170 msgr2=0x7f743819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f743810c170 0x7f743819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 -- 172.21.15.19:0/4238957143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743819d420 con 0x7f7438102d40 2024-09-12T03:33:03.942 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743c85d640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 0x7f74381989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743d05e640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7434005ab0 tx=0x7f74340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743403f030 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7434003ce0 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743819d6b0 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434035070 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.912+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381a4ff0 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.913+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743806b070 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.915+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7434042030 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.916+0000 7f742e7fc640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.916+0000 7f743c85d640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.916+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f74340efc80 con 0x7f7438102d40 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.916+0000 7f743c85d640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f743810d500 tx=0x7f7428005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:03.943 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:03.919+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f74340bcbf0 con 0x7f7438102d40 2024-09-12T03:33:04.127 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.127+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7438107210 con 0x7f7438102d40 2024-09-12T03:33:04.128 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.128+0000 7f742e7fc640 1 -- 172.21.15.19:0/4238957143 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+12009 (secure 0 0 0) 0x7f74340bc4f0 con 0x7f7438102d40 2024-09-12T03:33:04.128 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:33:04.129 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"btime":"2024-09-12T03:32:19:237768+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:32:19.167870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:33:04.129 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 63 2024-09-12T03:33:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 msgr2=0x7f74240a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 0x7f74240a5040 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f743810d500 tx=0x7f7428005e30 comp rx=0 tx=0).stop 2024-09-12T03:33:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 msgr2=0x7f7438198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f7434005ab0 tx=0x7f74340042f0 comp rx=0 tx=0).stop 2024-09-12T03:33:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 shutdown_connections 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f743810c170 0x7f743819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7438103660 0x7f74381989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f74240a2b80 0x7f74240a5040 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 --2- 172.21.15.19:0/4238957143 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7438102d40 0x7f7438198470 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 >> 172.21.15.19:0/4238957143 conn(0x7f74380fc7a0 msgr2=0x7f74380fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 shutdown_connections 2024-09-12T03:33:04.131 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:04.130+0000 7f743f2c2640 1 -- 172.21.15.19:0/4238957143 wait complete. 2024-09-12T03:33:04.501 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:04 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4238957143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:04.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:04 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4238957143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:04.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:04 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4238957143' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:04.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:33:05.108 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:05.513 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:05 smithi019 ceph-mon[30953]: pgmap v758: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-09-12T03:33:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 -- 172.21.15.19:0/1003296396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 msgr2=0x7f88081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 --2- 172.21.15.19:0/1003296396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f88081067e0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009900 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-09-12T03:33:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 -- 172.21.15.19:0/1003296396 shutdown_connections 2024-09-12T03:33:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 --2- 172.21.15.19:0/1003296396 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808106f10 0x7f880810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.622 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 --2- 172.21.15.19:0/1003296396 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f88081067e0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 --2- 172.21.15.19:0/1003296396 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808101de0 0x7f88081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.600+0000 7f880f568640 1 -- 172.21.15.19:0/1003296396 >> 172.21.15.19:0/1003296396 conn(0x7f88080fd8c0 msgr2=0x7f88080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- 172.21.15.19:0/1003296396 shutdown_connections 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- 172.21.15.19:0/1003296396 wait complete. 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 Processor -- start 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- start start 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808101de0 0x7f88080744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 0x7f8808074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8808111fb0 con 0x7f8808102790 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8808111e30 con 0x7f8808101de0 2024-09-12T03:33:05.623 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.601+0000 7f880f568640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8808112130 con 0x7f8808106f10 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880db05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 0x7f8808074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39614/0 (socket says 172.21.15.19:39614) 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 -- 172.21.15.19:0/1597221939 learned_addr learned my addr 172.21.15.19:0/1597221939 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880d304640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808101de0 0x7f88080744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 -- 172.21.15.19:0/1597221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 msgr2=0x7f8808074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 0x7f8808074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 -- 172.21.15.19:0/1597221939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808101de0 msgr2=0x7f88080744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808101de0 0x7f88080744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 -- 172.21.15.19:0/1597221939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88081a6970 con 0x7f8808102790 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880db05640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 0x7f8808074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f880cb03640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.602+0000 7f87fe7fc640 1 -- 172.21.15.19:0/1597221939 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f803f030 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.603+0000 7f87fe7fc640 1 -- 172.21.15.19:0/1597221939 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f8003ce0 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.603+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88081a6c00 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.603+0000 7f87fe7fc640 1 -- 172.21.15.19:0/1597221939 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8035070 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.603+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88081a70e0 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.605+0000 7f87fe7fc640 1 -- 172.21.15.19:0/1597221939 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87f8042030 con 0x7f8808102790 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.605+0000 7f8811941740 1 Processor -- start 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 -- start start 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2d0b40 0x559bfa2cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bfa2352f0 con 0x559bfa2cf5d0 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559bfa1dccf0 con 0x559bfa2d0b40 2024-09-12T03:33:05.625 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f8811941740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559bfa2312b0 con 0x559bfa2da370 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880d304640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f87fe7fc640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880d304640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39616/0 (socket says 172.21.15.19:39616) 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880d304640 1 -- 172.21.15.19:0/2695427969 learned_addr learned my addr 172.21.15.19:0/2695427969 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880db05640 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2d0b40 0x559bfa2cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880db05640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f87fe7fc640 1 -- 172.21.15.19:0/1597221939 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87f80ef170 con 0x7f8808102790 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.606+0000 7f880cb03640 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 -- 172.21.15.19:0/2695427969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2d0b40 msgr2=0x559bfa2cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2d0b40 0x559bfa2cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 -- 172.21.15.19:0/2695427969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 msgr2=0x559bfa2cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.626 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 -- 172.21.15.19:0/2695427969 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfa1c13a0 con 0x559bfa2da370 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880cb03640 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2dc760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87f80eccd0 tx=0x7f87f802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88683d99c5a9b255 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f880db05640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 0x7f87f40a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8800008380 tx=0x7f8800009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f87f37fe640 1 -- 172.21.15.19:0/2695427969 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8049050 con 0x559bfa2da370 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f87f37fe640 1 -- 172.21.15.19:0/2695427969 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f80bc1a0 con 0x559bfa2da370 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.607+0000 7f87f37fe640 1 -- 172.21.15.19:0/2695427969 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b8110 con 0x559bfa2da370 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- 172.21.15.19:0/2695427969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 msgr2=0x559bfa2dc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2dc760 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f87f80eccd0 tx=0x7f87f802a410 comp rx=0 tx=0).stop 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- 172.21.15.19:0/2695427969 shutdown_connections 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2dc760 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 --2- 172.21.15.19:0/2695427969 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2d0b40 0x559bfa2cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- 172.21.15.19:0/2695427969 >> 172.21.15.19:0/2695427969 conn(0x559bfa298310 msgr2=0x559bfa2cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- 172.21.15.19:0/2695427969 shutdown_connections 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- 172.21.15.19:0/2695427969 wait complete. 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 -- start start 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.608+0000 7f8811941740 1 Processor -- start 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2d9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 0x559bfa2d7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x559bfa2312b0 con 0x559bfa2da370 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559bfa1f0060 con 0x559bfa2cf5d0 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x559bfa2dce30 con 0x559bfa2d0b40 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f8811941740 10 client.0 _ll_register_callbacks cb 0x559bf9f438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880cb03640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 0x559bfa2d7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39628/0 (socket says 172.21.15.19:39628) 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 -- 172.21.15.19:0/840255648 learned_addr learned my addr 172.21.15.19:0/840255648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880d304640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2d9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 -- 172.21.15.19:0/840255648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 msgr2=0x559bfa2d7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 0x559bfa2d7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 -- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 msgr2=0x559bfa2d9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2d9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880db05640 1 -- 172.21.15.19:0/840255648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bf9fa63c0 con 0x559bfa2da370 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880cb03640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 0x559bfa2d7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.609+0000 7f880d304640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2d9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f880db05640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f880001a650 tx=0x7f880001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee060a17e4ecf93a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800014040 con 0x559bfa2da370 2024-09-12T03:33:05.629 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f880000b020 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87f80bc1a0 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f880000b220 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 10 client.18183 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x559bfa2e8750 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87f80bc1a0 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f880002f690 con 0x559bfa2da370 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 10 client.18183 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 10 client.18183 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 20 client.18183 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa2e9ce0 0x559bfa2ec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 4 client.18183 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.610+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bfa242860 con 0x559bfa2e9ce0 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.611+0000 7f880d304640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa2e9ce0 0x559bfa2ec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.630 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.611+0000 7f87f0ff9640 10 client.18183 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.611+0000 7f880d304640 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa2e9ce0 0x559bfa2ec120 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=735faee450293a24 server_cookie=e514b0f3ce8b7e00 in_seq=0 out_seq=0 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f87f0ff9640 1 -- 172.21.15.19:0/840255648 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bfa242860 con 0x559bfa2e9ce0 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f87f0ff9640 10 client.18183 handle_command_reply: tid=0 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 client.18183 shutdown 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 msgr2=0x559bfa2d7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f880001a650 tx=0x7f880001a000 comp rx=0 tx=0).stop 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 shutdown_connections 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2da370 0x559bfa2d7610 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa2e9ce0 0x559bfa2ec120 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2d0b40 0x559bfa2d7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 --2- 172.21.15.19:0/840255648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2d9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.631 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 >> 172.21.15.19:0/840255648 conn(0x559bfa298310 msgr2=0x559bfa2d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f87f0ff9640 0 client.18183 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:05.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 shutdown_connections 2024-09-12T03:33:05.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 1 -- 172.21.15.19:0/840255648 wait complete. 2024-09-12T03:33:05.632 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.612+0000 7f8811941740 20 client.18183 trim_cache size 0 max 16384 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 Processor -- start 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 -- start start 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2e5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2da370 0x559bfa2d68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88081af800 con 0x559bfa2d0b40 2024-09-12T03:33:05.636 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b1580 con 0x559bfa2da370 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f8811941740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88081afed0 con 0x559bfa2cf5d0 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880cb03640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2e5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45028/0 (socket says 172.21.15.19:45028) 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 -- 172.21.15.19:0/3935883393 learned_addr learned my addr 172.21.15.19:0/3935883393 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880db05640 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2da370 0x559bfa2d68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 -- 172.21.15.19:0/3935883393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2da370 msgr2=0x559bfa2d68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2da370 0x559bfa2d68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 -- 172.21.15.19:0/3935883393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 msgr2=0x559bfa2e5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2e5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880d304640 1 -- 172.21.15.19:0/3935883393 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bfa2d5fd0 con 0x559bfa2cf5d0 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.626+0000 7f880cb03640 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:05.637 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f880d304640 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559bfa2ec990 tx=0x7f8804005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=330b63e51569c6da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f87f2ffd640 1 -- 172.21.15.19:0/3935883393 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804005220 con 0x559bfa2cf5d0 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f87f2ffd640 1 -- 172.21.15.19:0/3935883393 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804004c80 con 0x559bfa2cf5d0 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f87f2ffd640 1 -- 172.21.15.19:0/3935883393 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8804011b90 con 0x559bfa2cf5d0 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f8811941740 1 -- 172.21.15.19:0/3935883393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 msgr2=0x559bfa2cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f8811941740 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559bfa2ec990 tx=0x7f8804005f90 comp rx=0 tx=0).stop 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f8811941740 1 -- 172.21.15.19:0/3935883393 shutdown_connections 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.627+0000 7f8811941740 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2da370 0x559bfa2d68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2e5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 --2- 172.21.15.19:0/3935883393 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2cf5d0 0x559bfa2cbe10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- 172.21.15.19:0/3935883393 >> 172.21.15.19:0/3935883393 conn(0x559bfa298310 msgr2=0x559bfa2cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- 172.21.15.19:0/3935883393 shutdown_connections 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- 172.21.15.19:0/3935883393 wait complete. 2024-09-12T03:33:05.638 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- start start 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 Processor -- start 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2ddee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2de960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b0df0 con 0x559bfa2d0b40 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88081b0660 con 0x559bfa2cf5d0 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.628+0000 7f8811941740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f88081ad720 con 0x559bfa2da370 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f8811941740 10 client.0 _ll_register_callbacks cb 0x559bf9f438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:05.639 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39640/0 (socket says 172.21.15.19:39640) 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880db05640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2de960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 -- 172.21.15.19:0/49455741 learned_addr learned my addr 172.21.15.19:0/49455741 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880d304640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2ddee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 -- 172.21.15.19:0/49455741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 msgr2=0x559bfa2de960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2de960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 -- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 msgr2=0x559bfa2ddee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:05.640 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2ddee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 -- 172.21.15.19:0/49455741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bfa2dcad0 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880db05640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f880cb03640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87f8004650 tx=0x7f87f80ef540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faef7cd91621ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f87f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8075070 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f87f80d4be0 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8804004c80 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f87f8049020 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f8811941740 10 client.18189 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.629+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x559bfa354ba0 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8804004c80 con 0x559bfa2d0b40 2024-09-12T03:33:05.641 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f87f8078020 con 0x559bfa2d0b40 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 10 client.18189 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 10 client.18189 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 20 client.18189 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa355220 0x559bfa357640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 4 client.18189 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bfa23cd70 con 0x559bfa355220 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.630+0000 7f880d304640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa355220 0x559bfa357640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f87f0ff9640 10 client.18189 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f880d304640 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa355220 0x559bfa357640 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bc200180c210f88 server_cookie=1cb8d5ce62d6d84e in_seq=0 out_seq=0 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f87f0ff9640 1 -- 172.21.15.19:0/49455741 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f880402d020 con 0x559bfa355220 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f87f0ff9640 10 client.18189 handle_command_reply: tid=0 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 client.18189 shutdown 2024-09-12T03:33:05.642 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 msgr2=0x559bfa2de420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f87f8004650 tx=0x7f87f80ef540 comp rx=0 tx=0).stop 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 shutdown_connections 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x559bfa355220 0x559bfa357640 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x559bfa2da370 0x559bfa2de960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x559bfa2d0b40 0x559bfa2de420 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f87f0ff9640 0 client.18189 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 --2- 172.21.15.19:0/49455741 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559bfa2cf5d0 0x559bfa2ddee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 >> 172.21.15.19:0/49455741 conn(0x559bfa298310 msgr2=0x559bfa2cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 shutdown_connections 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 1 -- 172.21.15.19:0/49455741 wait complete. 2024-09-12T03:33:05.643 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.631+0000 7f8811941740 20 client.18189 trim_cache size 0 max 16384 2024-09-12T03:33:05.644 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 msgr2=0x7f87f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.644 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 0x7f87f40a5040 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8800008380 tx=0x7f8800009f90 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 msgr2=0x7f8808074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006010 tx=0x7f87f80042f0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 shutdown_connections 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8808106f10 0x7f8808074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f87f40a2b80 0x7f87f40a5040 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8808102790 0x7f8808074a30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 --2- 172.21.15.19:0/1597221939 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8808101de0 0x7f88080744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 >> 172.21.15.19:0/1597221939 conn(0x7f88080fd8c0 msgr2=0x7f88080feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:05.645 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 shutdown_connections 2024-09-12T03:33:05.646 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:05.632+0000 7f880f568640 1 -- 172.21.15.19:0/1597221939 wait complete. 2024-09-12T03:33:05.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:05 smithi121 ceph-mon[34963]: pgmap v758: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-09-12T03:33:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:05 smithi135 ceph-mon[35474]: pgmap v758: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 422 op/s 2024-09-12T03:33:06.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:33:06.687 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 -- 172.21.15.19:0/2768087658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c102790 msgr2=0x7f575c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 --2- 172.21.15.19:0/2768087658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c102790 0x7f575c1067e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f5750009900 tx=0x7f575002a410 comp rx=0 tx=0).stop 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 -- 172.21.15.19:0/2768087658 shutdown_connections 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 --2- 172.21.15.19:0/2768087658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c106f10 0x7f575c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 --2- 172.21.15.19:0/2768087658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c102790 0x7f575c1067e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 --2- 172.21.15.19:0/2768087658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 -- 172.21.15.19:0/2768087658 >> 172.21.15.19:0/2768087658 conn(0x7f575c0fd8c0 msgr2=0x7f575c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 -- 172.21.15.19:0/2768087658 shutdown_connections 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 -- 172.21.15.19:0/2768087658 wait complete. 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.194+0000 7f5761f5c640 1 Processor -- start 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 -- start start 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f575c111f70 con 0x7f575c106f10 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f575c111df0 con 0x7f575c102790 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f5761f5c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1120f0 con 0x7f575c101de0 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39654/0 (socket says 172.21.15.19:39654) 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575affd640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:57156/0 (socket says 172.21.15.19:57156) 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575bfff640 1 -- 172.21.15.19:0/581491444 learned_addr learned my addr 172.21.15.19:0/581491444 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 -- 172.21.15.19:0/581491444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 msgr2=0x7f575c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 -- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 msgr2=0x7f575c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575b7fe640 1 -- 172.21.15.19:0/581491444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575c1a69d0 con 0x7f575c101de0 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575affd640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.195+0000 7f575bfff640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:07.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f575b7fe640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c0744f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e9f0 tx=0x7f574c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f5758ff9640 1 -- 172.21.15.19:0/581491444 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c00ccc0 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575c1a6cc0 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f5758ff9640 1 -- 172.21.15.19:0/581491444 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f574c005020 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f5758ff9640 1 -- 172.21.15.19:0/581491444 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f574c0049e0 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.196+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f575c1a7180 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.198+0000 7f5764335740 1 Processor -- start 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.198+0000 7f5764335740 1 -- start start 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.198+0000 7f5758ff9640 1 -- 172.21.15.19:0/581491444 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f574c005560 con 0x7f575c101de0 2024-09-12T03:33:07.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91e5d0 0x55798a91ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a92b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55798a8842f0 con 0x55798a929370 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55798a82bcf0 con 0x55798a91e5d0 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5764335740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55798a8802b0 con 0x55798a91fb40 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45070/0 (socket says 172.21.15.19:45070) 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 -- 172.21.15.19:0/2475293235 learned_addr learned my addr 172.21.15.19:0/2475293235 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575affd640 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a92b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575b7fe640 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91e5d0 0x55798a91ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 -- 172.21.15.19:0/2475293235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91e5d0 msgr2=0x55798a91ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91e5d0 0x55798a91ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 -- 172.21.15.19:0/2475293235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 msgr2=0x55798a92b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a92b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 -- 172.21.15.19:0/2475293235 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798a8103a0 con 0x55798a91fb40 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f575bfff640 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5754010810 tx=0x7f5754010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d03fc86dd6039fe6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5758ff9640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 0x7f57440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.199+0000 7f5741ffb640 1 -- 172.21.15.19:0/2475293235 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400eb30 con 0x55798a91fb40 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5741ffb640 1 -- 172.21.15.19:0/2475293235 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754015070 con 0x55798a91fb40 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f575b7fe640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 0x7f57440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5741ffb640 1 -- 172.21.15.19:0/2475293235 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57540116a0 con 0x55798a91fb40 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5758ff9640 1 -- 172.21.15.19:0/581491444 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f574c0cef80 con 0x7f575c101de0 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 -- 172.21.15.19:0/2475293235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 msgr2=0x55798a91e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f575b7fe640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 0x7f57440a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f574c0cce00 tx=0x7f574c086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5754010810 tx=0x7f5754010cd0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 -- 172.21.15.19:0/2475293235 shutdown_connections 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a92b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91e5d0 0x55798a91ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 --2- 172.21.15.19:0/2475293235 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a91e090 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 -- 172.21.15.19:0/2475293235 >> 172.21.15.19:0/2475293235 conn(0x55798a8e7310 msgr2=0x55798a91ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 -- 172.21.15.19:0/2475293235 shutdown_connections 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.200+0000 7f5764335740 1 -- 172.21.15.19:0/2475293235 wait complete. 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 -- start start 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 Processor -- start 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91fb40 0x55798a926000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a9265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55798a92c0c0 con 0x55798a929370 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55798a83f060 con 0x55798a91fb40 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55798a92c240 con 0x55798a91e5d0 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f5764335740 10 client.0 _ll_register_callbacks cb 0x55798a5928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a9265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f575bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f575bfff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45084/0 (socket says 172.21.15.19:45084) 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f575bfff640 1 -- 172.21.15.19:0/3968872891 learned_addr learned my addr 172.21.15.19:0/3968872891 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.201+0000 7f575affd640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91fb40 0x55798a926000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 -- 172.21.15.19:0/3968872891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91fb40 msgr2=0x55798a926000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91fb40 0x55798a926000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 -- 172.21.15.19:0/3968872891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 msgr2=0x55798a9265d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a9265d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 -- 172.21.15.19:0/3968872891 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798a5f53c0 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f575bfff640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57540099e0 tx=0x7f5754000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1e13bcdace2859 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f573affd640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754020070 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5754003970 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5754015070 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f575400ee80 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f5764335740 10 client.27469 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55798a937b40 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.202+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754015070 con 0x55798a91e5d0 2024-09-12T03:33:07.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f57540117d0 con 0x55798a91e5d0 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 10 client.27469 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 10 client.27469 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 20 client.27469 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a938f50 0x55798a93b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 4 client.27469 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798a891860 con 0x55798a938f50 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f575affd640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a938f50 0x55798a93b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f573affd640 10 client.27469 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.203+0000 7f575affd640 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a938f50 0x55798a93b340 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1b422a39bcbc4061 server_cookie=4ba6accc28b551 in_seq=0 out_seq=0 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.204+0000 7f573affd640 1 -- 172.21.15.19:0/3968872891 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798a891860 con 0x55798a938f50 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.204+0000 7f573affd640 10 client.27469 handle_command_reply: tid=0 2024-09-12T03:33:07.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.204+0000 7f5764335740 1 client.27469 shutdown 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 msgr2=0x55798a928d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f57540099e0 tx=0x7f5754000f40 comp rx=0 tx=0).stop 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 shutdown_connections 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a938f50 0x55798a93b340 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a929370 0x55798a9265d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a91fb40 0x55798a926000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 --2- 172.21.15.19:0/3968872891 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91e5d0 0x55798a928d50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f573affd640 0 client.27469 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 >> 172.21.15.19:0/3968872891 conn(0x55798a8e7310 msgr2=0x55798a920960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 shutdown_connections 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 1 -- 172.21.15.19:0/3968872891 wait complete. 2024-09-12T03:33:07.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.205+0000 7f5764335740 20 client.27469 trim_cache size 0 max 16384 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.218+0000 7f5764335740 1 Processor -- start 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.218+0000 7f5764335740 1 -- start start 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a921120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a921660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1af880 con 0x55798a91e5d0 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1b1600 con 0x55798a929370 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f5764335740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1aff50 con 0x55798a91fb40 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39688/0 (socket says 172.21.15.19:39688) 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 -- 172.21.15.19:0/2622309815 learned_addr learned my addr 172.21.15.19:0/2622309815 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575affd640 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a921120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a921660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 -- 172.21.15.19:0/2622309815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 msgr2=0x55798a921120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a921120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 -- 172.21.15.19:0/2622309815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 msgr2=0x55798a921660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a921660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 -- 172.21.15.19:0/2622309815 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798a928970 con 0x55798a91e5d0 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.219+0000 7f575bfff640 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f5754010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f86fd711103da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f57417fa640 1 -- 172.21.15.19:0/2622309815 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754020070 con 0x55798a91e5d0 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f57417fa640 1 -- 172.21.15.19:0/2622309815 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754015040 con 0x55798a91e5d0 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f57417fa640 1 -- 172.21.15.19:0/2622309815 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575401a760 con 0x55798a91e5d0 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 -- 172.21.15.19:0/2622309815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 msgr2=0x55798a919c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f5754010690 comp rx=0 tx=0).stop 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 -- 172.21.15.19:0/2622309815 shutdown_connections 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a921660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a921120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 --2- 172.21.15.19:0/2622309815 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a919c10 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 -- 172.21.15.19:0/2622309815 >> 172.21.15.19:0/2622309815 conn(0x55798a8e7310 msgr2=0x55798a919500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 -- 172.21.15.19:0/2622309815 shutdown_connections 2024-09-12T03:33:07.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.220+0000 7f5764335740 1 -- 172.21.15.19:0/2622309815 wait complete. 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 -- start start 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 Processor -- start 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a92cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a92d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1b0e70 con 0x55798a91e5d0 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1b06e0 con 0x55798a929370 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f575c1ad7a0 con 0x55798a91fb40 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f5764335740 10 client.0 _ll_register_callbacks cb 0x7f5750009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f575affd640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a92cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f575bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39704/0 (socket says 172.21.15.19:39704) 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.221+0000 7f575bfff640 1 -- 172.21.15.19:0/3368516706 learned_addr learned my addr 172.21.15.19:0/3368516706 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 -- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 msgr2=0x55798a92cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a92cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 -- 172.21.15.19:0/3368516706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 msgr2=0x55798a92d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a92d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 -- 172.21.15.19:0/3368516706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798a92d810 con 0x55798a91e5d0 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575affd640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a92cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f575bfff640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5754009370 tx=0x7f5754003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fb630b865bdce7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f573affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754046060 con 0x55798a91e5d0 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f5754011650 con 0x55798a91e5d0 2024-09-12T03:33:07.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5754015040 con 0x55798a91e5d0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f5754020040 con 0x55798a91e5d0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f5764335740 10 client.18207 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.222+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55798a9a36e0 con 0x55798a91e5d0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754015040 con 0x55798a91e5d0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f5754003fa0 con 0x55798a91e5d0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 10 client.18207 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 10 client.18207 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 20 client.18207 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a9a3cd0 0x55798a9a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 4 client.18207 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55798a88bd70 con 0x55798a9a3cd0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f575affd640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a9a3cd0 0x55798a9a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:07.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f575affd640 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a9a3cd0 0x55798a9a60f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=971d88d6cb91f9b3 server_cookie=5a00ae325d579504 in_seq=0 out_seq=0 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.223+0000 7f573affd640 10 client.18207 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.224+0000 7f573affd640 1 -- 172.21.15.19:0/3368516706 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55798a88bd70 con 0x55798a9a3cd0 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.224+0000 7f573affd640 10 client.18207 handle_command_reply: tid=0 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.224+0000 7f5764335740 1 client.18207 shutdown 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 msgr2=0x55798a9319e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f5754009370 tx=0x7f5754003f00 comp rx=0 tx=0).stop 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 shutdown_connections 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x55798a9a3cd0 0x55798a9a60f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55798a929370 0x55798a92d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55798a91fb40 0x55798a92cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f573affd640 0 client.18207 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:07.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 --2- 172.21.15.19:0/3368516706 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55798a91e5d0 0x55798a9319e0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 >> 172.21.15.19:0/3368516706 conn(0x55798a8e7310 msgr2=0x55798a919d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 shutdown_connections 2024-09-12T03:33:07.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 1 -- 172.21.15.19:0/3368516706 wait complete. 2024-09-12T03:33:07.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.225+0000 7f5764335740 20 client.18207 trim_cache size 0 max 16384 2024-09-12T03:33:07.239 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 msgr2=0x7f57440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 0x7f57440a5040 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f574c0cce00 tx=0x7f574c086040 comp rx=0 tx=0).stop 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 msgr2=0x7f575c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c0744f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f574c00e9f0 tx=0x7f574c00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 shutdown_connections 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f57440a2b80 0x7f57440a5040 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f575c106f10 0x7f575c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f575c102790 0x7f575c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 --2- 172.21.15.19:0/581491444 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f575c101de0 0x7f575c0744f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:07.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 >> 172.21.15.19:0/581491444 conn(0x7f575c0fd8c0 msgr2=0x7f575c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:07.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 shutdown_connections 2024-09-12T03:33:07.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:07.226+0000 7f5761f5c640 1 -- 172.21.15.19:0/581491444 wait complete. 2024-09-12T03:33:07.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:07 smithi019 ceph-mon[30953]: pgmap v759: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 379 op/s 2024-09-12T03:33:07.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:07 smithi121 ceph-mon[34963]: pgmap v759: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 379 op/s 2024-09-12T03:33:07.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:07 smithi135 ceph-mon[35474]: pgmap v759: 97 pgs: 97 active+clean; 1.1 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 379 op/s 2024-09-12T03:33:08.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:33:08.343 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 -- 172.21.15.19:0/2413414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f861010bfc0 msgr2=0x7f861010e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 --2- 172.21.15.19:0/2413414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f861010bfc0 0x7f861010e460 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8608009ca0 tx=0x7f860802a950 comp rx=0 tx=0).stop 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 -- 172.21.15.19:0/2413414170 shutdown_connections 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 --2- 172.21.15.19:0/2413414170 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f861010bfc0 0x7f861010e460 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 --2- 172.21.15.19:0/2413414170 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f861010b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 --2- 172.21.15.19:0/2413414170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8610102b90 0x7f8610102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.011+0000 7f861540e640 1 -- 172.21.15.19:0/2413414170 >> 172.21.15.19:0/2413414170 conn(0x7f86100fc610 msgr2=0x7f86100fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.012+0000 7f861540e640 1 -- 172.21.15.19:0/2413414170 shutdown_connections 2024-09-12T03:33:09.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.012+0000 7f861540e640 1 -- 172.21.15.19:0/2413414170 wait complete. 2024-09-12T03:33:09.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.013+0000 7f861540e640 1 Processor -- start 2024-09-12T03:33:09.013 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.013+0000 7f861540e640 1 -- start start 2024-09-12T03:33:09.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 0x7f86101982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.014 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f8610198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.015 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 0x7f861019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111e00 con 0x7f8610102b90 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111c80 con 0x7f861010bfc0 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.014+0000 7f861540e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8610111f80 con 0x7f86101034b0 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 0x7f86101982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f8607fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f8610198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f8607fff640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f8610198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45122/0 (socket says 172.21.15.19:45122) 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f8607fff640 1 -- 172.21.15.19:0/528480110 learned_addr learned my addr 172.21.15.19:0/528480110 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f8614c0d640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 0x7f861019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 -- 172.21.15.19:0/528480110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 msgr2=0x7f8610198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.016 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f8610198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 -- 172.21.15.19:0/528480110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 msgr2=0x7f861019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 0x7f861019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.015+0000 7f860ffff640 1 -- 172.21.15.19:0/528480110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f861019d290 con 0x7f8610102b90 2024-09-12T03:33:09.017 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.016+0000 7f8614c0d640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 0x7f861019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.016+0000 7f860ffff640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 0x7f86101982e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.017+0000 7f860dffb640 1 -- 172.21.15.19:0/528480110 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860000ccc0 con 0x7f8610102b90 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.017+0000 7f860dffb640 1 -- 172.21.15.19:0/528480110 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8600010910 con 0x7f8610102b90 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.017+0000 7f860dffb640 1 -- 172.21.15.19:0/528480110 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86000189c0 con 0x7f8610102b90 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.017+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861019d580 con 0x7f8610102b90 2024-09-12T03:33:09.018 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.018+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101a4e60 con 0x7f8610102b90 2024-09-12T03:33:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.021+0000 7f8618a49740 1 Processor -- start 2024-09-12T03:33:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 -- start start 2024-09-12T03:33:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78fd0 0x5621eba7d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba7e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5621eb9d42f0 con 0x5621eba78620 2024-09-12T03:33:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5621eb97bcf0 con 0x5621eba78fd0 2024-09-12T03:33:09.023 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8618a49740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621eb9d02b0 con 0x5621eba7dc20 2024-09-12T03:33:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39728/0 (socket says 172.21.15.19:39728) 2024-09-12T03:33:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.022+0000 7f8614c0d640 1 -- 172.21.15.19:0/2609123464 learned_addr learned my addr 172.21.15.19:0/2609123464 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:09.024 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f860dffb640 1 -- 172.21.15.19:0/528480110 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8600010ac0 con 0x7f8610102b90 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8607fff640 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba7e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f860ffff640 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78fd0 0x5621eba7d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8614c0d640 1 -- 172.21.15.19:0/2609123464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 msgr2=0x5621eba7e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8614c0d640 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba7e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8614c0d640 1 -- 172.21.15.19:0/2609123464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78fd0 msgr2=0x5621eba7d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8614c0d640 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78fd0 0x5621eba7d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.023+0000 7f8614c0d640 1 -- 172.21.15.19:0/2609123464 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621eb9603a0 con 0x5621eba78620 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.024+0000 7f8614c0d640 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f86080098e0 tx=0x7f8608009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8f569f50c40d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.024+0000 7f86067fc640 1 -- 172.21.15.19:0/2609123464 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608034020 con 0x5621eba78620 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.024+0000 7f86067fc640 1 -- 172.21.15.19:0/2609123464 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608003e50 con 0x5621eba78620 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.024+0000 7f86067fc640 1 -- 172.21.15.19:0/2609123464 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860802c040 con 0x5621eba78620 2024-09-12T03:33:09.025 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.024+0000 7f860dffb640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 0x7f85fc0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f860ffff640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 0x7f85fc0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f860dffb640 1 -- 172.21.15.19:0/528480110 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f86000cf2b0 con 0x7f8610102b90 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 -- 172.21.15.19:0/2609123464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 msgr2=0x5621eba78a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f86080098e0 tx=0x7f8608009910 comp rx=0 tx=0).stop 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 -- 172.21.15.19:0/2609123464 shutdown_connections 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba7e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78fd0 0x5621eba7d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 --2- 172.21.15.19:0/2609123464 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78620 0x5621eba78a00 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.028 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.025+0000 7f8618a49740 1 -- 172.21.15.19:0/2609123464 >> 172.21.15.19:0/2609123464 conn(0x5621eba37310 msgr2=0x5621eba6a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.026+0000 7f8618a49740 1 -- 172.21.15.19:0/2609123464 shutdown_connections 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.026+0000 7f860ffff640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 0x7f85fc0a5190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86000cd130 tx=0x7f860000ce60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.026+0000 7f8618a49740 1 -- 172.21.15.19:0/2609123464 wait complete. 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.026+0000 7f8618a49740 1 -- start start 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 Processor -- start 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba83fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba81260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba817a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba83fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8607fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba81260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba83fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45146/0 (socket says 172.21.15.19:45146) 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8614c0d640 1 -- 172.21.15.19:0/3835289683 learned_addr learned my addr 172.21.15.19:0/3835289683 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:09.029 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5621eba849f0 con 0x5621eba78fd0 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5621eb98f060 con 0x5621eba7dc20 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5621eba84b70 con 0x5621eba78620 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8618a49740 10 client.0 _ll_register_callbacks cb 0x5621eb6e28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.027+0000 7f8614c0d640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba817a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 -- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 msgr2=0x5621eba83fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba83fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 -- 172.21.15.19:0/3835289683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 msgr2=0x5621eba81260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba81260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.030 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621eb7453c0 con 0x5621eba7dc20 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8614c0d640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba817a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8608009620 tx=0x7f860803d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b8278e8f7ee20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f85ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608059060 con 0x5621eba7dc20 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608003e50 con 0x5621eba7dc20 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8608030ac0 con 0x5621eba7dc20 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.028+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8608034040 con 0x5621eba7dc20 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.029+0000 7f8618a49740 10 client.27509 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:09.032 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.029+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5621eba90330 con 0x5621eba7dc20 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.029+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608003e50 con 0x5621eba7dc20 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8608051040 con 0x5621eba7dc20 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 10 client.27509 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 10 client.27509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 20 client.27509 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621eba918e0 0x5621eba93d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 4 client.27509 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621eb9e1860 con 0x5621eba918e0 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.030+0000 7f8607fff640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621eba918e0 0x5621eba93d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.031+0000 7f85ef7fe640 10 client.27509 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.031+0000 7f8607fff640 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621eba918e0 0x5621eba93d20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3db693ea44b236d server_cookie=5f638a0182661160 in_seq=0 out_seq=0 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.032+0000 7f85ef7fe640 1 -- 172.21.15.19:0/3835289683 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621eb9e1860 con 0x5621eba918e0 2024-09-12T03:33:09.033 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.032+0000 7f85ef7fe640 10 client.27509 handle_command_reply: tid=0 2024-09-12T03:33:09.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.032+0000 7f8618a49740 1 client.27509 shutdown 2024-09-12T03:33:09.034 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.032+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 msgr2=0x5621eba817a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.032+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba817a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8608009620 tx=0x7f860803d040 comp rx=0 tx=0).stop 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 shutdown_connections 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621eba918e0 0x5621eba93d20 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba817a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba81260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 --2- 172.21.15.19:0/3835289683 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba83fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 >> 172.21.15.19:0/3835289683 conn(0x5621eba37310 msgr2=0x5621eba77c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f85ef7fe640 0 client.27509 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 shutdown_connections 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.033+0000 7f8618a49740 1 -- 172.21.15.19:0/3835289683 wait complete. 2024-09-12T03:33:09.035 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.034+0000 7f8618a49740 20 client.27509 trim_cache size 0 max 16384 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 Processor -- start 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 -- start start 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 0x5621eba8dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba831d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ab240 con 0x5621eba78fd0 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86101acfc0 con 0x5621eba78620 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8618a49740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ab910 con 0x5621eba7dc20 2024-09-12T03:33:09.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8607fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8607fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:39754/0 (socket says 172.21.15.19:39754) 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8607fff640 1 -- 172.21.15.19:0/1266283658 learned_addr learned my addr 172.21.15.19:0/1266283658 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.057+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 0x5621eba8dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 -- 172.21.15.19:0/1266283658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 msgr2=0x5621eba831d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba831d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 -- 172.21.15.19:0/1266283658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 msgr2=0x5621eba6a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 -- 172.21.15.19:0/1266283658 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621eba7d640 con 0x5621eba78620 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8607fff640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:09.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 0x5621eba8dc90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f860802fad0 tx=0x7f860802a950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aff6bd7a2093ca server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8605ffb640 1 -- 172.21.15.19:0/1266283658 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608059070 con 0x5621eba78620 2024-09-12T03:33:09.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8605ffb640 1 -- 172.21.15.19:0/1266283658 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608064040 con 0x5621eba78620 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.058+0000 7f8605ffb640 1 -- 172.21.15.19:0/1266283658 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608063070 con 0x5621eba78620 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 -- 172.21.15.19:0/1266283658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 msgr2=0x5621eba8dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 0x5621eba8dc90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f860802fad0 tx=0x7f860802a950 comp rx=0 tx=0).stop 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 -- 172.21.15.19:0/1266283658 shutdown_connections 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba7dc20 0x5621eba831d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba6a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 --2- 172.21.15.19:0/1266283658 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba78620 0x5621eba8dc90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.059+0000 7f8618a49740 1 -- 172.21.15.19:0/1266283658 >> 172.21.15.19:0/1266283658 conn(0x5621eba37310 msgr2=0x5621eba82ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.060+0000 7f8618a49740 1 -- 172.21.15.19:0/1266283658 shutdown_connections 2024-09-12T03:33:09.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.060+0000 7f8618a49740 1 -- 172.21.15.19:0/1266283658 wait complete. 2024-09-12T03:33:09.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.061+0000 7f8618a49740 1 -- start start 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.061+0000 7f8618a49740 1 Processor -- start 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba8a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba98980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ac830 con 0x5621eba78fd0 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86101ac0a0 con 0x5621eba7dc20 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f86101a9160 con 0x5621eba78620 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8618a49740 10 client.0 _ll_register_callbacks cb 0x7f860802fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8614c0d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:45172/0 (socket says 172.21.15.19:45172) 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8614c0d640 1 -- 172.21.15.19:0/1225195419 learned_addr learned my addr 172.21.15.19:0/1225195419 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:09.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba98980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.062+0000 7f8607fff640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba8a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 -- 172.21.15.19:0/1225195419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 msgr2=0x5621eba98980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba98980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 -- 172.21.15.19:0/1225195419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 msgr2=0x5621eba8a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba8a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 -- 172.21.15.19:0/1225195419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621eba84320 con 0x5621eba78620 2024-09-12T03:33:09.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f8614c0d640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f860802b140 tx=0x7f860802f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3f64036483205a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.063+0000 7f85ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608063070 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8608004050 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8608064040 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8608059070 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f8618a49740 10 client.27490 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5621ebafbe20 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.064+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8608004d50 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8608054650 con 0x5621eba78620 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 10 client.27490 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 10 client.27490 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 20 client.27490 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:09.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621ebafc2f0 0x5621ebafe710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 4 client.27490 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8608064040 con 0x5621ebafc2f0 2024-09-12T03:33:09.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.065+0000 7f8607fff640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621ebafc2f0 0x5621ebafe710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8607fff640 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621ebafc2f0 0x5621ebafe710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=780118023f9bfb90 server_cookie=a3b2047069bd4d79 in_seq=0 out_seq=0 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f85ef7fe640 10 client.27490 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f85ef7fe640 1 -- 172.21.15.19:0/1225195419 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8608064040 con 0x5621ebafc2f0 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f85ef7fe640 10 client.27490 handle_command_reply: tid=0 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 client.27490 shutdown 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 msgr2=0x5621eba89fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f860802b140 tx=0x7f860802f980 comp rx=0 tx=0).stop 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 shutdown_connections 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5621ebafc2f0 0x5621ebafe710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5621eba7dc20 0x5621eba98980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5621eba78fd0 0x5621eba8a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 --2- 172.21.15.19:0/1225195419 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5621eba78620 0x5621eba89fe0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f85ef7fe640 0 client.27490 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 >> 172.21.15.19:0/1225195419 conn(0x5621eba37310 msgr2=0x5621eba77c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 shutdown_connections 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 1 -- 172.21.15.19:0/1225195419 wait complete. 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.066+0000 7f8618a49740 20 client.27490 trim_cache size 0 max 16384 2024-09-12T03:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 msgr2=0x7f85fc0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 0x7f85fc0a5190 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86000cd130 tx=0x7f860000ce60 comp rx=0 tx=0).stop 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 msgr2=0x7f86101982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 0x7f86101982e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f860000e9f0 tx=0x7f860000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 shutdown_connections 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f861010bfc0 0x7f861019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f86101034b0 0x7f8610198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f85fc0a2cd0 0x7f85fc0a5190 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 --2- 172.21.15.19:0/528480110 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8610102b90 0x7f86101982e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:09.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 >> 172.21.15.19:0/528480110 conn(0x7f86100fc610 msgr2=0x7f86100fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:09.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.067+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 shutdown_connections 2024-09-12T03:33:09.072 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:09.068+0000 7f861540e640 1 -- 172.21.15.19:0/528480110 wait complete. 2024-09-12T03:33:09.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:09 smithi121 ceph-mon[34963]: pgmap v760: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-12T03:33:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:09 smithi019 ceph-mon[30953]: pgmap v760: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-12T03:33:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:09 smithi135 ceph-mon[35474]: pgmap v760: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 341 op/s 2024-09-12T03:33:09.920 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:33:10.191 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:10.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 -- 172.21.15.19:0/1603962475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc4103660 msgr2=0x7f8bc410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 --2- 172.21.15.19:0/1603962475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc4103660 0x7f8bc410bb50 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4009900 tx=0x7f8bb402a480 comp rx=0 tx=0).stop 2024-09-12T03:33:10.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 -- 172.21.15.19:0/1603962475 shutdown_connections 2024-09-12T03:33:10.731 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 --2- 172.21.15.19:0/1603962475 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc410c170 0x7f8bc410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 --2- 172.21.15.19:0/1603962475 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc4103660 0x7f8bc410bb50 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 --2- 172.21.15.19:0/1603962475 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4102d40 0x7f8bc4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 -- 172.21.15.19:0/1603962475 >> 172.21.15.19:0/1603962475 conn(0x7f8bc40fc7a0 msgr2=0x7f8bc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 -- 172.21.15.19:0/1603962475 shutdown_connections 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 -- 172.21.15.19:0/1603962475 wait complete. 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.718+0000 7f8bcc194640 1 Processor -- start 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 -- start start 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc4102d40 0x7f8bc41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 0x7f8bc4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.732 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4111fb0 con 0x7f8bc410c170 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4111e30 con 0x7f8bc4102d40 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bcc194640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4112130 con 0x7f8bc4103660 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 0x7f8bc4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:39216/0 (socket says 172.21.15.19:39216) 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bc972f640 1 -- 172.21.15.19:0/2003798347 learned_addr learned my addr 172.21.15.19:0/2003798347 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc4102d40 0x7f8bc41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 -- 172.21.15.19:0/2003798347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 msgr2=0x7f8bc4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.733 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 -- 172.21.15.19:0/2003798347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc4102d40 msgr2=0x7f8bc41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc4102d40 0x7f8bc41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bca731640 1 -- 172.21.15.19:0/2003798347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc41aae40 con 0x7f8bc410c170 2024-09-12T03:33:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.719+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:10.734 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bca731640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 0x7f8bc4101420 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00be20 tx=0x7f8bbc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bbaffd640 1 -- 172.21.15.19:0/2003798347 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc019070 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc41ab070 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bbaffd640 1 -- 172.21.15.19:0/2003798347 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bbc014920 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bbaffd640 1 -- 172.21.15.19:0/2003798347 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc012920 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.720+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc41ab580 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.722+0000 7f8bbaffd640 1 -- 172.21.15.19:0/2003798347 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8bbc01c070 con 0x7f8bc410c170 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.722+0000 7f8bce56d740 1 Processor -- start 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.722+0000 7f8bce56d740 1 -- start start 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d30090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 0x5571c0d30a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.735 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571c0c962f0 con 0x5571c0d31b40 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571c0c3dcf0 con 0x5571c0d3b370 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bce56d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5571c0c922b0 con 0x5571c0d305d0 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d30090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 0x5571c0d30a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d30090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41740/0 (socket says 172.21.15.19:41740) 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc972f640 1 -- 172.21.15.19:0/3181515925 learned_addr learned my addr 172.21.15.19:0/3181515925 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc972f640 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.736 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 -- 172.21.15.19:0/3181515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 msgr2=0x5571c0d3d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d3d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 -- 172.21.15.19:0/3181515925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 msgr2=0x5571c0d30090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d30090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bc9f30640 1 -- 172.21.15.19:0/3181515925 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c0c223a0 con 0x5571c0d305d0 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.723+0000 7f8bbaffd640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 0x7f8bb00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 0x5571c0d30a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009c20 tx=0x7f8bc0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892fd9f0245d27a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 0x7f8bb00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8bbaffd640 1 -- 172.21.15.19:0/2003798347 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8bbc0ce900 con 0x7f8bc410c170 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8baffff640 1 -- 172.21.15.19:0/3181515925 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0004050 con 0x5571c0d305d0 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8baffff640 1 -- 172.21.15.19:0/3181515925 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0005500 con 0x5571c0d305d0 2024-09-12T03:33:10.737 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.724+0000 7f8baffff640 1 -- 172.21.15.19:0/3181515925 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00035f0 con 0x5571c0d305d0 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- 172.21.15.19:0/3181515925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 msgr2=0x5571c0d30a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 0x7f8bb00a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4031ba0 tx=0x7f8bb4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 0x5571c0d30a10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0009c20 tx=0x7f8bc0011810 comp rx=0 tx=0).stop 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- 172.21.15.19:0/3181515925 shutdown_connections 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d3d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d305d0 0x5571c0d30a10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 --2- 172.21.15.19:0/3181515925 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d30090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- 172.21.15.19:0/3181515925 >> 172.21.15.19:0/3181515925 conn(0x5571c0cf9310 msgr2=0x5571c0d2cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- 172.21.15.19:0/3181515925 shutdown_connections 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- 172.21.15.19:0/3181515925 wait complete. 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 -- start start 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.725+0000 7f8bce56d740 1 Processor -- start 2024-09-12T03:33:10.738 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 0x5571c0d37f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5571c0d3e070 con 0x5571c0d305d0 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5571c0c51060 con 0x5571c0d31b40 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5571c0d3e1f0 con 0x5571c0d3b370 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bce56d740 10 client.0 _ll_register_callbacks cb 0x5571c09a48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:10.739 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41744/0 (socket says 172.21.15.19:41744) 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 -- 172.21.15.19:0/1097343318 learned_addr learned my addr 172.21.15.19:0/1097343318 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 0x5571c0d37f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bca731640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 -- 172.21.15.19:0/1097343318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 msgr2=0x5571c0d38480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d38480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 -- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 msgr2=0x5571c0d37f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 0x5571c0d37f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc9f30640 1 -- 172.21.15.19:0/1097343318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c0a073c0 con 0x5571c0d305d0 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.726+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 0x5571c0d37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:10.740 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0004b80 tx=0x7f8bc0004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d16efd9c55e52eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc00044a0 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8bc00052d0 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bc0005500 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8bc00035f0 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 10 client.18237 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5571c0d49a60 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bc0005500 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8bc0018070 con 0x5571c0d305d0 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 10 client.18237 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 10 client.18237 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 20 client.18237 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:10.741 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0d4ae60 0x5571c0d4d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 4 client.18237 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.727+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571c0ca3860 con 0x5571c0d4ae60 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.728+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0d4ae60 0x5571c0d4d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.728+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0d4ae60 0x5571c0d4d250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c30e5f3abceb31a server_cookie=c2eb9d9f69c74563 in_seq=0 out_seq=0 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.728+0000 7f8bad7fa640 10 client.18237 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bad7fa640 1 -- 172.21.15.19:0/1097343318 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5571c0ca3860 con 0x5571c0d4ae60 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bad7fa640 10 client.18237 handle_command_reply: tid=0 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 client.18237 shutdown 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 msgr2=0x5571c0d3ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0004b80 tx=0x7f8bc0004ed0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 shutdown_connections 2024-09-12T03:33:10.742 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0d4ae60 0x5571c0d4d250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d38480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bad7fa640 0 client.18237 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d31b40 0x5571c0d37f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1097343318 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d3ac90 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 >> 172.21.15.19:0/1097343318 conn(0x5571c0cf9310 msgr2=0x5571c0d32930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 shutdown_connections 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 1 -- 172.21.15.19:0/1097343318 wait complete. 2024-09-12T03:33:10.743 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.729+0000 7f8bce56d740 20 client.18237 trim_cache size 0 max 16384 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 Processor -- start 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 -- start start 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 0x5571c0d47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d33660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b3b90 con 0x5571c0d31b40 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b5910 con 0x5571c0d305d0 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bce56d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b4260 con 0x5571c0d3b370 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41750/0 (socket says 172.21.15.19:41750) 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 -- 172.21.15.19:0/1053976975 learned_addr learned my addr 172.21.15.19:0/1053976975 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 0x5571c0d47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bca731640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d33660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 -- 172.21.15.19:0/1053976975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 msgr2=0x5571c0d33660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d33660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 -- 172.21.15.19:0/1053976975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 msgr2=0x5571c0d47340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 0x5571c0d47340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc972f640 1 -- 172.21.15.19:0/1053976975 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571c0d2d960 con 0x5571c0d31b40 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.743+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 0x5571c0d47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8bc972f640 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005ab0 tx=0x7f8bb4041070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed10cafe1f0fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8baf7fe640 1 -- 172.21.15.19:0/1053976975 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4035040 con 0x5571c0d31b40 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8baf7fe640 1 -- 172.21.15.19:0/1053976975 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4004c40 con 0x5571c0d31b40 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8baf7fe640 1 -- 172.21.15.19:0/1053976975 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4003cf0 con 0x5571c0d31b40 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8bce56d740 1 -- 172.21.15.19:0/1053976975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 msgr2=0x5571c0d33120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.744+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4005ab0 tx=0x7f8bb4041070 comp rx=0 tx=0).stop 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 -- 172.21.15.19:0/1053976975 shutdown_connections 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d3b370 0x5571c0d33660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d31b40 0x5571c0d33120 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 --2- 172.21.15.19:0/1053976975 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d305d0 0x5571c0d47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 -- 172.21.15.19:0/1053976975 >> 172.21.15.19:0/1053976975 conn(0x5571c0cf9310 msgr2=0x5571c0d37030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 -- 172.21.15.19:0/1053976975 shutdown_connections 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 -- 172.21.15.19:0/1053976975 wait complete. 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 -- start start 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.745+0000 7f8bce56d740 1 Processor -- start 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 0x5571c0d439f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d34570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b5180 con 0x5571c0d305d0 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b49f0 con 0x5571c0d3b370 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc41b1ab0 con 0x5571c0d31b40 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bce56d740 10 client.0 _ll_register_callbacks cb 0x7f8bc0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:10.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41752/0 (socket says 172.21.15.19:41752) 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 -- 172.21.15.19:0/2582449963 learned_addr learned my addr 172.21.15.19:0/2582449963 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bca731640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d34570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 0x5571c0d439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 -- 172.21.15.19:0/2582449963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 msgr2=0x5571c0d439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 0x5571c0d439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 -- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 msgr2=0x5571c0d34570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d34570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc9f30640 1 -- 172.21.15.19:0/2582449963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571c0d34ab0 con 0x5571c0d305d0 2024-09-12T03:33:10.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bca731640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.746+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 0x5571c0d439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bc9f30640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0004b80 tx=0x7f8bc0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51277024a3f7526 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0030a80 con 0x5571c0d305d0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8bc0004360 con 0x5571c0d305d0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb4004c40 con 0x5571c0d305d0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8bc0030480 con 0x5571c0d305d0 2024-09-12T03:33:10.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bce56d740 10 client.18243 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5571c0db5240 con 0x5571c0d305d0 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.747+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb4004c40 con 0x5571c0d305d0 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8bc0018040 con 0x5571c0d305d0 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 10 client.18243 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 10 client.18243 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 20 client.18243 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0db5a20 0x5571c0db7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 4 client.18243 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5571c0c9dd70 con 0x5571c0db5a20 2024-09-12T03:33:10.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0db5a20 0x5571c0db7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bc972f640 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0db5a20 0x5571c0db7e40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=983ffc36409132f8 server_cookie=b273a524c2f1ecb in_seq=0 out_seq=0 2024-09-12T03:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.748+0000 7f8bad7fa640 10 client.18243 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:10.755 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bad7fa640 1 -- 172.21.15.19:0/2582449963 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5571c0c9dd70 con 0x5571c0db5a20 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bad7fa640 10 client.18243 handle_command_reply: tid=0 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 client.18243 shutdown 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 msgr2=0x5571c0d434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0004b80 tx=0x7f8bc0011810 comp rx=0 tx=0).stop 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 shutdown_connections 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x5571c0db5a20 0x5571c0db7e40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5571c0d3b370 0x5571c0d34570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5571c0d31b40 0x5571c0d439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bad7fa640 0 client.18243 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:10.756 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 --2- 172.21.15.19:0/2582449963 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5571c0d305d0 0x5571c0d434b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 >> 172.21.15.19:0/2582449963 conn(0x5571c0cf9310 msgr2=0x5571c0d2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 shutdown_connections 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 1 -- 172.21.15.19:0/2582449963 wait complete. 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.749+0000 7f8bce56d740 20 client.18243 trim_cache size 0 max 16384 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:10.757 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 msgr2=0x7f8bb00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 0x7f8bb00a50c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4031ba0 tx=0x7f8bb4005a20 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 msgr2=0x7f8bc4101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 0x7f8bc4101420 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00be20 tx=0x7f8bbc00bf10 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 shutdown_connections 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8bb00a2c00 0x7f8bb00a50c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8bc410c170 0x7f8bc4101420 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8bc4103660 0x7f8bc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 --2- 172.21.15.19:0/2003798347 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8bc4102d40 0x7f8bc41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 >> 172.21.15.19:0/2003798347 conn(0x7f8bc40fc7a0 msgr2=0x7f8bc40fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 shutdown_connections 2024-09-12T03:33:10.758 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:10.750+0000 7f8bcc194640 1 -- 172.21.15.19:0/2003798347 wait complete. 2024-09-12T03:33:11.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:11 smithi019 ceph-mon[30953]: pgmap v761: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 653 KiB/s wr, 260 op/s 2024-09-12T03:33:11.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:33:11.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:11 smithi121 ceph-mon[34963]: pgmap v761: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 653 KiB/s wr, 260 op/s 2024-09-12T03:33:11.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:11 smithi135 ceph-mon[35474]: pgmap v761: 97 pgs: 97 active+clean; 1.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 653 KiB/s wr, 260 op/s 2024-09-12T03:33:11.818 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 -- 172.21.15.19:0/1483537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304106a80 msgr2=0x7f1304106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 --2- 172.21.15.19:0/1483537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304106a80 0x7f1304106e60 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009930 tx=0x7f12f802a420 comp rx=0 tx=0).stop 2024-09-12T03:33:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 -- 172.21.15.19:0/1483537457 shutdown_connections 2024-09-12T03:33:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 --2- 172.21.15.19:0/1483537457 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f130410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 --2- 172.21.15.19:0/1483537457 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304100a80 0x7f1304100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 --2- 172.21.15.19:0/1483537457 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304106a80 0x7f1304106e60 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 -- 172.21.15.19:0/1483537457 >> 172.21.15.19:0/1483537457 conn(0x7f13040fc7a0 msgr2=0x7f13040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.336+0000 7f130b04f640 1 -- 172.21.15.19:0/1483537457 shutdown_connections 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 -- 172.21.15.19:0/1483537457 wait complete. 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 Processor -- start 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 -- start start 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f13041997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304106a80 0x7f13041a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110e90 con 0x7f1304100a80 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1304110d10 con 0x7f1304106a80 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f130b04f640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f1304111010 con 0x7f1304101420 2024-09-12T03:33:12.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41778/0 (socket says 172.21.15.19:41778) 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.337+0000 7f1308deb640 1 -- 172.21.15.19:0/1302257164 learned_addr learned my addr 172.21.15.19:0/1302257164 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f13095ec640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304106a80 0x7f13041a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1303fff640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f13041997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 -- 172.21.15.19:0/1302257164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 msgr2=0x7f13041997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f13041997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 -- 172.21.15.19:0/1302257164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304106a80 msgr2=0x7f13041a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304106a80 0x7f13041a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 -- 172.21.15.19:0/1302257164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13041a4850 con 0x7f1304100a80 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1303fff640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f13041997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1308deb640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f12f802a940 tx=0x7f12f8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1301ffb640 1 -- 172.21.15.19:0/1302257164 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f803f070 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13041a4a20 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1301ffb640 1 -- 172.21.15.19:0/1302257164 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f8031c90 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f1301ffb640 1 -- 172.21.15.19:0/1302257164 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8035030 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.338+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13041a4eb0 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f1301ffb640 1 -- 172.21.15.19:0/1302257164 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12f8042070 con 0x7f1304100a80 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 Processor -- start 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 -- start start 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567cf5002f0 con 0x5567cf59a5d0 2024-09-12T03:33:12.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5567cf4a7cf0 con 0x5567cf59bb40 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f130d428740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5567cf4fc2b0 con 0x5567cf5a5370 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41792/0 (socket says 172.21.15.19:41792) 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f1308deb640 1 -- 172.21.15.19:0/102788286 learned_addr learned my addr 172.21.15.19:0/102788286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f13095ec640 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.341+0000 7f1303fff640 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 -- 172.21.15.19:0/102788286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 msgr2=0x5567cf5a7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 -- 172.21.15.19:0/102788286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 msgr2=0x5567cf59a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.350 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 -- 172.21.15.19:0/102788286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cf48c3a0 con 0x5567cf59a5d0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1301ffb640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 0x7f12f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f13095ec640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 0x7f12f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1308deb640 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f12f80f5c10 tx=0x7f12f80f5d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7331f069da777b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f1301ffb640 1 -- 172.21.15.19:0/1302257164 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12f80ef8d0 con 0x7f1304100a80 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f12eeffd640 1 -- 172.21.15.19:0/102788286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80b83e0 con 0x5567cf59a5d0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f12eeffd640 1 -- 172.21.15.19:0/102788286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f80bc840 con 0x5567cf59a5d0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.342+0000 7f12eeffd640 1 -- 172.21.15.19:0/102788286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80bbaa0 con 0x5567cf59a5d0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 -- 172.21.15.19:0/102788286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 msgr2=0x5567cf59aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f13095ec640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 0x7f12f40a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0092f0 tx=0x7f12fc009280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f12f80f5c10 tx=0x7f12f80f5d00 comp rx=0 tx=0).stop 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 -- 172.21.15.19:0/102788286 shutdown_connections 2024-09-12T03:33:12.351 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf59aa10 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 --2- 172.21.15.19:0/102788286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 -- 172.21.15.19:0/102788286 >> 172.21.15.19:0/102788286 conn(0x5567cf563310 msgr2=0x5567cf596c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 -- 172.21.15.19:0/102788286 shutdown_connections 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.343+0000 7f130d428740 1 -- 172.21.15.19:0/102788286 wait complete. 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 -- start start 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 Processor -- start 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 0x5567cf5a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5567cf5a7e30 con 0x5567cf59bb40 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5567cf4bb060 con 0x5567cf59a5d0 2024-09-12T03:33:12.352 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5567cf5a7fb0 con 0x5567cf5a5370 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f130d428740 10 client.0 _ll_register_callbacks cb 0x5567cf20e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f1303fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f1303fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41796/0 (socket says 172.21.15.19:41796) 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f1303fff640 1 -- 172.21.15.19:0/799133665 learned_addr learned my addr 172.21.15.19:0/799133665 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f13095ec640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.344+0000 7f1308deb640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 0x5567cf5a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 -- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 msgr2=0x5567cf5a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 -- 172.21.15.19:0/799133665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 msgr2=0x5567cf5a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 0x5567cf5a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.353 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 -- 172.21.15.19:0/799133665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cf2713c0 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1308deb640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 0x5567cf5a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f1303fff640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f12f000f670 tx=0x7f12f000fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e13b80348de629c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f12e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f0016070 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12f000b9d0 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f80bc840 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f12f000bbd0 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f130d428740 10 client.18255 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.345+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5567cf5b38f0 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f80bc840 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f12f001a520 con 0x5567cf59bb40 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 10 client.18255 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:12.354 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 10 client.18255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 20 client.18255 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf5b4e00 0x5567cf5b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 4 client.18255 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567cf50d860 con 0x5567cf5b4e00 2024-09-12T03:33:12.355 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.346+0000 7f1308deb640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf5b4e00 0x5567cf5b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f1308deb640 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf5b4e00 0x5567cf5b72c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5777f33baa2563aa server_cookie=53b772b092b8d4fd in_seq=0 out_seq=0 2024-09-12T03:33:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f12e3fff640 10 client.18255 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f12e3fff640 1 -- 172.21.15.19:0/799133665 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5567cf50d860 con 0x5567cf5b4e00 2024-09-12T03:33:12.356 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f12e3fff640 10 client.18255 handle_command_reply: tid=0 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 client.18255 shutdown 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 msgr2=0x5567cf5a2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f12f000f670 tx=0x7f12f000fa70 comp rx=0 tx=0).stop 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 shutdown_connections 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf5a5370 0x5567cf5a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59bb40 0x5567cf5a2080 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf5b4e00 0x5567cf5b72c0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 --2- 172.21.15.19:0/799133665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59a5d0 0x5567cf5a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f12e3fff640 0 client.18255 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.347+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 >> 172.21.15.19:0/799133665 conn(0x5567cf563310 msgr2=0x5567cf59c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.348+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 shutdown_connections 2024-09-12T03:33:12.357 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.348+0000 7f130d428740 1 -- 172.21.15.19:0/799133665 wait complete. 2024-09-12T03:33:12.358 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.348+0000 7f130d428740 20 client.18255 trim_cache size 0 max 16384 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 Processor -- start 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 -- start start 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59bb40 0x5567cf5bcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 0x5567cf5bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13041ad6d0 con 0x5567cf59a5d0 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13041af450 con 0x5567cf5a5370 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f130d428740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13041adda0 con 0x5567cf59bb40 2024-09-12T03:33:12.362 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41802/0 (socket says 172.21.15.19:41802) 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 -- 172.21.15.19:0/1216271563 learned_addr learned my addr 172.21.15.19:0/1216271563 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f13095ec640 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 0x5567cf5bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 -- 172.21.15.19:0/1216271563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59bb40 msgr2=0x5567cf5bcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59bb40 0x5567cf5bcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 -- 172.21.15.19:0/1216271563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 msgr2=0x5567cf5bd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 0x5567cf5bd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 -- 172.21.15.19:0/1216271563 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567cf5a2b80 con 0x5567cf59a5d0 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f13095ec640 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 0x5567cf5bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.361+0000 7f1308deb640 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f12f80bc0d0 tx=0x7f12f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8973c0c1771b14f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f12ee7fc640 1 -- 172.21.15.19:0/1216271563 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8049040 con 0x5567cf59a5d0 2024-09-12T03:33:12.363 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f12ee7fc640 1 -- 172.21.15.19:0/1216271563 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f80f53a0 con 0x5567cf59a5d0 2024-09-12T03:33:12.364 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f12ee7fc640 1 -- 172.21.15.19:0/1216271563 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f80bbc80 con 0x5567cf59a5d0 2024-09-12T03:33:12.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 -- 172.21.15.19:0/1216271563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 msgr2=0x5567cf5a1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f12f80bc0d0 tx=0x7f12f8009260 comp rx=0 tx=0).stop 2024-09-12T03:33:12.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 -- 172.21.15.19:0/1216271563 shutdown_connections 2024-09-12T03:33:12.367 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf5a5370 0x5567cf5bd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59bb40 0x5567cf5bcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 --2- 172.21.15.19:0/1216271563 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf59a5d0 0x5567cf5a1060 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 -- 172.21.15.19:0/1216271563 >> 172.21.15.19:0/1216271563 conn(0x5567cf563310 msgr2=0x5567cf5a0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 -- 172.21.15.19:0/1216271563 shutdown_connections 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.362+0000 7f130d428740 1 -- 172.21.15.19:0/1216271563 wait complete. 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 -- start start 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 Processor -- start 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59a5d0 0x5567cf5ada50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.368 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f13041aecc0 con 0x5567cf5a5370 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13041ae530 con 0x5567cf59bb40 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f13041ab5f0 con 0x5567cf59a5d0 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f130d428740 10 client.0 _ll_register_callbacks cb 0x7f12f000d340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f13095ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f13095ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41812/0 (socket says 172.21.15.19:41812) 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f13095ec640 1 -- 172.21.15.19:0/1559204147 learned_addr learned my addr 172.21.15.19:0/1559204147 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f1308deb640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59a5d0 0x5567cf5ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.369 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.363+0000 7f1303fff640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 -- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59a5d0 msgr2=0x5567cf5ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59a5d0 0x5567cf5ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 -- 172.21.15.19:0/1559204147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 msgr2=0x5567cf59eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 -- 172.21.15.19:0/1559204147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567cf59fca0 con 0x5567cf5a5370 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f13095ec640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f12fc012b20 tx=0x7f12fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af41a4734039310 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:12.370 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f12e3fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12fc02a440 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12fc01f450 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f80f53a0 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f12fc01f650 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f130d428740 10 client.18261 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.364+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5567cf61fc30 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12f80f53a0 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f12fc028070 con 0x5567cf5a5370 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 10 client.18261 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 10 client.18261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 20 client.18261 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf620470 0x5567cf622890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:12.371 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 4 client.18261 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5567cf507d70 con 0x5567cf620470 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f1308deb640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf620470 0x5567cf622890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f1308deb640 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf620470 0x5567cf622890 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634d0a6d209c789c server_cookie=30ea51520b592d2d in_seq=0 out_seq=0 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.365+0000 7f12e3fff640 10 client.18261 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f12e3fff640 1 -- 172.21.15.19:0/1559204147 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5567cf507d70 con 0x5567cf620470 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f12e3fff640 10 client.18261 handle_command_reply: tid=0 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 client.18261 shutdown 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 msgr2=0x5567cf59f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f12fc012b20 tx=0x7f12fc004a40 comp rx=0 tx=0).stop 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 shutdown_connections 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5567cf5a5370 0x5567cf59f470 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.372 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x5567cf620470 0x5567cf622890 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f12e3fff640 0 client.18261 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5567cf59bb40 0x5567cf59eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 --2- 172.21.15.19:0/1559204147 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5567cf59a5d0 0x5567cf5ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 >> 172.21.15.19:0/1559204147 conn(0x5567cf563310 msgr2=0x5567cf5a1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 shutdown_connections 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 1 -- 172.21.15.19:0/1559204147 wait complete. 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.366+0000 7f130d428740 20 client.18261 trim_cache size 0 max 16384 2024-09-12T03:33:12.373 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:12.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 msgr2=0x7f12f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 0x7f12f40a5040 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0092f0 tx=0x7f12fc009280 comp rx=0 tx=0).stop 2024-09-12T03:33:12.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 msgr2=0x7f1304199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:12.374 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f12f802a940 tx=0x7f12f8003b90 comp rx=0 tx=0).stop 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 shutdown_connections 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1304106a80 0x7f13041a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f1304101420 0x7f13041997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12f40a2b80 0x7f12f40a5040 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 --2- 172.21.15.19:0/1302257164 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f1304100a80 0x7f1304199290 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 >> 172.21.15.19:0/1302257164 conn(0x7f13040fc7a0 msgr2=0x7f13040fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 shutdown_connections 2024-09-12T03:33:12.375 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:12.367+0000 7f130b04f640 1 -- 172.21.15.19:0/1302257164 wait complete. 2024-09-12T03:33:13.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:13 smithi121 ceph-mon[34963]: pgmap v762: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1010 KiB/s wr, 302 op/s 2024-09-12T03:33:13.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:13 smithi019 ceph-mon[30953]: pgmap v762: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1010 KiB/s wr, 302 op/s 2024-09-12T03:33:13.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:13 smithi135 ceph-mon[35474]: pgmap v762: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1010 KiB/s wr, 302 op/s 2024-09-12T03:33:14.505 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:33:14.505 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:33:14.776 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2901664140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c810e480 msgr2=0x7f63c8110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2901664140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c810e480 0x7f63c8110870 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009900 tx=0x7f63c002a410 comp rx=0 tx=0).stop 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2901664140 shutdown_connections 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2901664140 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c810e480 0x7f63c8110870 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2901664140 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c8101280 0x7f63c810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2901664140 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2901664140 >> 172.21.15.19:0/2901664140 conn(0x7f63c80fc7a0 msgr2=0x7f63c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2901664140 shutdown_connections 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.300+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2901664140 wait complete. 2024-09-12T03:33:15.323 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 Processor -- start 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 -- start start 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c810e480 0x7f63c819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81141d0 con 0x7f63c8101280 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8114050 con 0x7f63c810e480 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cf6e5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c8114350 con 0x7f63c81008d0 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41836/0 (socket says 172.21.15.19:41836) 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63ccc80640 1 -- 172.21.15.19:0/2447396996 learned_addr learned my addr 172.21.15.19:0/2447396996 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.301+0000 7f63cdc82640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c810e480 0x7f63c819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63cd481640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 -- 172.21.15.19:0/2447396996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 msgr2=0x7f63c819a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c819a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 -- 172.21.15.19:0/2447396996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c810e480 msgr2=0x7f63c819ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c810e480 0x7f63c819ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 -- 172.21.15.19:0/2447396996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c819f660 con 0x7f63c8101280 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63cd481640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63ccc80640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d950 tx=0x7f63b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63be7fc640 1 -- 172.21.15.19:0/2447396996 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8017070 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63be7fc640 1 -- 172.21.15.19:0/2447396996 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b8009c30 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63be7fc640 1 -- 172.21.15.19:0/2447396996 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b800f6b0 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c819f950 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.302+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c81a7230 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.304+0000 7f63be7fc640 1 -- 172.21.15.19:0/2447396996 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63b80109d0 con 0x7f63c8101280 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 Processor -- start 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 -- start start 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b0fe480 0x55ba6b108800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b063280 con 0x55ba6b108d40 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b00ab10 con 0x55ba6b0fe480 2024-09-12T03:33:15.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63d1abe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b05f240 con 0x55ba6b0fdb60 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41840/0 (socket says 172.21.15.19:41840) 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63ccc80640 1 -- 172.21.15.19:0/15353910 learned_addr learned my addr 172.21.15.19:0/15353910 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.305+0000 7f63be7fc640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 0x7f63b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63cd481640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b0fe480 0x55ba6b108800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63cdc82640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63cdc82640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 0x7f63b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63be7fc640 1 -- 172.21.15.19:0/2447396996 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f63b80cecc0 con 0x7f63c8101280 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 -- 172.21.15.19:0/15353910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 msgr2=0x55ba6b0fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 -- 172.21.15.19:0/15353910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b0fe480 msgr2=0x55ba6b108800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b0fe480 0x55ba6b108800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.327 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 -- 172.21.15.19:0/15353910 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6b0171f0 con 0x55ba6b108d40 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63cdc82640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63ccc80640 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f63b80a3c70 tx=0x7f63b8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4770afd3d08d5d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63cdc82640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 0x7f63b40a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f63c0004770 tx=0x7f63c0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63b37fe640 1 -- 172.21.15.19:0/15353910 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80d0030 con 0x55ba6b108d40 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63b37fe640 1 -- 172.21.15.19:0/15353910 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b809bc30 con 0x55ba6b108d40 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.306+0000 7f63b37fe640 1 -- 172.21.15.19:0/15353910 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b8020070 con 0x55ba6b108d40 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- 172.21.15.19:0/15353910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 msgr2=0x55ba6b10b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f63b80a3c70 tx=0x7f63b8000bc0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- 172.21.15.19:0/15353910 shutdown_connections 2024-09-12T03:33:15.328 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b108d40 0x55ba6b10b130 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b0fe480 0x55ba6b108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 --2- 172.21.15.19:0/15353910 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- 172.21.15.19:0/15353910 >> 172.21.15.19:0/15353910 conn(0x55ba6b0fa0f0 msgr2=0x55ba6b0fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- 172.21.15.19:0/15353910 shutdown_connections 2024-09-12T03:33:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- 172.21.15.19:0/15353910 wait complete. 2024-09-12T03:33:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.307+0000 7f63d1abe740 1 -- start start 2024-09-12T03:33:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 Processor -- start 2024-09-12T03:33:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b105c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b106690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b10b8b0 con 0x55ba6b0fe480 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b01df10 con 0x55ba6b108d40 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55ba6b10ba30 con 0x55ba6b0fdb60 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63d1abe740 10 client.0 _ll_register_callbacks cb 0x55ba6afe6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41856/0 (socket says 172.21.15.19:41856) 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 -- 172.21.15.19:0/1146818626 learned_addr learned my addr 172.21.15.19:0/1146818626 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63cdc82640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b106690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63cd481640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b105c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.331 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 -- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 msgr2=0x55ba6b105c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b105c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 -- 172.21.15.19:0/1146818626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 msgr2=0x55ba6b106690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b106690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63ccc80640 1 -- 172.21.15.19:0/1146818626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6add46a0 con 0x55ba6b0fe480 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.308+0000 7f63cd481640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63ccc80640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f63b809b4c0 tx=0x7f63b809b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6aa7a43e9eb870 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b80d0020 con 0x55ba6b0fe480 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f63b809a980 con 0x55ba6b0fe480 2024-09-12T03:33:15.332 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63b809bc30 con 0x55ba6b0fe480 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f63b8020070 con 0x55ba6b0fe480 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 10 client.18273 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ba6b1174b0 con 0x55ba6b0fe480 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b809bc30 con 0x55ba6b0fe480 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f63b8010460 con 0x55ba6b0fe480 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 10 client.18273 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 10 client.18273 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 20 client.18273 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b118800 0x55ba6b11abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 4 client.18273 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:15.333 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.309+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba6b0707f0 con 0x55ba6b118800 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.310+0000 7f63cd481640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b118800 0x55ba6b11abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.310+0000 7f63cd481640 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b118800 0x55ba6b11abf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202e9afc26c86527 server_cookie=4c8da1651a88ab51 in_seq=0 out_seq=0 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.310+0000 7f63b0ff9640 10 client.18273 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63b0ff9640 1 -- 172.21.15.19:0/1146818626 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba6b0707f0 con 0x55ba6b118800 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63b0ff9640 10 client.18273 handle_command_reply: tid=0 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 client.18273 shutdown 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 msgr2=0x55ba6b106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f63b809b4c0 tx=0x7f63b809b620 comp rx=0 tx=0).stop 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 shutdown_connections 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b118800 0x55ba6b11abf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fe480 0x55ba6b106150 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63b0ff9640 0 client.18273 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 --2- 172.21.15.19:0/1146818626 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fdb60 0x55ba6b105c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 >> 172.21.15.19:0/1146818626 conn(0x55ba6b0fa0f0 msgr2=0x55ba6b0f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 shutdown_connections 2024-09-12T03:33:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 1 -- 172.21.15.19:0/1146818626 wait complete. 2024-09-12T03:33:15.335 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.311+0000 7f63d1abe740 20 client.18273 trim_cache size 0 max 16384 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 Processor -- start 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 -- start start 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b1201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b120700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.340 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81af830 con 0x55ba6b0fdb60 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b15b0 con 0x55ba6b108d40 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63d1abe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81aff00 con 0x55ba6b0fe480 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41868/0 (socket says 172.21.15.19:41868) 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 -- 172.21.15.19:0/3382557601 learned_addr learned my addr 172.21.15.19:0/3382557601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63ccc80640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b1201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cdc82640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 -- 172.21.15.19:0/3382557601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 msgr2=0x55ba6b1201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b1201c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 -- 172.21.15.19:0/3382557601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 msgr2=0x55ba6b120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b120700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 -- 172.21.15.19:0/3382557601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6b0fe2b0 con 0x55ba6b0fdb60 2024-09-12T03:33:15.341 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63ccc80640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b1201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.329+0000 7f63cd481640 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009740 tx=0x7f63c40098d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9922f4772afbfa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63b2ffd640 1 -- 172.21.15.19:0/3382557601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4026070 con 0x55ba6b0fdb60 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63b2ffd640 1 -- 172.21.15.19:0/3382557601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c400c390 con 0x55ba6b0fdb60 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63b2ffd640 1 -- 172.21.15.19:0/3382557601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c4017ad0 con 0x55ba6b0fdb60 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 -- 172.21.15.19:0/3382557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 msgr2=0x55ba6b0ffcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009740 tx=0x7f63c40098d0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 -- 172.21.15.19:0/3382557601 shutdown_connections 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b1201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 --2- 172.21.15.19:0/3382557601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b0ffcc0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 -- 172.21.15.19:0/3382557601 >> 172.21.15.19:0/3382557601 conn(0x55ba6b0fa0f0 msgr2=0x55ba6b0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 -- 172.21.15.19:0/3382557601 shutdown_connections 2024-09-12T03:33:15.342 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.330+0000 7f63d1abe740 1 -- 172.21.15.19:0/3382557601 wait complete. 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 -- start start 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 Processor -- start 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b10d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b0e20 con 0x55ba6b0fdb60 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81b0690 con 0x55ba6b108d40 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f63c81ad750 con 0x55ba6b0fe480 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63d1abe740 10 client.0 _ll_register_callbacks cb 0x7f63c4009770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63cd481640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63cd481640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41874/0 (socket says 172.21.15.19:41874) 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.331+0000 7f63cd481640 1 -- 172.21.15.19:0/791109648 learned_addr learned my addr 172.21.15.19:0/791109648 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:15.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63ccc80640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cdc82640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b10d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 -- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 msgr2=0x55ba6b10d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b10d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 -- 172.21.15.19:0/791109648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 msgr2=0x55ba6b10d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b10d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 -- 172.21.15.19:0/791109648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6b1822f0 con 0x55ba6b0fdb60 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cdc82640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63cd481640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f63c4024040 tx=0x7f63c4017ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bdf3e784b949a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63b0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63c402e060 con 0x55ba6b0fdb60 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f63c400a650 con 0x55ba6b0fdb60 2024-09-12T03:33:15.344 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63c400c390 con 0x55ba6b0fdb60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f63c4026070 con 0x55ba6b0fdb60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63d1abe740 10 client.18279 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.332+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55ba6b183540 con 0x55ba6b0fdb60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63c400c390 con 0x55ba6b0fdb60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f63c4021ad0 con 0x55ba6b0fdb60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 10 client.18279 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 10 client.18279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 20 client.18279 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b183c60 0x55ba6b186080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 4 client.18279 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ba6b06ad00 con 0x55ba6b183c60 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.333+0000 7f63ccc80640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b183c60 0x55ba6b186080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:15.345 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.334+0000 7f63ccc80640 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b183c60 0x55ba6b186080 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f627d17b886fa8 server_cookie=46f81736f7992a4b in_seq=0 out_seq=0 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.334+0000 7f63b0ff9640 10 client.18279 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.334+0000 7f63b0ff9640 1 -- 172.21.15.19:0/791109648 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ba6b06ad00 con 0x55ba6b183c60 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63b0ff9640 10 client.18279 handle_command_reply: tid=0 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 client.18279 shutdown 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 msgr2=0x55ba6b10cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f63c4024040 tx=0x7f63c4017ae0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 shutdown_connections 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55ba6b183c60 0x55ba6b186080 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55ba6b108d40 0x55ba6b10d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55ba6b0fe480 0x55ba6b10d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63b0ff9640 0 client.18279 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 --2- 172.21.15.19:0/791109648 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55ba6b0fdb60 0x55ba6b10cb60 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 >> 172.21.15.19:0/791109648 conn(0x55ba6b0fa0f0 msgr2=0x55ba6b0f9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.346 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 shutdown_connections 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 1 -- 172.21.15.19:0/791109648 wait complete. 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.335+0000 7f63d1abe740 20 client.18279 trim_cache size 0 max 16384 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "return_code": 0, 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_tag": "25c1796e-bc2a-434e-a19c-dbccfc6d0bd1", 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": "asynchronous" 2024-09-12T03:33:15.347 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 msgr2=0x7f63b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 0x7f63b40a5040 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f63c0004770 tx=0x7f63c0005fd0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 msgr2=0x7f63c819abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d950 tx=0x7f63b800de10 comp rx=0 tx=0).stop 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 shutdown_connections 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f63c810e480 0x7f63c819ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f63b40a2b80 0x7f63b40a5040 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f63c8101280 0x7f63c819abf0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 --2- 172.21.15.19:0/2447396996 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f63c81008d0 0x7f63c819a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 >> 172.21.15.19:0/2447396996 conn(0x7f63c80fc7a0 msgr2=0x7f63c80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 shutdown_connections 2024-09-12T03:33:15.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:15.336+0000 7f63cf6e5640 1 -- 172.21.15.19:0/2447396996 wait complete. 2024-09-12T03:33:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:15 smithi121 ceph-mon[34963]: pgmap v763: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 271 op/s 2024-09-12T03:33:15.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:15 smithi019 ceph-mon[30953]: pgmap v763: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 271 op/s 2024-09-12T03:33:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:15 smithi135 ceph-mon[35474]: pgmap v763: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 271 op/s 2024-09-12T03:33:16.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:33:16.620 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub queued for path: / 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.003 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:16 smithi135 ceph-mon[35474]: scrub summary: active paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub queued for path: / 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:16 smithi121 ceph-mon[34963]: scrub summary: active paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub queued for path: / 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:33:17.004 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:33:17.005 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.005 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:16 smithi019 ceph-mon[30953]: scrub summary: active paths [/] 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/1700663835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0106a90 msgr2=0x7fe8f0106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/1700663835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0106a90 0x7fe8f0106e70 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009930 tx=0x7fe8e002a460 comp rx=0 tx=0).stop 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/1700663835 shutdown_connections 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/1700663835 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/1700663835 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0100a90 0x7fe8f0100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/1700663835 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0106a90 0x7fe8f0106e70 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/1700663835 >> 172.21.15.19:0/1700663835 conn(0x7fe8f00fc7d0 msgr2=0x7fe8f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/1700663835 shutdown_connections 2024-09-12T03:33:17.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.226+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/1700663835 wait complete. 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 Processor -- start 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 -- start start 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0100a90 0x7fe8f0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 0x7fe8f019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0110f50 con 0x7fe8f0100a90 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f0110dd0 con 0x7fe8f0101430 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.228+0000 7fe8f4e8a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01110d0 con 0x7fe8f0106a90 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.229+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.229+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f0197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:49356/0 (socket says 172.21.15.19:49356) 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.229+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 0x7fe8f019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.229+0000 7fe8e7fff640 1 -- 172.21.15.19:0/981365440 learned_addr learned my addr 172.21.15.19:0/981365440 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0100a90 0x7fe8f0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 -- 172.21.15.19:0/981365440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 msgr2=0x7fe8f0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 -- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0100a90 msgr2=0x7fe8f0197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0100a90 0x7fe8f0197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 -- 172.21.15.19:0/981365440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019c4c0 con 0x7fe8f0106a90 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.230+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 0x7fe8f019bde0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ff40 tx=0x7fe8e800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.231+0000 7fe8e77fe640 1 -- 172.21.15.19:0/981365440 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800d0d0 con 0x7fe8f0106a90 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.231+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019c750 con 0x7fe8f0106a90 2024-09-12T03:33:17.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.231+0000 7fe8e77fe640 1 -- 172.21.15.19:0/981365440 <== mon.1 v2:172.21.15.135:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e800bba0 con 0x7fe8f0106a90 2024-09-12T03:33:17.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.231+0000 7fe8e77fe640 1 -- 172.21.15.19:0/981365440 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800d0d0 con 0x7fe8f0106a90 2024-09-12T03:33:17.233 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.232+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f006a790 con 0x7fe8f0106a90 2024-09-12T03:33:17.234 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.234+0000 7fe8e77fe640 1 -- 172.21.15.19:0/981365440 <== mon.1 v2:172.21.15.135:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8e800b6c0 con 0x7fe8f0106a90 2024-09-12T03:33:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.235+0000 7fe8e77fe640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 0x7fe8dc0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.235 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.235+0000 7fe8e77fe640 1 -- 172.21.15.19:0/981365440 <== mon.1 v2:172.21.15.135:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e8010cf0 con 0x7fe8f0106a90 2024-09-12T03:33:17.236 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.236+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 0x7fe8dc0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.237+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 0x7fe8dc0a54d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009230 tx=0x7fe8e00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.237+0000 7fe8f7263740 1 Processor -- start 2024-09-12T03:33:17.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 -- start start 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 0x55d27b7b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b55d0 0x55d27b7bf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d27b7122e0 con 0x55d27b7bfe90 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7c2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41914/0 (socket says 172.21.15.19:41914) 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8eed9d640 1 -- 172.21.15.19:0/119474848 learned_addr learned my addr 172.21.15.19:0/119474848 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d27b6b9ce0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.238+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d27b70e2a0 con 0x55d27b7b55d0 2024-09-12T03:33:17.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 0x55d27b7b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 -- 172.21.15.19:0/119474848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b55d0 msgr2=0x55d27b7bf950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b55d0 0x55d27b7bf950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 -- 172.21.15.19:0/119474848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 msgr2=0x55d27b7c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 -- 172.21.15.19:0/119474848 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27b69e440 con 0x55d27b7b4cb0 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.239+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 0x55d27b7b5090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005940 tx=0x7fe8e003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e3fd6e2e96bade4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.240+0000 7fe8e4ff9640 1 -- 172.21.15.19:0/119474848 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035040 con 0x55d27b7b4cb0 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.240+0000 7fe8e4ff9640 1 -- 172.21.15.19:0/119474848 <== mon.2 v2:172.21.15.121:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e004e070 con 0x55d27b7b4cb0 2024-09-12T03:33:17.241 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.240+0000 7fe8e4ff9640 1 -- 172.21.15.19:0/119474848 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0049c80 con 0x55d27b7b4cb0 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 msgr2=0x55d27b7b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 0x55d27b7b5090 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0005940 tx=0x7fe8e003d000 comp rx=0 tx=0).stop 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 shutdown_connections 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b55d0 0x55d27b7bf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 --2- 172.21.15.19:0/119474848 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b4cb0 0x55d27b7b5090 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.241+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 >> 172.21.15.19:0/119474848 conn(0x55d27b7b1240 msgr2=0x55d27b7b1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.242+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 shutdown_connections 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.242+0000 7fe8f7263740 1 -- 172.21.15.19:0/119474848 wait complete. 2024-09-12T03:33:17.243 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.243+0000 7fe8f7263740 1 -- start start 2024-09-12T03:33:17.244 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.243+0000 7fe8f7263740 1 Processor -- start 2024-09-12T03:33:17.246 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.243+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b55d0 0x55d27b7bcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7bfe90 0x55d27b7bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55d27b7c2a60 con 0x55d27b7b55d0 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55d27b6cd050 con 0x55d27b7bfe90 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55d27b7c2be0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8f7263740 10 client.0 _ll_register_callbacks cb 0x55d27b70d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.135:3300/0 says I am v2:172.21.15.19:39374/0 (socket says 172.21.15.19:39374) 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 -- 172.21.15.19:0/3089731005 learned_addr learned my addr 172.21.15.19:0/3089731005 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:17.247 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7bfe90 0x55d27b7bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 -- 172.21.15.19:0/3089731005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7bfe90 msgr2=0x55d27b7bd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7bfe90 0x55d27b7bd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 -- 172.21.15.19:0/3089731005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b55d0 msgr2=0x55d27b7bcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b55d0 0x55d27b7bcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.244+0000 7fe8ee59c640 1 -- 172.21.15.19:0/3089731005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27b4833c0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.245+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8e004b040 tx=0x7fe8e0003a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8dfe7f8e47139fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.245+0000 7fe8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.135:3300/0 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.245+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mon.1 v2:172.21.15.135:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035040 con 0x55d27b7b4cb0 2024-09-12T03:33:17.248 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.246+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mon.1 v2:172.21.15.135:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e005e680 con 0x55d27b7b4cb0 2024-09-12T03:33:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.246+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e004e070 con 0x55d27b7b4cb0 2024-09-12T03:33:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.246+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mon.1 v2:172.21.15.135:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe8e005e8c0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.246+0000 7fe8f7263740 10 client.27532 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.246+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d27b7c2a60 con 0x55d27b7b4cb0 2024-09-12T03:33:17.249 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mon.1 v2:172.21.15.135:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e0049480 con 0x55d27b7b4cb0 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mon.1 v2:172.21.15.135:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fe8e003c440 con 0x55d27b7b4cb0 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 10 client.27532 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 10 client.27532 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 20 client.27532 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b7d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 4 client.27532 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.247+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe8e004e070 con 0x55d27b7cf5f0 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.248+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b7d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.248+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b7d19e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef74915a5e171b6b server_cookie=744e342da3501d4 in_seq=0 out_seq=0 2024-09-12T03:33:17.252 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.248+0000 7fe8d5ffb640 10 client.27532 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.249+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/3089731005 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fe8e004e070 con 0x55d27b7cf5f0 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.249+0000 7fe8d5ffb640 10 client.27532 handle_command_reply: tid=0 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 client.27532 shutdown 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 msgr2=0x55d27b7bf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe8e004b040 tx=0x7fe8e0003a50 comp rx=0 tx=0).stop 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 shutdown_connections 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b7d19e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7bfe90 0x55d27b7bd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b55d0 0x55d27b7bcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8d5ffb640 0 client.27532 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 --2- 172.21.15.19:0/3089731005 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7bf840 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 >> 172.21.15.19:0/3089731005 conn(0x55d27b7b1240 msgr2=0x55d27b7a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.253 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.250+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 shutdown_connections 2024-09-12T03:33:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.251+0000 7fe8f7263740 1 -- 172.21.15.19:0/3089731005 wait complete. 2024-09-12T03:33:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.251+0000 7fe8f7263740 20 client.27532 trim_cache size 0 max 16384 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.277+0000 7fe8f7263740 1 Processor -- start 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.277+0000 7fe8f7263740 1 -- start start 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ab4c0 con 0x55d27b7bfe90 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ad240 con 0x55d27b7b55d0 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8f7263740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01abb90 con 0x55d27b7b4cb0 2024-09-12T03:33:17.278 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41934/0 (socket says 172.21.15.19:41934) 2024-09-12T03:33:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 -- 172.21.15.19:0/943440394 learned_addr learned my addr 172.21.15.19:0/943440394 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:17.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8e7fff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 -- 172.21.15.19:0/943440394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 msgr2=0x55d27b7a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 -- 172.21.15.19:0/943440394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 msgr2=0x55d27b7d71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7d71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 -- 172.21.15.19:0/943440394 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d27b7bbaa0 con 0x55d27b7bfe90 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:17.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.278+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80a72c0 tx=0x7fe8e809e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322a9c1fce363802 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.279+0000 7fe8d7fff640 1 -- 172.21.15.19:0/943440394 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80a77f0 con 0x55d27b7bfe90 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.279+0000 7fe8d7fff640 1 -- 172.21.15.19:0/943440394 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e800b9b0 con 0x55d27b7bfe90 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.279+0000 7fe8d7fff640 1 -- 172.21.15.19:0/943440394 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e809b1f0 con 0x55d27b7bfe90 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.279+0000 7fe8f7263740 1 -- 172.21.15.19:0/943440394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 msgr2=0x55d27b7d7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.279+0000 7fe8f7263740 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80a72c0 tx=0x7fe8e809e5a0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 -- 172.21.15.19:0/943440394 shutdown_connections 2024-09-12T03:33:17.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7bfe90 0x55d27b7d7710 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 --2- 172.21.15.19:0/943440394 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7b4cb0 0x55d27b7a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 -- 172.21.15.19:0/943440394 >> 172.21.15.19:0/943440394 conn(0x55d27b7b1240 msgr2=0x55d27b7a7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 -- 172.21.15.19:0/943440394 shutdown_connections 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 -- 172.21.15.19:0/943440394 wait complete. 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 -- start start 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 Processor -- start 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.280+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7c5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7bfe90 0x55d27b7c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01acab0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ac320 con 0x55d27b7b55d0 2024-09-12T03:33:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01a93e0 con 0x55d27b7bfe90 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:41940/0 (socket says 172.21.15.19:41940) 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 -- 172.21.15.19:0/2843538660 learned_addr learned my addr 172.21.15.19:0/2843538660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8f7263740 10 client.0 _ll_register_callbacks cb 0x7fe8e0031a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8eed9d640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7bfe90 0x55d27b7c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 -- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7bfe90 msgr2=0x55d27b7c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7bfe90 0x55d27b7c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 -- 172.21.15.19:0/2843538660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 msgr2=0x55d27b7c5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7c5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 -- 172.21.15.19:0/2843538660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d27b7b5400 con 0x55d27b7b4cb0 2024-09-12T03:33:17.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8ee59c640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004650 tx=0x7fe8e003c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f5d0071a8363c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.281+0000 7fe8d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035040 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e800b9b0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e005e690 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7fe8e007a050 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 10 client.18294 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55d27b83a030 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e800b9b0 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7fe8e0049860 con 0x55d27b7b4cb0 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 10 client.18294 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 10 client.18294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:17.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 20 client.18294 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b83c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 4 client.18294 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:33:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.282+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d27b71f850 con 0x55d27b7cf5f0 2024-09-12T03:33:17.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b83c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8e7fff640 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b83c760 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bde209d70e62eec2 server_cookie=8c78c086656ed603 in_seq=0 out_seq=0 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8d5ffb640 10 client.18294 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8d5ffb640 1 -- 172.21.15.19:0/2843538660 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d27b71f850 con 0x55d27b7cf5f0 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8d5ffb640 10 client.18294 handle_command_reply: tid=0 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8f7263740 1 client.18294 shutdown 2024-09-12T03:33:17.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 msgr2=0x55d27b7c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.283+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0004650 tx=0x7fe8e003c670 comp rx=0 tx=0).stop 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 shutdown_connections 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55d27b7bfe90 0x55d27b7c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55d27b7cf5f0 0x55d27b83c760 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55d27b7b55d0 0x55d27b7c5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 --2- 172.21.15.19:0/2843538660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55d27b7b4cb0 0x55d27b7c85e0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 >> 172.21.15.19:0/2843538660 conn(0x55d27b7b1240 msgr2=0x55d27b7a7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8d5ffb640 0 client.18294 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 shutdown_connections 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 1 -- 172.21.15.19:0/2843538660 wait complete. 2024-09-12T03:33:17.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.284+0000 7fe8f7263740 20 client.18294 trim_cache size 0 max 16384 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": { 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "25c1796e-bc2a-434e-a19c-dbccfc6d0bd1": { 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "path": "/", 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tag": "25c1796e-bc2a-434e-a19c-dbccfc6d0bd1", 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: "options": "recursive,force" 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:33:17.293 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:33:17.294 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:33:17.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 msgr2=0x7fe8dc0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 0x7fe8dc0a54d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009230 tx=0x7fe8e00058d0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 msgr2=0x7fe8f019bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 0x7fe8f019bde0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe8e800ff40 tx=0x7fe8e800ff70 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 shutdown_connections 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fe8f0106a90 0x7fe8f019bde0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe8f0101430 0x7fe8f0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fe8dc0a3010 0x7fe8dc0a54d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 --2- 172.21.15.19:0/981365440 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fe8f0100a90 0x7fe8f0197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 >> 172.21.15.19:0/981365440 conn(0x7fe8f00fc7d0 msgr2=0x7fe8f00fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 shutdown_connections 2024-09-12T03:33:17.295 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:17.285+0000 7fe8f4e8a640 1 -- 172.21.15.19:0/981365440 wait complete. 2024-09-12T03:33:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:17 smithi019 ceph-mon[30953]: pgmap v764: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 751 KiB/s wr, 236 op/s 2024-09-12T03:33:17.414 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:17 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:17 smithi121 ceph-mon[34963]: pgmap v764: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 751 KiB/s wr, 236 op/s 2024-09-12T03:33:17.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:17 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:17.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:17 smithi135 ceph-mon[35474]: pgmap v764: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 751 KiB/s rd, 751 KiB/s wr, 236 op/s 2024-09-12T03:33:17.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:17 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:18.196 INFO:tasks.cephfs.filesystem:scrub status for tag:25c1796e-bc2a-434e-a19c-dbccfc6d0bd1 - {'path': '/', 'tag': '25c1796e-bc2a-434e-a19c-dbccfc6d0bd1', 'options': 'recursive,force'} 2024-09-12T03:33:19.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:19 smithi121 ceph-mon[34963]: pgmap v765: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 907 KiB/s wr, 241 op/s 2024-09-12T03:33:19.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:19 smithi121 ceph-mon[34963]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:19 smithi019 ceph-mon[30953]: pgmap v765: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 907 KiB/s wr, 241 op/s 2024-09-12T03:33:19.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:19 smithi019 ceph-mon[30953]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:19 smithi135 ceph-mon[35474]: pgmap v765: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 907 KiB/s wr, 241 op/s 2024-09-12T03:33:19.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:19 smithi135 ceph-mon[35474]: scrub summary: idle+waiting paths [/] 2024-09-12T03:33:21.596 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:21 smithi135 ceph-mon[35474]: pgmap v766: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 710 KiB/s wr, 237 op/s 2024-09-12T03:33:21.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:21 smithi121 ceph-mon[34963]: pgmap v766: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 710 KiB/s wr, 237 op/s 2024-09-12T03:33:21.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:21 smithi019 ceph-mon[30953]: pgmap v766: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 710 KiB/s wr, 237 op/s 2024-09-12T03:33:23.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:23 smithi121 ceph-mon[34963]: pgmap v767: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-12T03:33:23.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:23 smithi019 ceph-mon[30953]: pgmap v767: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-12T03:33:23.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:23 smithi135 ceph-mon[35474]: pgmap v767: 97 pgs: 97 active+clean; 1.0 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 285 op/s 2024-09-12T03:33:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:33:24.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:24 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:33:24.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:33:24.665 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:24 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:33:24.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:33:24.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:24 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:33:25.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:25 smithi121 ceph-mon[34963]: pgmap v768: 97 pgs: 97 active+clean; 1017 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:33:25.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:25 smithi121 ceph-mon[34963]: scrub summary: idle 2024-09-12T03:33:25.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:25 smithi019 ceph-mon[30953]: pgmap v768: 97 pgs: 97 active+clean; 1017 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:33:25.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:25 smithi019 ceph-mon[30953]: scrub summary: idle 2024-09-12T03:33:25.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:25 smithi135 ceph-mon[35474]: pgmap v768: 97 pgs: 97 active+clean; 1017 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 1.0 MiB/s wr, 264 op/s 2024-09-12T03:33:25.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:25 smithi135 ceph-mon[35474]: scrub summary: idle 2024-09-12T03:33:27.529 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-09-12T03:33:27.529 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:33:27.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:27 smithi121 ceph-mon[34963]: pgmap v769: 97 pgs: 97 active+clean; 1005 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 863 KiB/s wr, 264 op/s 2024-09-12T03:33:27.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:27 smithi019 ceph-mon[30953]: pgmap v769: 97 pgs: 97 active+clean; 1005 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 863 KiB/s wr, 264 op/s 2024-09-12T03:33:27.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:27 smithi135 ceph-mon[35474]: pgmap v769: 97 pgs: 97 active+clean; 1005 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 863 KiB/s wr, 264 op/s 2024-09-12T03:33:27.841 DEBUG:teuthology.parallel:result is None 2024-09-12T03:33:27.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:33:27.889 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:33:27.889 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T03:33:27.904 INFO:tasks.quiescer:joining Quiescers 2024-09-12T03:33:27.904 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T03:33:27.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:33:28.217 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:28.870 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 -- 172.21.15.19:0/2849547081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74106f10 msgr2=0x7efc7410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 --2- 172.21.15.19:0/2849547081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74106f10 0x7efc7410ea90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009ca0 tx=0x7efc6c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 -- 172.21.15.19:0/2849547081 shutdown_connections 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 --2- 172.21.15.19:0/2849547081 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74106f10 0x7efc7410ea90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 --2- 172.21.15.19:0/2849547081 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74102790 0x7efc741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 --2- 172.21.15.19:0/2849547081 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 0x7efc741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 -- 172.21.15.19:0/2849547081 >> 172.21.15.19:0/2849547081 conn(0x7efc740fd8c0 msgr2=0x7efc740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 -- 172.21.15.19:0/2849547081 shutdown_connections 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.853+0000 7efc7c1db640 1 -- 172.21.15.19:0/2849547081 wait complete. 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 Processor -- start 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 -- start start 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 0x7efc740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.871 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 0x7efc74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efc74111f90 con 0x7efc74102790 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc74111e10 con 0x7efc74101de0 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc7c1db640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efc74112110 con 0x7efc74106f10 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc79776640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc79776640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:38246/0 (socket says 172.21.15.19:38246) 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc79776640 1 -- 172.21.15.19:0/3731705531 learned_addr learned my addr 172.21.15.19:0/3731705531 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.854+0000 7efc79f77640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 0x7efc740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc7a778640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 0x7efc74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 -- 172.21.15.19:0/3731705531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 msgr2=0x7efc74074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 0x7efc74074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 -- 172.21.15.19:0/3731705531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 msgr2=0x7efc740744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 0x7efc740744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 -- 172.21.15.19:0/3731705531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc741a69c0 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc7a778640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 0x7efc74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc79776640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efc7000b4a0 tx=0x7efc7000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70014070 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc70010af0 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc741a6cb0 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc70017d10 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.855+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc741a71a0 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.856+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc741021c0 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.858+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc70017070 con 0x7efc74102790 2024-09-12T03:33:28.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.858+0000 7efc6affd640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 0x7efc600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.859+0000 7efc79f77640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 0x7efc600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.859+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efc700ce7f0 con 0x7efc74102790 2024-09-12T03:33:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.859+0000 7efc79f77640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 0x7efc600a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efc6400bbe0 tx=0x7efc6400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:28.874 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:28.862+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc7009b700 con 0x7efc74102790 2024-09-12T03:33:29.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.077+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc74075a20 con 0x7efc74102790 2024-09-12T03:33:29.885 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:29 smithi135 ceph-mon[35474]: pgmap v770: 97 pgs: 97 active+clean; 988 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-12T03:33:29.885 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:29 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3731705531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:29.885 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:29 smithi121 ceph-mon[34963]: pgmap v770: 97 pgs: 97 active+clean; 988 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-12T03:33:29.885 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:29 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3731705531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:29.885 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.077+0000 7efc6affd640 1 -- 172.21.15.19:0/3731705531 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+12009 (secure 0 0 0) 0x7efc74075a20 con 0x7efc74102790 2024-09-12T03:33:29.886 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:33:29.886 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"btime":"2024-09-12T03:32:19:237768+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:32:19.167870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:33:29.886 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 63 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 msgr2=0x7efc600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 0x7efc600a5040 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efc6400bbe0 tx=0x7efc6400b390 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 msgr2=0x7efc74074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7efc7000b4a0 tx=0x7efc7000b960 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 shutdown_connections 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efc74106f10 0x7efc74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efc600a2b80 0x7efc600a5040 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efc74102790 0x7efc74074a30 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 --2- 172.21.15.19:0/3731705531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc74101de0 0x7efc740744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 >> 172.21.15.19:0/3731705531 conn(0x7efc740fd8c0 msgr2=0x7efc740fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 shutdown_connections 2024-09-12T03:33:29.887 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:29.080+0000 7efc7c1db640 1 -- 172.21.15.19:0/3731705531 wait complete. 2024-09-12T03:33:29.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:29 smithi019 ceph-mon[30953]: pgmap v770: 97 pgs: 97 active+clean; 988 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 347 op/s 2024-09-12T03:33:29.888 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:29 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3731705531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:30.231 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24487 quiesce db --cancel --all' 2024-09-12T03:33:30.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24487 quiesce db --cancel --all 2024-09-12T03:33:30.500 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 -- 172.21.15.19:0/4193244605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102570 msgr2=0x7faf901029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 --2- 172.21.15.19:0/4193244605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102570 0x7faf901029b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7faf84009900 tx=0x7faf8402a480 comp rx=0 tx=0).stop 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 -- 172.21.15.19:0/4193244605 shutdown_connections 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 --2- 172.21.15.19:0/4193244605 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102ef0 0x7faf9010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 --2- 172.21.15.19:0/4193244605 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102570 0x7faf901029b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 --2- 172.21.15.19:0/4193244605 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 0x7faf90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 -- 172.21.15.19:0/4193244605 >> 172.21.15.19:0/4193244605 conn(0x7faf900fc7d0 msgr2=0x7faf900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 -- 172.21.15.19:0/4193244605 shutdown_connections 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.042+0000 7faf94f41640 1 -- 172.21.15.19:0/4193244605 wait complete. 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 Processor -- start 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 -- start start 2024-09-12T03:33:31.058 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 0x7faf901984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102570 0x7faf90198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf90110fa0 con 0x7faf90102ef0 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf90110e20 con 0x7faf90102570 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf94f41640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf90111120 con 0x7faf90100e20 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.043+0000 7faf8ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37002/0 (socket says 172.21.15.19:37002) 2024-09-12T03:33:31.059 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 -- 172.21.15.19:0/3680625234 learned_addr learned my addr 172.21.15.19:0/3680625234 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8e59c640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 0x7faf901984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102570 0x7faf90198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 -- 172.21.15.19:0/3680625234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 msgr2=0x7faf901984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 0x7faf901984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 -- 172.21.15.19:0/3680625234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102570 msgr2=0x7faf90198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102570 0x7faf90198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 -- 172.21.15.19:0/3680625234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019d4a0 con 0x7faf90102ef0 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf8ed9d640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b970 tx=0x7faf8800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf7f7fe640 1 -- 172.21.15.19:0/3680625234 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88019070 con 0x7faf90102ef0 2024-09-12T03:33:31.060 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9019d790 con 0x7faf90102ef0 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf7f7fe640 1 -- 172.21.15.19:0/3680625234 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf88014440 con 0x7faf90102ef0 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf7f7fe640 1 -- 172.21.15.19:0/3680625234 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf880075d0 con 0x7faf90102ef0 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.044+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf901a5070 con 0x7faf90102ef0 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.047+0000 7faf7f7fe640 1 -- 172.21.15.19:0/3680625234 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faf8801c070 con 0x7faf90102ef0 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.047+0000 7faf9731a740 1 Processor -- start 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.047+0000 7faf9731a740 1 -- start start 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.047+0000 7faf9731a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61bb8c0 0x55f6c61bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.061 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf9731a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf9731a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 0x55f6c61cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf9731a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6c6120200 con 0x55f6c61bc1e0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf9731a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f6c60c6dc0 con 0x55f6c61bb8c0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf9731a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f6c611c1c0 con 0x55f6c61caff0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37012/0 (socket says 172.21.15.19:37012) 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 -- 172.21.15.19:0/3011998979 learned_addr learned my addr 172.21.15.19:0/3011998979 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 0x55f6c61cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61bb8c0 0x55f6c61bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf7f7fe640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 0x7faf780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.062 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8ed9d640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 0x7faf780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 -- 172.21.15.19:0/3011998979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 msgr2=0x55f6c61cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 0x55f6c61cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 -- 172.21.15.19:0/3011998979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61bb8c0 msgr2=0x55f6c61bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61bb8c0 0x55f6c61bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 -- 172.21.15.19:0/3011998979 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c5e91620 con 0x55f6c61bc1e0 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 0x55f6c61cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf7f7fe640 1 -- 172.21.15.19:0/3680625234 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faf880ce6e0 con 0x7faf90102ef0 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.048+0000 7faf8e59c640 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7faf8000ad00 tx=0x7faf8000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208b09790e0a4ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.063 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf7cff9640 1 -- 172.21.15.19:0/3011998979 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8001d070 con 0x55f6c61bc1e0 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf8ed9d640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 0x7faf780a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b940 tx=0x7faf88065b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf7cff9640 1 -- 172.21.15.19:0/3011998979 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf80018b20 con 0x55f6c61bc1e0 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf7cff9640 1 -- 172.21.15.19:0/3011998979 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80010b80 con 0x55f6c61bc1e0 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 -- 172.21.15.19:0/3011998979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 msgr2=0x55f6c61caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7faf8000ad00 tx=0x7faf8000f790 comp rx=0 tx=0).stop 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 -- 172.21.15.19:0/3011998979 shutdown_connections 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61caff0 0x55f6c61cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61caab0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 --2- 172.21.15.19:0/3011998979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61bb8c0 0x55f6c61bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.064 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.049+0000 7faf9731a740 1 -- 172.21.15.19:0/3011998979 >> 172.21.15.19:0/3011998979 conn(0x55f6c61b7f90 msgr2=0x55f6c61b8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- 172.21.15.19:0/3011998979 shutdown_connections 2024-09-12T03:33:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- 172.21.15.19:0/3011998979 wait complete. 2024-09-12T03:33:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- start start 2024-09-12T03:33:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 Processor -- start 2024-09-12T03:33:31.065 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61c8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55f6c61cdbc0 con 0x55f6c61bc1e0 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55f6c60bac50 con 0x55f6c61caff0 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55f6c61cdd40 con 0x55f6c61bb8c0 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf9731a740 10 client.0 _ll_register_callbacks cb 0x55f6c60ccba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.066 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37024/0 (socket says 172.21.15.19:37024) 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf8e59c640 1 -- 172.21.15.19:0/579100503 learned_addr learned my addr 172.21.15.19:0/579100503 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.050+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 -- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 msgr2=0x55f6c61ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 -- 172.21.15.19:0/579100503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 msgr2=0x55f6c61c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 -- 172.21.15.19:0/579100503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c61c8990 con 0x55f6c61bc1e0 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf8e59c640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faf8000a550 tx=0x7faf80019f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98256add20ae3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.067 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8001d040 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf80018b20 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faf80018740 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7faf80018940 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf9731a740 10 client.18312 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f6c61d97b0 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.051+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf80018b20 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7faf80020030 con 0x55f6c61bc1e0 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 10 client.18312 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:31.068 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 10 client.18312 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 20 client.18312 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c61dab60 0x55f6c61dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 4 client.18312 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c612d770 con 0x55f6c61dab60 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c61dab60 0x55f6c61dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf75ffb640 10 client.18312 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.052+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c61dab60 0x55f6c61dcf50 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=badbf5b2925229f5 server_cookie=b43521f547bf3781 in_seq=0 out_seq=0 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf75ffb640 1 -- 172.21.15.19:0/579100503 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6c612d770 con 0x55f6c61dab60 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf75ffb640 10 client.18312 handle_command_reply: tid=0 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 client.18312 shutdown 2024-09-12T03:33:31.069 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 msgr2=0x55f6c61c7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7faf8000a550 tx=0x7faf80019f90 comp rx=0 tx=0).stop 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 shutdown_connections 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c61dab60 0x55f6c61dcf50 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61c7c00 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf75ffb640 0 client.18312 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 --2- 172.21.15.19:0/579100503 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 >> 172.21.15.19:0/579100503 conn(0x55f6c61b7f90 msgr2=0x55f6c61bdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 shutdown_connections 2024-09-12T03:33:31.070 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.053+0000 7faf9731a740 1 -- 172.21.15.19:0/579100503 wait complete. 2024-09-12T03:33:31.071 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.054+0000 7faf9731a740 20 client.18312 trim_cache size 0 max 16384 2024-09-12T03:33:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.070+0000 7faf9731a740 1 Processor -- start 2024-09-12T03:33:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.070+0000 7faf9731a740 1 -- start start 2024-09-12T03:33:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.076 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61e2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ab410 con 0x55f6c61bc1e0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ad190 con 0x55f6c61caff0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf9731a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf901abae0 con 0x55f6c61bb8c0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37034/0 (socket says 172.21.15.19:37034) 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 -- 172.21.15.19:0/1355026054 learned_addr learned my addr 172.21.15.19:0/1355026054 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61e2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 -- 172.21.15.19:0/1355026054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 msgr2=0x55f6c61ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.077 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 -- 172.21.15.19:0/1355026054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 msgr2=0x55f6c61e2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61e2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 -- 172.21.15.19:0/1355026054 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c61c6150 con 0x55f6c61bc1e0 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.071+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf8000a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9471c1329ec113e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf77fff640 1 -- 172.21.15.19:0/1355026054 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8001d070 con 0x55f6c61bc1e0 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf77fff640 1 -- 172.21.15.19:0/1355026054 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf80028070 con 0x55f6c61bc1e0 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf77fff640 1 -- 172.21.15.19:0/1355026054 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80004d80 con 0x55f6c61bc1e0 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 -- 172.21.15.19:0/1355026054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 msgr2=0x55f6c61e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7faf80000c00 tx=0x7faf8000a580 comp rx=0 tx=0).stop 2024-09-12T03:33:31.078 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 -- 172.21.15.19:0/1355026054 shutdown_connections 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c61e2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bc1e0 0x55f6c61e25b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 --2- 172.21.15.19:0/1355026054 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bb8c0 0x55f6c61ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 -- 172.21.15.19:0/1355026054 >> 172.21.15.19:0/1355026054 conn(0x55f6c61b7f90 msgr2=0x55f6c61b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 -- 172.21.15.19:0/1355026054 shutdown_connections 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.072+0000 7faf9731a740 1 -- 172.21.15.19:0/1355026054 wait complete. 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 -- start start 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 Processor -- start 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bc1e0 0x55f6c61cf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.079 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c6244580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7faf901aca00 con 0x55f6c61bb8c0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ac270 con 0x55f6c61caff0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7faf901a9330 con 0x55f6c61bc1e0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf9731a740 10 client.0 _ll_register_callbacks cb 0x7faf8001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37046/0 (socket says 172.21.15.19:37046) 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 -- 172.21.15.19:0/1249196747 learned_addr learned my addr 172.21.15.19:0/1249196747 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bc1e0 0x55f6c61cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c6244580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.080 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 -- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bc1e0 msgr2=0x55f6c61cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bc1e0 0x55f6c61cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 -- 172.21.15.19:0/1249196747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 msgr2=0x55f6c6244580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c6244580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.073+0000 7faf8dd9b640 1 -- 172.21.15.19:0/1249196747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c61cff00 con 0x55f6c61bb8c0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf8dd9b640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7faf8402a9a0 tx=0x7faf80005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605670fa94ce82f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf84035040 con 0x55f6c61bb8c0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7faf840039c0 con 0x55f6c61bb8c0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf80028070 con 0x55f6c61bb8c0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7faf84003bc0 con 0x55f6c61bb8c0 2024-09-12T03:33:31.081 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 10 client.18318 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55f6c6245c70 con 0x55f6c61bb8c0 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf80028070 con 0x55f6c61bb8c0 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7faf84004930 con 0x55f6c61bb8c0 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 10 client.18318 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 10 client.18318 resolve_mds: validated gid 24487 aka daemon mds.i 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 20 client.18318 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c6246430 0x55f6c6248850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 4 client.18318 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T03:33:31.082 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.074+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55f6c6127c80 con 0x55f6c6246430 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c6246430 0x55f6c6248850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf75ffb640 10 client.18318 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf8e59c640 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c6246430 0x55f6c6248850 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bca6783bbb4580 server_cookie=15625f488531fc0b in_seq=0 out_seq=0 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf75ffb640 1 -- 172.21.15.19:0/1249196747 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55f6c6127c80 con 0x55f6c6246430 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf75ffb640 10 client.18318 handle_command_reply: tid=0 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.075+0000 7faf9731a740 1 client.18318 shutdown 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 msgr2=0x55f6c61cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7faf8402a9a0 tx=0x7faf80005e60 comp rx=0 tx=0).stop 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 shutdown_connections 2024-09-12T03:33:31.083 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x55f6c6246430 0x55f6c6248850 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55f6c61caff0 0x55f6c6244580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55f6c61bc1e0 0x55f6c61cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 --2- 172.21.15.19:0/1249196747 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55f6c61bb8c0 0x55f6c61cf0c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf75ffb640 0 client.18318 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 >> 172.21.15.19:0/1249196747 conn(0x55f6c61b7f90 msgr2=0x55f6c61bd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 shutdown_connections 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 1 -- 172.21.15.19:0/1249196747 wait complete. 2024-09-12T03:33:31.084 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf9731a740 20 client.18318 trim_cache size 0 max 16384 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 63, 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "leader": 24487, 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "set_version": 48, 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout: "sets": {} 2024-09-12T03:33:31.085 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:33:31.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 msgr2=0x7faf780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 0x7faf780a5040 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b940 tx=0x7faf88065b70 comp rx=0 tx=0).stop 2024-09-12T03:33:31.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 msgr2=0x7faf9019cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:31.086 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7faf8800b970 tx=0x7faf8800bd70 comp rx=0 tx=0).stop 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 shutdown_connections 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7faf780a2b80 0x7faf780a5040 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.076+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7faf90102ef0 0x7faf9019cdc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.077+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7faf90102570 0x7faf90198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.077+0000 7faf94f41640 1 --2- 172.21.15.19:0/3680625234 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7faf90100e20 0x7faf901984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.077+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 >> 172.21.15.19:0/3680625234 conn(0x7faf900fc7d0 msgr2=0x7faf9006a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.077+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 shutdown_connections 2024-09-12T03:33:31.087 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:31.077+0000 7faf94f41640 1 -- 172.21.15.19:0/3680625234 wait complete. 2024-09-12T03:33:31.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:31 smithi121 ceph-mon[34963]: pgmap v771: 97 pgs: 97 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:31.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:31 smithi019 ceph-mon[30953]: pgmap v771: 97 pgs: 97 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:31.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:31 smithi135 ceph-mon[35474]: pgmap v771: 97 pgs: 97 active+clean; 980 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 358 op/s 2024-09-12T03:33:31.810 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T03:33:31.810 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T03:33:31.822 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T03:33:33.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:33 smithi121 ceph-mon[34963]: pgmap v772: 97 pgs: 97 active+clean; 972 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 393 op/s 2024-09-12T03:33:33.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:33 smithi019 ceph-mon[30953]: pgmap v772: 97 pgs: 97 active+clean; 972 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 393 op/s 2024-09-12T03:33:33.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:33 smithi135 ceph-mon[35474]: pgmap v772: 97 pgs: 97 active+clean; 972 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 393 op/s 2024-09-12T03:33:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:35 smithi121 ceph-mon[34963]: pgmap v773: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-09-12T03:33:35.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:35 smithi019 ceph-mon[30953]: pgmap v773: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-09-12T03:33:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:35 smithi135 ceph-mon[35474]: pgmap v773: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 335 op/s 2024-09-12T03:33:37.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:37 smithi121 ceph-mon[34963]: pgmap v774: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 711 KiB/s wr, 275 op/s 2024-09-12T03:33:37.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:37 smithi019 ceph-mon[30953]: pgmap v774: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 711 KiB/s wr, 275 op/s 2024-09-12T03:33:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:37 smithi135 ceph-mon[35474]: pgmap v774: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 711 KiB/s wr, 275 op/s 2024-09-12T03:33:39.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:39 smithi121 ceph-mon[34963]: pgmap v775: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 710 KiB/s wr, 221 op/s 2024-09-12T03:33:39.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:39 smithi019 ceph-mon[30953]: pgmap v775: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 710 KiB/s wr, 221 op/s 2024-09-12T03:33:39.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:39 smithi135 ceph-mon[35474]: pgmap v775: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 710 KiB/s wr, 221 op/s 2024-09-12T03:33:41.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:41 smithi121 ceph-mon[34963]: pgmap v776: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 118 op/s 2024-09-12T03:33:41.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:41 smithi019 ceph-mon[30953]: pgmap v776: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 118 op/s 2024-09-12T03:33:41.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:41 smithi135 ceph-mon[35474]: pgmap v776: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 350 KiB/s wr, 118 op/s 2024-09-12T03:33:43.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:43 smithi121 ceph-mon[34963]: pgmap v777: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 80 op/s 2024-09-12T03:33:43.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:43 smithi019 ceph-mon[30953]: pgmap v777: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 80 op/s 2024-09-12T03:33:43.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:43 smithi135 ceph-mon[35474]: pgmap v777: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 349 KiB/s wr, 80 op/s 2024-09-12T03:33:45.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:45 smithi121 ceph-mon[34963]: pgmap v778: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T03:33:45.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:45 smithi019 ceph-mon[30953]: pgmap v778: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T03:33:45.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:45 smithi135 ceph-mon[35474]: pgmap v778: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-09-12T03:33:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:46.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:46 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:33:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:46.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:46 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:33:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:33:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:33:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:33:46.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:46 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' 2024-09-12T03:33:47.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:47 smithi121 ceph-mon[34963]: pgmap v779: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T03:33:47.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:47 smithi019 ceph-mon[30953]: pgmap v779: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T03:33:47.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:47 smithi135 ceph-mon[35474]: pgmap v779: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 8.5 KiB/s wr, 11 op/s 2024-09-12T03:33:48.197 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:33:48.472 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:49.261 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2364376949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 msgr2=0x7f6c84100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2364376949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c84100560 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009900 tx=0x7f6c7002a410 comp rx=0 tx=0).stop 2024-09-12T03:33:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2364376949 shutdown_connections 2024-09-12T03:33:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2364376949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 0x7f6c8410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2364376949 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c84100560 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2364376949 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 0x7f6c84106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2364376949 >> 172.21.15.19:0/2364376949 conn(0x7f6c84077230 msgr2=0x7f6c840fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2364376949 shutdown_connections 2024-09-12T03:33:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2364376949 wait complete. 2024-09-12T03:33:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.988+0000 7f6c8abc9640 1 Processor -- start 2024-09-12T03:33:49.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 -- start start 2024-09-12T03:33:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 0x7f6c84101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 0x7f6c84101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84112070 con 0x7f6c84100100 2024-09-12T03:33:49.264 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84111ef0 con 0x7f6c84100aa0 2024-09-12T03:33:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c8abc9640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841121f0 con 0x7f6c84106b80 2024-09-12T03:33:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36028/0 (socket says 172.21.15.19:36028) 2024-09-12T03:33:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 -- 172.21.15.19:0/2159455466 learned_addr learned my addr 172.21.15.19:0/2159455466 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:49.265 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c83fff640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 0x7f6c84101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c89166640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 0x7f6c84101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 -- 172.21.15.19:0/2159455466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 msgr2=0x7f6c84101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 0x7f6c84101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 -- 172.21.15.19:0/2159455466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 msgr2=0x7f6c84101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 0x7f6c84101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.266 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.989+0000 7f6c88965640 1 -- 172.21.15.19:0/2159455466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c841a6ac0 con 0x7f6c84100100 2024-09-12T03:33:49.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c88965640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800b4a0 tx=0x7f6c7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c81ffb640 1 -- 172.21.15.19:0/2159455466 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78014070 con 0x7f6c84100100 2024-09-12T03:33:49.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c841a6db0 con 0x7f6c84100100 2024-09-12T03:33:49.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c81ffb640 1 -- 172.21.15.19:0/2159455466 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c78010af0 con 0x7f6c84100100 2024-09-12T03:33:49.267 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c81ffb640 1 -- 172.21.15.19:0/2159455466 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c78017d10 con 0x7f6c84100100 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.990+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a7260 con 0x7f6c84100100 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.992+0000 7f6c81ffb640 1 -- 172.21.15.19:0/2159455466 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6c78017070 con 0x7f6c84100100 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.992+0000 7f6c8cfa2740 1 Processor -- start 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 -- start start 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31639880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.268 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b31644140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b3159f2e0 con 0x558b31644680 2024-09-12T03:33:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558b31546ce0 con 0x558b316394a0 2024-09-12T03:33:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b3159b2a0 con 0x558b31639dc0 2024-09-12T03:33:49.269 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36036/0 (socket says 172.21.15.19:36036) 2024-09-12T03:33:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 -- 172.21.15.19:0/3401354603 learned_addr learned my addr 172.21.15.19:0/3401354603 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c89166640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31639880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.270 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c88965640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b31644140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 -- 172.21.15.19:0/3401354603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 msgr2=0x558b31644140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b31644140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.271 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c81ffb640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 0x7f6c740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 -- 172.21.15.19:0/3401354603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 msgr2=0x558b31639880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31639880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.993+0000 7f6c83fff640 1 -- 172.21.15.19:0/3401354603 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b3152b440 con 0x558b31644680 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c89166640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31639880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c81ffb640 1 -- 172.21.15.19:0/2159455466 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c780ce3b0 con 0x7f6c84100100 2024-09-12T03:33:49.272 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c83fff640 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009900 tx=0x7f6c70031630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd82d78b3c2203c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c89166640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 0x7f6c740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c6effd640 1 -- 172.21.15.19:0/3401354603 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70040070 con 0x558b31644680 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c6effd640 1 -- 172.21.15.19:0/3401354603 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x558b31644680 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.994+0000 7f6c6effd640 1 -- 172.21.15.19:0/3401354603 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70035040 con 0x558b31644680 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/3401354603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 msgr2=0x558b31646a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.273 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009900 tx=0x7f6c70031630 comp rx=0 tx=0).stop 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c89166640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 0x7f6c740a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098b0 tx=0x7f6c7c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/3401354603 shutdown_connections 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31646a70 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b31644140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/3401354603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31639880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.274 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/3401354603 >> 172.21.15.19:0/3401354603 conn(0x558b31635a30 msgr2=0x558b31635e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/3401354603 shutdown_connections 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/3401354603 wait complete. 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 -- start start 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.995+0000 7f6c8cfa2740 1 Processor -- start 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31643f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.275 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b31641720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x558b31647250 con 0x558b31639dc0 2024-09-12T03:33:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x558b3155a050 con 0x558b316394a0 2024-09-12T03:33:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x558b316473d0 con 0x558b31644680 2024-09-12T03:33:49.276 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c8cfa2740 10 client.0 _ll_register_callbacks cb 0x558b3159a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:49.278 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:33:49.278 INFO:teuthology.orchestra.run.smithi019.stdout: "status": "no active scrubs running", 2024-09-12T03:33:49.278 INFO:teuthology.orchestra.run.smithi019.stdout: "scrubs": {} 2024-09-12T03:33:49.278 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:33:49.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c83fff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36042/0 (socket says 172.21.15.19:36042) 2024-09-12T03:33:49.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c83fff640 1 -- 172.21.15.19:0/220916291 learned_addr learned my addr 172.21.15.19:0/220916291 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:49.279 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31643f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c89166640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b31641720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 -- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 msgr2=0x558b31641720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b31641720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 -- 172.21.15.19:0/220916291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 msgr2=0x558b316411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c88965640 1 -- 172.21.15.19:0/220916291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b313103c0 con 0x558b316394a0 2024-09-12T03:33:49.280 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.996+0000 7f6c83fff640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c88965640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31643f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x558b31636890 tx=0x7f6c78017730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=404e8fe4ca58ce1c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mon.2 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7801e050 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mon.2 v2:172.21.15.121:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c78010ca0 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c70005020 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6c7809a5d0 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c8cfa2740 10 client.27599 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x558b31652b70 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.997+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mon.2 v2:172.21.15.121:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c70005020 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mon.2 v2:172.21.15.121:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f6c780d1c40 con 0x558b316394a0 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 10 client.27599 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:49.281 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 10 client.27599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 20 client.27599 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b31653fd0 0x558b31656450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 4 client.27599 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b315ac850 con 0x558b31653fd0 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c83fff640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b31653fd0 0x558b31656450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c63fff640 10 client.27599 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.998+0000 7f6c83fff640 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b31653fd0 0x558b31656450 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3014f79820c30e8e server_cookie=85bf47d80b10a12c in_seq=0 out_seq=0 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c63fff640 1 -- 172.21.15.19:0/220916291 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558b315ac850 con 0x558b31653fd0 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c63fff640 10 client.27599 handle_command_reply: tid=0 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 client.27599 shutdown 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 msgr2=0x558b31643f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.282 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31643f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x558b31636890 tx=0x7f6c78017730 comp rx=0 tx=0).stop 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 shutdown_connections 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b31641720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31639dc0 0x558b316411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b31653fd0 0x558b31656450 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/220916291 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31643f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c63fff640 0 client.27599 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:48.999+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 >> 172.21.15.19:0/220916291 conn(0x558b31635a30 msgr2=0x558b3163b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.000+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 shutdown_connections 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.000+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/220916291 wait complete. 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.000+0000 7f6c8cfa2740 20 client.27599 trim_cache size 0 max 16384 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 Processor -- start 2024-09-12T03:33:49.283 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 -- start start 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 0x558b31635e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b3163b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841af8d0 con 0x558b316394a0 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b1650 con 0x558b31639dc0 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841affa0 con 0x558b31644680 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36052/0 (socket says 172.21.15.19:36052) 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 -- 172.21.15.19:0/2988388166 learned_addr learned my addr 172.21.15.19:0/2988388166 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:49.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c89166640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b3163b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c83fff640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 0x558b31635e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 -- 172.21.15.19:0/2988388166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 msgr2=0x558b3163b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b3163b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.017+0000 7f6c88965640 1 -- 172.21.15.19:0/2988388166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 msgr2=0x558b31635e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c88965640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 0x558b31635e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c88965640 1 -- 172.21.15.19:0/2988388166 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b31640190 con 0x558b316394a0 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c89166640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b3163b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:49.285 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c83fff640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 0x558b31635e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c88965640 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6c7809ac00 tx=0x7f6c7809a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64da3b71d381f300 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/2988388166 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7801e070 con 0x558b316394a0 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/2988388166 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c780b7070 con 0x558b316394a0 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.018+0000 7f6c6e7fc640 1 -- 172.21.15.19:0/2988388166 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c780d1c40 con 0x558b316394a0 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/2988388166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 msgr2=0x558b316504b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6c7809ac00 tx=0x7f6c7809a0c0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/2988388166 shutdown_connections 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31644680 0x558b3163b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b31639dc0 0x558b31635e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.286 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/2988388166 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b316394a0 0x558b316504b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/2988388166 >> 172.21.15.19:0/2988388166 conn(0x558b31635a30 msgr2=0x558b3163aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/2988388166 shutdown_connections 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/2988388166 wait complete. 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 -- start start 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.019+0000 7f6c8cfa2740 1 Processor -- start 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31648390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b316488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.287 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b0ec0 con 0x558b31644680 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841b0730 con 0x558b316394a0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f6c841ad7f0 con 0x558b31639dc0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c8cfa2740 10 client.0 _ll_register_callbacks cb 0x7f6c7800b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:36068/0 (socket says 172.21.15.19:36068) 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 -- 172.21.15.19:0/4181130430 learned_addr learned my addr 172.21.15.19:0/4181130430 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c83fff640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b316488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c88965640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31648390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 -- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 msgr2=0x558b316488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b316488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 -- 172.21.15.19:0/4181130430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 msgr2=0x558b31648390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:49.288 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31648390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 -- 172.21.15.19:0/4181130430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b316bdd20 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c83fff640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b316488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c89166640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c013b70 tx=0x7f6c7c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23baf02ffcf2a5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.020+0000 7f6c63fff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c020db0 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f6c7c01e470 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c780b7070 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6c7c01e670 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 10 client.18336 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x558b316befc0 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c780b7070 con 0x558b31644680 2024-09-12T03:33:49.289 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f6c7c033020 con 0x558b31644680 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 10 client.18336 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 20 client.18336 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b316bf820 0x558b316c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 4 client.18336 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b315a6d60 con 0x558b316bf820 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.021+0000 7f6c88965640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b316bf820 0x558b316c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c88965640 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b316bf820 0x558b316c1c40 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bec5e38e68cc28c server_cookie=b66345040505c76e in_seq=0 out_seq=0 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c63fff640 10 client.18336 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c63fff640 1 -- 172.21.15.19:0/4181130430 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558b315a6d60 con 0x558b316bf820 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c63fff640 10 client.18336 handle_command_reply: tid=0 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 client.18336 shutdown 2024-09-12T03:33:49.290 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 msgr2=0x558b31648e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c013b70 tx=0x7f6c7c004ab0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 shutdown_connections 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x558b31644680 0x558b31648e10 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x558b31639dc0 0x558b316488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x558b316bf820 0x558b316c1c40 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 --2- 172.21.15.19:0/4181130430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x558b316394a0 0x558b31648390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 >> 172.21.15.19:0/4181130430 conn(0x558b31635a30 msgr2=0x558b31642a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.022+0000 7f6c63fff640 0 client.18336 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 shutdown_connections 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8cfa2740 1 -- 172.21.15.19:0/4181130430 wait complete. 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8cfa2740 20 client.18336 trim_cache size 0 max 16384 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 msgr2=0x7f6c740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.291 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 0x7f6c740a5040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098b0 tx=0x7f6c7c004a40 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 msgr2=0x7f6c841012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f6c7800b4a0 tx=0x7f6c7800b960 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 shutdown_connections 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f6c84106b80 0x7f6c84101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6c84100aa0 0x7f6c84101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f6c740a2b80 0x7f6c740a5040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 --2- 172.21.15.19:0/2159455466 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f6c84100100 0x7f6c841012e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 >> 172.21.15.19:0/2159455466 conn(0x7f6c84077230 msgr2=0x7f6c84104c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 shutdown_connections 2024-09-12T03:33:49.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:49.023+0000 7f6c8abc9640 1 -- 172.21.15.19:0/2159455466 wait complete. 2024-09-12T03:33:49.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:49 smithi121 ceph-mon[34963]: pgmap v780: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T03:33:49.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:49 smithi019 ceph-mon[30953]: pgmap v780: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T03:33:49.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:49 smithi135 ceph-mon[35474]: pgmap v780: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-09-12T03:33:50.224 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:33:50.224 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:33:50.486 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 -- 172.21.15.19:0/2025489293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 msgr2=0x7f9e7410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2025489293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7410da40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009ca0 tx=0x7f9e6c02a950 comp rx=0 tx=0).stop 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 -- 172.21.15.19:0/2025489293 shutdown_connections 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2025489293 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7410da40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2025489293 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74102570 0x7f9e741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2025489293 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74100e20 0x7f9e74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 -- 172.21.15.19:0/2025489293 >> 172.21.15.19:0/2025489293 conn(0x7f9e740fc7d0 msgr2=0x7f9e740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 -- 172.21.15.19:0/2025489293 shutdown_connections 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.988+0000 7f9e7a848640 1 -- 172.21.15.19:0/2025489293 wait complete. 2024-09-12T03:33:51.009 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 Processor -- start 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 -- start start 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74100e20 0x7f9e741984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74102570 0x7f9e74198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74110fa0 con 0x7f9e74102ef0 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74110e20 con 0x7f9e74100e20 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e7a848640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f9e74111120 con 0x7f9e74102570 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e78de5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e78de5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57864/0 (socket says 172.21.15.19:57864) 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.989+0000 7f9e78de5640 1 -- 172.21.15.19:0/2853242156 learned_addr learned my addr 172.21.15.19:0/2853242156 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e73fff640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74100e20 0x7f9e741984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:51.010 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e737fe640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74102570 0x7f9e74198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 -- 172.21.15.19:0/2853242156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74102570 msgr2=0x7f9e74198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74102570 0x7f9e74198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 -- 172.21.15.19:0/2853242156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74100e20 msgr2=0x7f9e741984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74100e20 0x7f9e741984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 -- 172.21.15.19:0/2853242156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7419d490 con 0x7f9e74102ef0 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e78de5640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c02ae70 tx=0x7f9e6c02f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c034040 con 0x7f9e74102ef0 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7419d720 con 0x7f9e74102ef0 2024-09-12T03:33:51.011 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e6c030920 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c045480 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.990+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741a5060 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.991+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74101200 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.993+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e6c037020 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.994+0000 7f9e717fa640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 0x7f9e5c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.994+0000 7f9e73fff640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 0x7f9e5c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.994+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f9e6c0f01d0 con 0x7f9e74102ef0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.994+0000 7f9e73fff640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 0x7f9e5c0a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009890 tx=0x7f9e64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:51.012 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:50.997+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9e6c0811b0 con 0x7f9e74102ef0 2024-09-12T03:33:51.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.209+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e7410a0a0 con 0x7f9e74102ef0 2024-09-12T03:33:51.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.210+0000 7f9e717fa640 1 -- 172.21.15.19:0/2853242156 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+12009 (secure 0 0 0) 0x7f9e6c0bd0e0 con 0x7f9e74102ef0 2024-09-12T03:33:51.214 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:33:51.214 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":63,"btime":"2024-09-12T03:32:19:237768+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:32:19.167870+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24487,24467,24463,24421,24407]},"id":1}]} 2024-09-12T03:33:51.214 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 63 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 msgr2=0x7f9e5c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 0x7f9e5c0a5040 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9e64009890 tx=0x7f9e64008040 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 msgr2=0x7f9e7419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c02ae70 tx=0x7f9e6c02f5d0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 shutdown_connections 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f9e5c0a2b80 0x7f9e5c0a5040 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f9e74102ef0 0x7f9e7419cdb0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f9e74102570 0x7f9e74198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 --2- 172.21.15.19:0/2853242156 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e74100e20 0x7f9e741984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 >> 172.21.15.19:0/2853242156 conn(0x7f9e740fc7d0 msgr2=0x7f9e7406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 shutdown_connections 2024-09-12T03:33:51.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:51.212+0000 7f9e7a848640 1 -- 172.21.15.19:0/2853242156 wait complete. 2024-09-12T03:33:51.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:51 smithi019 ceph-mon[30953]: pgmap v781: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:33:51.584 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:51 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2853242156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:51.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:51 smithi121 ceph-mon[34963]: pgmap v781: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:33:51.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:51 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2853242156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:51.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:51 smithi135 ceph-mon[35474]: pgmap v781: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:33:51.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:51 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2853242156' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:33:51.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:33:52.195 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 -- 172.21.15.19:0/980545335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4108300 msgr2=0x7f95e410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/980545335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4108300 0x7f95e410ec70 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009cd0 tx=0x7f95dc02a8d0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 -- 172.21.15.19:0/980545335 shutdown_connections 2024-09-12T03:33:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/980545335 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4108300 0x7f95e410ec70 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.675 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/980545335 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4103780 0x7f95e4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/980545335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 0x7f95e41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 -- 172.21.15.19:0/980545335 >> 172.21.15.19:0/980545335 conn(0x7f95e40fc7d0 msgr2=0x7f95e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 -- 172.21.15.19:0/980545335 shutdown_connections 2024-09-12T03:33:52.676 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.675+0000 7f95e8a1e640 1 -- 172.21.15.19:0/980545335 wait complete. 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 Processor -- start 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 -- start start 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 0x7f95e41983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4108300 0x7f95e419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41120d0 con 0x7f95e4103780 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4111f50 con 0x7f95e4102dd0 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e8a1e640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95e4112250 con 0x7f95e4108300 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.678 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57886/0 (socket says 172.21.15.19:57886) 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 -- 172.21.15.19:0/912466641 learned_addr learned my addr 172.21.15.19:0/912466641 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e259c640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 0x7f95e41983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4108300 0x7f95e419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 -- 172.21.15.19:0/912466641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4108300 msgr2=0x7f95e419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4108300 0x7f95e419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 -- 172.21.15.19:0/912466641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 msgr2=0x7f95e41983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.676+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 0x7f95e41983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95e1d9b640 1 -- 172.21.15.19:0/912466641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e419d380 con 0x7f95e4103780 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00c8c0 tx=0x7f95cc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95db7fe640 1 -- 172.21.15.19:0/912466641 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc007bb0 con 0x7f95e4103780 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95db7fe640 1 -- 172.21.15.19:0/912466641 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc007d50 con 0x7f95e4103780 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e419d670 con 0x7f95e4103780 2024-09-12T03:33:52.679 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95db7fe640 1 -- 172.21.15.19:0/912466641 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc018790 con 0x7f95e4103780 2024-09-12T03:33:52.680 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.677+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e41a4f50 con 0x7f95e4103780 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.679+0000 7f95eadf7740 1 Processor -- start 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.679+0000 7f95db7fe640 1 -- 172.21.15.19:0/912466641 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f95cc018970 con 0x7f95e4103780 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 -- start start 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af40171b40 0x55af40170090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af40170a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af400d62f0 con 0x55af4017b370 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55af4007dcf0 con 0x55af401705d0 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95eadf7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af400d22b0 con 0x55af40171b40 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57894/0 (socket says 172.21.15.19:57894) 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95e1d9b640 1 -- 172.21.15.19:0/180393780 learned_addr learned my addr 172.21.15.19:0/180393780 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95e259c640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af40170a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.680+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af40171b40 0x55af40170090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95db7fe640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 0x7f95d00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 -- 172.21.15.19:0/180393780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af40171b40 msgr2=0x55af40170090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af40171b40 0x55af40170090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 -- 172.21.15.19:0/180393780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 msgr2=0x55af40170a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af40170a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 -- 172.21.15.19:0/180393780 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af400623a0 con 0x55af4017b370 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 0x7f95d00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e259c640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af40170a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95db7fe640 1 -- 172.21.15.19:0/912466641 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f95cc0cefe0 con 0x7f95e4103780 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013d90 tx=0x7f95cc0148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cca1938bd6816f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95d8ff9640 1 -- 172.21.15.19:0/180393780 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc09b250 con 0x55af4017b370 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 0x7f95d00a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006010 tx=0x7f95dc0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95d8ff9640 1 -- 172.21.15.19:0/180393780 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95cc01b120 con 0x55af4017b370 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.681+0000 7f95d8ff9640 1 -- 172.21.15.19:0/180393780 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc013bf0 con 0x55af4017b370 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 -- 172.21.15.19:0/180393780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 msgr2=0x55af4017d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f95cc013d90 tx=0x7f95cc0148a0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 -- 172.21.15.19:0/180393780 shutdown_connections 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af4017b370 0x55af4017d760 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af40170a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 --2- 172.21.15.19:0/180393780 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af40171b40 0x55af40170090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 -- 172.21.15.19:0/180393780 >> 172.21.15.19:0/180393780 conn(0x55af40139310 msgr2=0x55af4016cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 -- 172.21.15.19:0/180393780 shutdown_connections 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.682+0000 7f95eadf7740 1 -- 172.21.15.19:0/180393780 wait complete. 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 -- start start 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 Processor -- start 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af4017ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40178600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x55af4017e010 con 0x55af40171b40 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55af40091060 con 0x55af401705d0 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x55af4017e190 con 0x55af4017b370 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95eadf7740 10 client.0 _ll_register_callbacks cb 0x55af3fde48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57908/0 (socket says 172.21.15.19:57908) 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 -- 172.21.15.19:0/3656792539 learned_addr learned my addr 172.21.15.19:0/3656792539 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e259c640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af4017ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40178600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 -- 172.21.15.19:0/3656792539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 msgr2=0x55af40178600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40178600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 -- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 msgr2=0x55af4017ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af4017ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e1d9b640 1 -- 172.21.15.19:0/3656792539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af3fe473c0 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.683+0000 7f95e259c640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af4017ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09b880 tx=0x7f95cc0cf350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2142df70d612a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95cc09b8f0 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f95cc099bf0 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55af400e3860 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f95cc09fca0 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 10 client.18354 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55af4017e010 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55af400e3860 con 0x55af40171b40 2024-09-12T03:33:52.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f95cc09bb50 con 0x55af40171b40 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 10 client.18354 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 10 client.18354 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 20 client.18354 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af4018ac80 0x55af4018d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 4 client.18354 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.684+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af400ddd70 con 0x55af4018ac80 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.685+0000 7f95e259c640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af4018ac80 0x55af4018d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.685+0000 7f95e259c640 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af4018ac80 0x55af4018d070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d7d475781b89ca90 server_cookie=550127daa9bbb273 in_seq=0 out_seq=0 2024-09-12T03:33:52.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.685+0000 7f95c9ffb640 10 client.18354 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:52.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95c9ffb640 1 -- 172.21.15.19:0/3656792539 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af400ddd70 con 0x55af4018ac80 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95c9ffb640 10 client.18354 handle_command_reply: tid=0 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 client.18354 shutdown 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 msgr2=0x55af40178010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f95cc09b880 tx=0x7f95cc0cf350 comp rx=0 tx=0).stop 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 shutdown_connections 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af4018ac80 0x55af4018d070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40178600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af40171b40 0x55af40178010 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95c9ffb640 0 client.18354 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 --2- 172.21.15.19:0/3656792539 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af401705d0 0x55af4017ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 >> 172.21.15.19:0/3656792539 conn(0x55af40139310 msgr2=0x55af40172990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 shutdown_connections 2024-09-12T03:33:52.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 1 -- 172.21.15.19:0/3656792539 wait complete. 2024-09-12T03:33:52.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.686+0000 7f95eadf7740 20 client.18354 trim_cache size 0 max 16384 2024-09-12T03:33:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 Processor -- start 2024-09-12T03:33:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 -- start start 2024-09-12T03:33:52.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40179f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40173120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41ad450 con 0x55af401705d0 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.699+0000 7f95eadf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41af1d0 con 0x55af40171b40 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95eadf7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41adb20 con 0x55af4017b370 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57918/0 (socket says 172.21.15.19:57918) 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 -- 172.21.15.19:0/210145614 learned_addr learned my addr 172.21.15.19:0/210145614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40179f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40173120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 -- 172.21.15.19:0/210145614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 msgr2=0x55af40173120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40173120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 -- 172.21.15.19:0/210145614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 msgr2=0x55af40179f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40179f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 -- 172.21.15.19:0/210145614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af40179390 con 0x55af401705d0 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95e259c640 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55af4018d890 tx=0x7f95d4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658d5f48b2a154de server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95cbfff640 1 -- 172.21.15.19:0/210145614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d402a070 con 0x55af401705d0 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95cbfff640 1 -- 172.21.15.19:0/210145614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d401e8f0 con 0x55af401705d0 2024-09-12T03:33:52.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.700+0000 7f95cbfff640 1 -- 172.21.15.19:0/210145614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d40259a0 con 0x55af401705d0 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- 172.21.15.19:0/210145614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 msgr2=0x55af401778d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x55af4018d890 tx=0x7f95d4010800 comp rx=0 tx=0).stop 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- 172.21.15.19:0/210145614 shutdown_connections 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40173120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40179f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 --2- 172.21.15.19:0/210145614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af401778d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- 172.21.15.19:0/210145614 >> 172.21.15.19:0/210145614 conn(0x55af40139310 msgr2=0x55af40179870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.706 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- 172.21.15.19:0/210145614 shutdown_connections 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- 172.21.15.19:0/210145614 wait complete. 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 -- start start 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.701+0000 7f95eadf7740 1 Processor -- start 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40174e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40175480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41aea40 con 0x55af401705d0 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41ae2b0 con 0x55af40171b40 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f95e41ab370 con 0x55af4017b370 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95eadf7740 10 client.0 _ll_register_callbacks cb 0x7f95d400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.707 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57934/0 (socket says 172.21.15.19:57934) 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 -- 172.21.15.19:0/1280579887 learned_addr learned my addr 172.21.15.19:0/1280579887 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40174e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e2d9d640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40175480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 -- 172.21.15.19:0/1280579887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 msgr2=0x55af40175480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40175480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 -- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 msgr2=0x55af40174e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40174e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 -- 172.21.15.19:0/1280579887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af40175cb0 con 0x55af401705d0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95e259c640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f95d4012f90 tx=0x7f95d4012fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f96ada2fc695ea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.702+0000 7f95c9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d402a040 con 0x55af401705d0 2024-09-12T03:33:52.708 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f95d401eaa0 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d401e8f0 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f95d401e710 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 10 client.18360 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55af401f5a20 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95d401e8f0 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f95d4004b70 con 0x55af401705d0 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 10 client.18360 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 10 client.18360 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.k 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 20 client.18360 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af401f6140 0x55af401f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 4 client.18360 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:52.709 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.703+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 --> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55af4005a170 con 0x55af401f6140 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af401f6140 0x55af401f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95c9ffb640 10 client.18360 ms_handle_connect on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95e1d9b640 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af401f6140 0x55af401f8560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5e09505ff2fed7c server_cookie=c39d4838876f22f9 in_seq=0 out_seq=0 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95c9ffb640 1 -- 172.21.15.19:0/1280579887 <== mds.4 v2:172.21.15.121:6832/3300883065 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55af4005a170 con 0x55af401f6140 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95c9ffb640 10 client.18360 handle_command_reply: tid=0 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 client.18360 shutdown 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 msgr2=0x55af40183cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f95d4012f90 tx=0x7f95d4012fc0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 shutdown_connections 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:6832/3300883065,v1:172.21.15.121:6833/3300883065] conn(0x55af401f6140 0x55af401f8560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x55af4017b370 0x55af40175480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55af40171b40 0x55af40174e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.710 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95c9ffb640 0 client.18360 ms_handle_reset on v2:172.21.15.121:6832/3300883065 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 --2- 172.21.15.19:0/1280579887 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x55af401705d0 0x55af40183cb0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 >> 172.21.15.19:0/1280579887 conn(0x55af40139310 msgr2=0x55af4016bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 shutdown_connections 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.704+0000 7f95eadf7740 1 -- 172.21.15.19:0/1280579887 wait complete. 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95eadf7740 20 client.18360 trim_cache size 0 max 16384 2024-09-12T03:33:52.711 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 msgr2=0x7f95d00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 0x7f95d00a50c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006010 tx=0x7f95dc0095c0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 msgr2=0x7f95e4198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00c8c0 tx=0x7f95cc00ccc0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 shutdown_connections 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f95e4108300 0x7f95e419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f95d00a2c00 0x7f95d00a50c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f95e4103780 0x7f95e4198910 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.712 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 --2- 172.21.15.19:0/912466641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e4102dd0 0x7f95e41983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 >> 172.21.15.19:0/912466641 conn(0x7f95e40fc7d0 msgr2=0x7f95e40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 shutdown_connections 2024-09-12T03:33:52.713 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:52.705+0000 7f95e8a1e640 1 -- 172.21.15.19:0/912466641 wait complete. 2024-09-12T03:33:53.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:33:53.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:53 smithi019 ceph-mon[30953]: pgmap v782: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:33:53.692 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:53.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:53 smithi135 ceph-mon[35474]: pgmap v782: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:33:53.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:53 smithi121 ceph-mon[34963]: pgmap v782: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T03:33:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 -- 172.21.15.19:0/265874096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 msgr2=0x7f73dc110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 --2- 172.21.15.19:0/265874096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc110ca0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009ca0 tx=0x7f73cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 -- 172.21.15.19:0/265874096 shutdown_connections 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 --2- 172.21.15.19:0/265874096 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc110ca0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 --2- 172.21.15.19:0/265874096 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 0x7f73dc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.183+0000 7f73e4151640 1 --2- 172.21.15.19:0/265874096 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- 172.21.15.19:0/265874096 >> 172.21.15.19:0/265874096 conn(0x7f73dc0ffa90 msgr2=0x7f73dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- 172.21.15.19:0/265874096 shutdown_connections 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- 172.21.15.19:0/265874096 wait complete. 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 Processor -- start 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- start start 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 0x7f73dc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc114150 con 0x7f73dc109120 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc113fd0 con 0x7f73dc103ff0 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.184+0000 7f73e4151640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1142d0 con 0x7f73dc1049a0 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57962/0 (socket says 172.21.15.19:57962) 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 -- 172.21.15.19:0/2786801344 learned_addr learned my addr 172.21.15.19:0/2786801344 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 0x7f73dc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 -- 172.21.15.19:0/2786801344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 msgr2=0x7f73dc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 0x7f73dc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 -- 172.21.15.19:0/2786801344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 msgr2=0x7f73dc19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 -- 172.21.15.19:0/2786801344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc19f560 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e26ee640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f73cc031a20 tx=0x7f73cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73daffd640 1 -- 172.21.15.19:0/2786801344 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc003c30 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73daffd640 1 -- 172.21.15.19:0/2786801344 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73cc003dd0 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc19f7f0 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73daffd640 1 -- 172.21.15.19:0/2786801344 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc02fcf0 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.185+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc1a7130 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.187+0000 7f73daffd640 1 -- 172.21.15.19:0/2786801344 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73cc004050 con 0x7f73dc109120 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 Processor -- start 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 -- start start 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa6b40 0x562904aa5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aa5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562904a0b2f0 con 0x562904ab0370 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5629049b2cf0 con 0x562904aa6b40 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e652a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562904a072b0 con 0x562904aa55d0 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e16ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73daffd640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 0x7f73c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e16ec640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57968/0 (socket says 172.21.15.19:57968) 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.188+0000 7f73e16ec640 1 -- 172.21.15.19:0/2865193112 learned_addr learned my addr 172.21.15.19:0/2865193112 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aa5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e26ee640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 0x7f73c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa6b40 0x562904aa5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 -- 172.21.15.19:0/2865193112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 msgr2=0x562904aa5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73daffd640 1 -- 172.21.15.19:0/2786801344 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f73cc0f0540 con 0x7f73dc109120 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aa5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 -- 172.21.15.19:0/2865193112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa6b40 msgr2=0x562904aa5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa6b40 0x562904aa5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 -- 172.21.15.19:0/2865193112 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629049973a0 con 0x562904ab0370 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bf90 tx=0x7f73d000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d562b7f5b2af7a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73c3fff640 1 -- 172.21.15.19:0/2865193112 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004300 con 0x562904ab0370 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73e26ee640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 0x7f73c40a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0ee0a0 tx=0x7f73cc02a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73c3fff640 1 -- 172.21.15.19:0/2865193112 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00044a0 con 0x562904ab0370 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.189+0000 7f73c3fff640 1 -- 172.21.15.19:0/2865193112 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0003850 con 0x562904ab0370 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- 172.21.15.19:0/2865193112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 msgr2=0x562904ab2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bf90 tx=0x7f73d000b790 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- 172.21.15.19:0/2865193112 shutdown_connections 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904ab0370 0x562904ab2760 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aa5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 --2- 172.21.15.19:0/2865193112 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa6b40 0x562904aa5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- 172.21.15.19:0/2865193112 >> 172.21.15.19:0/2865193112 conn(0x562904a6e310 msgr2=0x562904aa1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- 172.21.15.19:0/2865193112 shutdown_connections 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- 172.21.15.19:0/2865193112 wait complete. 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 -- start start 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.190+0000 7f73e652a740 1 Processor -- start 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904aad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x562904ab3070 con 0x562904aa6b40 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5629049c6060 con 0x562904ab0370 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x562904ab31f0 con 0x562904aa55d0 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e652a740 10 client.0 _ll_register_callbacks cb 0x5629047198c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57978/0 (socket says 172.21.15.19:57978) 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 -- 172.21.15.19:0/2169211867 learned_addr learned my addr 172.21.15.19:0/2169211867 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904aad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 -- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 msgr2=0x562904aafd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aafd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 -- 172.21.15.19:0/2169211867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 msgr2=0x562904aad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904aad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.219 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e1eed640 1 -- 172.21.15.19:0/2169211867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56290477c3c0 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.191+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904aad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f73d40106a0 tx=0x7f73d4010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af8df06a61270f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4010ea0 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f73d4004220 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d00044a0 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f73d4023050 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 10 client.18372 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562904abeb70 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d00044a0 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f73d400f040 con 0x562904aa6b40 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 10 client.18372 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:54.220 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 10 client.18372 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 20 client.18372 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904ac0040 0x562904ac2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 4 client.18372 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.192+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562904a18860 con 0x562904ac0040 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.193+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904ac0040 0x562904ac2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.193+0000 7f73c17fa640 10 client.18372 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.193+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904ac0040 0x562904ac2460 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38667fafc0f8b607 server_cookie=7b2819a71eb9e77e in_seq=0 out_seq=0 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.194+0000 7f73c17fa640 1 -- 172.21.15.19:0/2169211867 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562904a18860 con 0x562904ac0040 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.194+0000 7f73c17fa640 10 client.18372 handle_command_reply: tid=0 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.194+0000 7f73e652a740 1 client.18372 shutdown 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.194+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 msgr2=0x562904aad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.194+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f73d40106a0 tx=0x7f73d4010b60 comp rx=0 tx=0).stop 2024-09-12T03:33:54.221 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 shutdown_connections 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904ac0040 0x562904ac2460 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904aad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aad020 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73c17fa640 0 client.18372 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 --2- 172.21.15.19:0/2169211867 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904aafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 >> 172.21.15.19:0/2169211867 conn(0x562904a6e310 msgr2=0x562904aa7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 shutdown_connections 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 1 -- 172.21.15.19:0/2169211867 wait complete. 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.195+0000 7f73e652a740 20 client.18372 trim_cache size 0 max 16384 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 Processor -- start 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 -- start start 2024-09-12T03:33:54.222 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904abc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904ac8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1af7b0 con 0x562904aa6b40 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1b1530 con 0x562904ab0370 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e652a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1afe80 con 0x562904aa55d0 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:57990/0 (socket says 172.21.15.19:57990) 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.208+0000 7f73e1eed640 1 -- 172.21.15.19:0/2496694388 learned_addr learned my addr 172.21.15.19:0/2496694388 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904ac8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 -- 172.21.15.19:0/2496694388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 msgr2=0x562904abc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904abc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904abc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.223 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 -- 172.21.15.19:0/2496694388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 msgr2=0x562904ac8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904ac8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 -- 172.21.15.19:0/2496694388 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562904aa2960 con 0x562904aa6b40 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904abc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f73d4001eb0 tx=0x7f73d4009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7d38f61c5ab2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73c37fe640 1 -- 172.21.15.19:0/2496694388 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d402c070 con 0x562904aa6b40 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73c37fe640 1 -- 172.21.15.19:0/2496694388 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4004000 con 0x562904aa6b40 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.209+0000 7f73c37fe640 1 -- 172.21.15.19:0/2496694388 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d400eb30 con 0x562904aa6b40 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- 172.21.15.19:0/2496694388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 msgr2=0x562904ac7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f73d4001eb0 tx=0x7f73d4009650 comp rx=0 tx=0).stop 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- 172.21.15.19:0/2496694388 shutdown_connections 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904ab0370 0x562904ac8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.224 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904ac7d30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 --2- 172.21.15.19:0/2496694388 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904aa55d0 0x562904abc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- 172.21.15.19:0/2496694388 >> 172.21.15.19:0/2496694388 conn(0x562904a6e310 msgr2=0x562904aaf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- 172.21.15.19:0/2496694388 shutdown_connections 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- 172.21.15.19:0/2496694388 wait complete. 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 -- start start 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.210+0000 7f73e652a740 1 Processor -- start 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 0x562904aaa220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904ab0370 0x562904aaaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1b0da0 con 0x562904aa6b40 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1b0610 con 0x562904aa55d0 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc1ad6d0 con 0x562904ab0370 2024-09-12T03:33:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 10 client.0 _ll_register_callbacks cb 0x7f73d40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58004/0 (socket says 172.21.15.19:58004) 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 -- 172.21.15.19:0/2004975286 learned_addr learned my addr 172.21.15.19:0/2004975286 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904ab0370 0x562904aaaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 0x562904aaa220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 -- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904ab0370 msgr2=0x562904aaaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904ab0370 0x562904aaaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 -- 172.21.15.19:0/2004975286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 msgr2=0x562904aaa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 0x562904aaa220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 -- 172.21.15.19:0/2004975286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562904aa8120 con 0x562904aa6b40 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 0x562904aaa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:54.226 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e1eed640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005c60 tx=0x7f73d4010280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5284c8f44d388f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d4035060 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f73d40043b0 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.211+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73d4004000 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f73d402c040 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 10 client.18378 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x562904b2ae80 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73d4004000 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f73d4012450 con 0x562904aa6b40 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 10 client.18378 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 10 client.18378 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.l 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 20 client.18378 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:54.227 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904b2b570 0x562904b2d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:54.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 4 client.18378 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:54.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 --> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562904a12d70 con 0x562904b2b570 2024-09-12T03:33:54.228 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.212+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904b2b570 0x562904b2d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e16ec640 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904b2b570 0x562904b2d990 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fed1f77a1a59106e server_cookie=cbd0c5f73fa9fff3 in_seq=0 out_seq=0 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73c17fa640 10 client.18378 ms_handle_connect on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73c17fa640 1 -- 172.21.15.19:0/2004975286 <== mds.3 v2:172.21.15.135:6832/990973415 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562904a12d70 con 0x562904b2b570 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73c17fa640 10 client.18378 handle_command_reply: tid=0 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 client.18378 shutdown 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 msgr2=0x562904aaa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005c60 tx=0x7f73d4010280 comp rx=0 tx=0).stop 2024-09-12T03:33:54.229 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 shutdown_connections 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:6832/990973415,v1:172.21.15.135:6833/990973415] conn(0x562904b2b570 0x562904b2d990 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x562904ab0370 0x562904aaaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x562904aa6b40 0x562904aaa760 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 --2- 172.21.15.19:0/2004975286 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x562904aa55d0 0x562904aaa220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73c17fa640 0 client.18378 ms_handle_reset on v2:172.21.15.135:6832/990973415 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 >> 172.21.15.19:0/2004975286 conn(0x562904a6e310 msgr2=0x562904aab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 shutdown_connections 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 1 -- 172.21.15.19:0/2004975286 wait complete. 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.213+0000 7f73e652a740 20 client.18378 trim_cache size 0 max 16384 2024-09-12T03:33:54.230 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 msgr2=0x7f73c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 0x7f73c40a5040 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f73cc0ee0a0 tx=0x7f73cc02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 msgr2=0x7f73dc19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:54.231 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f73cc031a20 tx=0x7f73cc031a50 comp rx=0 tx=0).stop 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 shutdown_connections 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f73c40a2b80 0x7f73c40a5040 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f73dc109120 0x7f73dc19ee80 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f73dc1049a0 0x7f73dc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 --2- 172.21.15.19:0/2786801344 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f73dc103ff0 0x7f73dc19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 >> 172.21.15.19:0/2786801344 conn(0x7f73dc0ffa90 msgr2=0x7f73dc101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 shutdown_connections 2024-09-12T03:33:54.232 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:54.214+0000 7f73e4151640 1 -- 172.21.15.19:0/2786801344 wait complete. 2024-09-12T03:33:54.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:33:55.152 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:55.567 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:55 smithi019 ceph-mon[30953]: pgmap v783: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 -- 172.21.15.19:0/2278741453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 msgr2=0x7f8fbc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/2278741453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc10ea90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009ca0 tx=0x7f8fb402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 -- 172.21.15.19:0/2278741453 shutdown_connections 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/2278741453 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc10ea90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/2278741453 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc102790 0x7f8fbc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/2278741453 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc101de0 0x7f8fbc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 -- 172.21.15.19:0/2278741453 >> 172.21.15.19:0/2278741453 conn(0x7f8fbc0fd8c0 msgr2=0x7f8fbc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.687 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 -- 172.21.15.19:0/2278741453 shutdown_connections 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 -- 172.21.15.19:0/2278741453 wait complete. 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.652+0000 7f8fc1c48640 1 Processor -- start 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 -- start start 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 0x7f8fbc19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc102790 0x7f8fbc19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc114190 con 0x7f8fbc106f10 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc114010 con 0x7f8fbc101de0 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fc1c48640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc114310 con 0x7f8fbc102790 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58026/0 (socket says 172.21.15.19:58026) 2024-09-12T03:33:55.688 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 -- 172.21.15.19:0/3210056465 learned_addr learned my addr 172.21.15.19:0/3210056465 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 0x7f8fbc19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 -- 172.21.15.19:0/3210056465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc102790 msgr2=0x7f8fbc19abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc102790 0x7f8fbc19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc102790 0x7f8fbc19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 -- 172.21.15.19:0/3210056465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 msgr2=0x7f8fbc19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.653+0000 7f8fbbfff640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 0x7f8fbc19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fbbfff640 1 -- 172.21.15.19:0/3210056465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc19f620 con 0x7f8fbc106f10 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 0x7f8fbc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fbbfff640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4031a20 tx=0x7f8fb4031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fb8ff9640 1 -- 172.21.15.19:0/3210056465 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4003c30 con 0x7f8fbc106f10 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fb8ff9640 1 -- 172.21.15.19:0/3210056465 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb4003dd0 con 0x7f8fbc106f10 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc19f8b0 con 0x7f8fbc106f10 2024-09-12T03:33:55.689 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fb8ff9640 1 -- 172.21.15.19:0/3210056465 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb402fcf0 con 0x7f8fbc106f10 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.654+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a71f0 con 0x7f8fbc106f10 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.656+0000 7f8fb8ff9640 1 -- 172.21.15.19:0/3210056465 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8fb4004050 con 0x7f8fbc106f10 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 Processor -- start 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 -- start start 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a5350a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a535d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590a52b62f0 con 0x5590a5351b40 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5590a525dcf0 con 0x5590a535b370 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fc4021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590a52b22b0 con 0x5590a53505d0 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58040/0 (socket says 172.21.15.19:58040) 2024-09-12T03:33:55.690 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 -- 172.21.15.19:0/973336601 learned_addr learned my addr 172.21.15.19:0/973336601 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fb8ff9640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 0x7f8fa40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a5350a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbbfff640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 0x7f8fa40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a535d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 -- 172.21.15.19:0/973336601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 msgr2=0x5590a5350a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a5350a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 -- 172.21.15.19:0/973336601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 msgr2=0x5590a535d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a535d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.657+0000 7f8fbaffd640 1 -- 172.21.15.19:0/973336601 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a52423a0 con 0x5590a5351b40 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.658+0000 7f8fb8ff9640 1 -- 172.21.15.19:0/3210056465 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8fb40f0460 con 0x7f8fbc106f10 2024-09-12T03:33:55.691 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.658+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005e00 tx=0x7f8fb0009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14388d8fc8c9f205 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.658+0000 7f8fa1ffb640 1 -- 172.21.15.19:0/973336601 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x5590a5351b40 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.658+0000 7f8fa1ffb640 1 -- 172.21.15.19:0/973336601 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0005500 con 0x5590a5351b40 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.658+0000 7f8fa1ffb640 1 -- 172.21.15.19:0/973336601 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0013400 con 0x5590a5351b40 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- 172.21.15.19:0/973336601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 msgr2=0x5590a5350090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0005e00 tx=0x7f8fb0009590 comp rx=0 tx=0).stop 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fbbfff640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 0x7f8fa40a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b9730 tx=0x7f8fb408fe20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- 172.21.15.19:0/973336601 shutdown_connections 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a535d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a5350a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 --2- 172.21.15.19:0/973336601 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5350090 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- 172.21.15.19:0/973336601 >> 172.21.15.19:0/973336601 conn(0x5590a5319310 msgr2=0x5590a534cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.692 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- 172.21.15.19:0/973336601 shutdown_connections 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- 172.21.15.19:0/973336601 wait complete. 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 -- start start 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.659+0000 7f8fc4021740 1 Processor -- start 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a535acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a53584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x5590a535e070 con 0x5590a5351b40 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x5590a5271060 con 0x5590a535b370 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x5590a535e1f0 con 0x5590a53505d0 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 10 client.0 _ll_register_callbacks cb 0x5590a4fc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58042/0 (socket says 172.21.15.19:58042) 2024-09-12T03:33:55.693 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1007581231 learned_addr learned my addr 172.21.15.19:0/1007581231 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a53584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 msgr2=0x5590a535acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a535acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1007581231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 msgr2=0x5590a53584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a53584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1007581231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a50273c0 con 0x5590a5351b40 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8fac010500 tx=0x7f8fac0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf80b0cd75da8cf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8f9affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac018070 con 0x5590a5351b40 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8fac014640 con 0x5590a5351b40 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.660+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb0005500 con 0x5590a5351b40 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8fac024050 con 0x5590a5351b40 2024-09-12T03:33:55.694 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 10 client.18390 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5590a5369ae0 con 0x5590a5351b40 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb0005500 con 0x5590a5351b40 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8fac012a60 con 0x5590a5351b40 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 10 client.18390 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 10 client.18390 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 20 client.18390 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a536b080 0x5590a536d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 4 client.18390 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.661+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590a52c3860 con 0x5590a536b080 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.662+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a536b080 0x5590a536d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.662+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a536b080 0x5590a536d4e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=775894b5d3b5e146 server_cookie=e2ff5613c6de971d in_seq=0 out_seq=0 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.662+0000 7f8f9affd640 10 client.18390 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:55.695 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8f9affd640 1 -- 172.21.15.19:0/1007581231 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590a52c3860 con 0x5590a536b080 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8f9affd640 10 client.18390 handle_command_reply: tid=0 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 client.18390 shutdown 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 msgr2=0x5590a5357f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f8fac010500 tx=0x7f8fac0109c0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 shutdown_connections 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a536b080 0x5590a536d4e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a535b370 0x5590a53584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a5357f70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1007581231 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a53505d0 0x5590a535acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8f9affd640 0 client.18390 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 >> 172.21.15.19:0/1007581231 conn(0x5590a5319310 msgr2=0x5590a5352960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 shutdown_connections 2024-09-12T03:33:55.696 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 1 -- 172.21.15.19:0/1007581231 wait complete. 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.663+0000 7f8fc4021740 20 client.18390 trim_cache size 0 max 16384 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 Processor -- start 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 -- start start 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a5357830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a5351b40 0x5590a5353120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1af8a0 con 0x5590a535b370 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b1620 con 0x5590a53505d0 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.676+0000 7f8fc4021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1aff70 con 0x5590a5351b40 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58054/0 (socket says 172.21.15.19:58054) 2024-09-12T03:33:55.697 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 -- 172.21.15.19:0/4004099258 learned_addr learned my addr 172.21.15.19:0/4004099258 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a5351b40 0x5590a5353120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a5357830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 -- 172.21.15.19:0/4004099258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a5351b40 msgr2=0x5590a5353120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a5351b40 0x5590a5353120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 -- 172.21.15.19:0/4004099258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 msgr2=0x5590a5357830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a5357830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 -- 172.21.15.19:0/4004099258 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590a5356f20 con 0x5590a535b370 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x5590a534dc00 tx=0x7f8fb001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04e11cb69af095c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fa17fa640 1 -- 172.21.15.19:0/4004099258 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017040 con 0x5590a535b370 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fa17fa640 1 -- 172.21.15.19:0/4004099258 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb002c070 con 0x5590a535b370 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.677+0000 7f8fa17fa640 1 -- 172.21.15.19:0/4004099258 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb001b400 con 0x5590a535b370 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- 172.21.15.19:0/4004099258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 msgr2=0x5590a5353660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.698 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x5590a534dc00 tx=0x7f8fb001e040 comp rx=0 tx=0).stop 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- 172.21.15.19:0/4004099258 shutdown_connections 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a535b370 0x5590a5353660 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a5351b40 0x5590a5353120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 --2- 172.21.15.19:0/4004099258 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a5357830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- 172.21.15.19:0/4004099258 >> 172.21.15.19:0/4004099258 conn(0x5590a5319310 msgr2=0x5590a5359ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- 172.21.15.19:0/4004099258 shutdown_connections 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- 172.21.15.19:0/4004099258 wait complete. 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 -- start start 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.678+0000 7f8fc4021740 1 Processor -- start 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a53cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 0x5590a535f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.699 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b0e90 con 0x5590a5351b40 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1b0700 con 0x5590a53505d0 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc1ad7c0 con 0x5590a535b370 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fc4021740 10 client.0 _ll_register_callbacks cb 0x7f8fac007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58070/0 (socket says 172.21.15.19:58070) 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1401761837 learned_addr learned my addr 172.21.15.19:0/1401761837 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 0x5590a535f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a53cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 msgr2=0x5590a535f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 0x5590a535f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1401761837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 msgr2=0x5590a53cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:55.700 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a53cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 -- 172.21.15.19:0/1401761837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590a5360220 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 0x5590a535f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8fbb7fe640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8fac004810 tx=0x7f8fac007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b338e045e951102 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8f9affd640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac02d060 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.679+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f8fac013660 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fb002c070 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f8fac018070 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 10 client.18396 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x5590a53d6950 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fb002c070 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f8fac014640 con 0x5590a5351b40 2024-09-12T03:33:55.701 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 10 client.18396 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 10 client.18396 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.c 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 20 client.18396 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a53d7160 0x5590a53d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 4 client.18396 mds_command: new command op to 24463 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 --> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590a52bdd70 con 0x5590a53d7160 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.680+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a53d7160 0x5590a53d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8f9affd640 10 client.18396 ms_handle_connect on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fbaffd640 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a53d7160 0x5590a53d9580 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6b08deba7ab92ce server_cookie=cfad035390ba1626 in_seq=0 out_seq=0 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8f9affd640 1 -- 172.21.15.19:0/1401761837 <== mds.2 v2:172.21.15.135:6836/1827195054 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590a52bdd70 con 0x5590a53d7160 2024-09-12T03:33:55.702 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8f9affd640 10 client.18396 handle_command_reply: tid=0 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 client.18396 shutdown 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 msgr2=0x5590a53cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8fac004810 tx=0x7f8fac007e10 comp rx=0 tx=0).stop 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 shutdown_connections 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:6836/1827195054,v1:172.21.15.135:6837/1827195054] conn(0x5590a53d7160 0x5590a53d9580 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x5590a535b370 0x5590a535f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x5590a5351b40 0x5590a53cf690 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 --2- 172.21.15.19:0/1401761837 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x5590a53505d0 0x5590a53cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8f9affd640 0 client.18396 ms_handle_reset on v2:172.21.15.135:6836/1827195054 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 >> 172.21.15.19:0/1401761837 conn(0x5590a5319310 msgr2=0x5590a534b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.703 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 shutdown_connections 2024-09-12T03:33:55.704 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 1 -- 172.21.15.19:0/1401761837 wait complete. 2024-09-12T03:33:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.681+0000 7f8fc4021740 20 client.18396 trim_cache size 0 max 16384 2024-09-12T03:33:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 msgr2=0x7f8fa40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.704 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 0x7f8fa40a5040 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40b9730 tx=0x7f8fb408fe20 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 msgr2=0x7f8fbc19ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4031a20 tx=0x7f8fb4031a50 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 shutdown_connections 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f8fa40a2b80 0x7f8fa40a5040 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f8fbc106f10 0x7f8fbc19ef40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f8fbc102790 0x7f8fbc19abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 --2- 172.21.15.19:0/3210056465 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8fbc101de0 0x7f8fbc19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 >> 172.21.15.19:0/3210056465 conn(0x7f8fbc0fd8c0 msgr2=0x7f8fbc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 shutdown_connections 2024-09-12T03:33:55.705 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:55.682+0000 7f8fc1c48640 1 -- 172.21.15.19:0/3210056465 wait complete. 2024-09-12T03:33:55.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:55 smithi135 ceph-mon[35474]: pgmap v783: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:33:55.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:55 smithi121 ceph-mon[34963]: pgmap v783: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:33:56.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:33:56.681 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.179+0000 7f1122d41640 1 -- 172.21.15.19:0/3590806897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c1055b0 msgr2=0x7f111c10d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.179+0000 7f1122d41640 1 --2- 172.21.15.19:0/3590806897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c1055b0 0x7f111c10d130 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f111400adb0 tx=0x7f111402b7a0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 -- 172.21.15.19:0/3590806897 shutdown_connections 2024-09-12T03:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 --2- 172.21.15.19:0/3590806897 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c1055b0 0x7f111c10d130 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.189 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 --2- 172.21.15.19:0/3590806897 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 0x7f111c104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 --2- 172.21.15.19:0/3590806897 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c100480 0x7f111c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 -- 172.21.15.19:0/3590806897 >> 172.21.15.19:0/3590806897 conn(0x7f111c077210 msgr2=0x7f111c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 -- 172.21.15.19:0/3590806897 shutdown_connections 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 -- 172.21.15.19:0/3590806897 wait complete. 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 Processor -- start 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 -- start start 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.180+0000 7f1122d41640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 0x7f111c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c1055b0 0x7f111c19ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10fd90 con 0x7f111c100480 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10fc10 con 0x7f111c1055b0 2024-09-12T03:33:57.190 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10ff10 con 0x7f111c100e30 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58090/0 (socket says 172.21.15.19:58090) 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 -- 172.21.15.19:0/29533614 learned_addr learned my addr 172.21.15.19:0/29533614 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f111bfff640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 0x7f111c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f11212de640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c1055b0 0x7f111c19ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 -- 172.21.15.19:0/29533614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 msgr2=0x7f111c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 0x7f111c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 -- 172.21.15.19:0/29533614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c1055b0 msgr2=0x7f111c19ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c1055b0 0x7f111c19ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 -- 172.21.15.19:0/29533614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c19f530 con 0x7f111c100480 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1120add640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c920 tx=0x7f110c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1119ffb640 1 -- 172.21.15.19:0/29533614 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c007c70 con 0x7f111c100480 2024-09-12T03:33:57.191 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1119ffb640 1 -- 172.21.15.19:0/29533614 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f110c007e10 con 0x7f111c100480 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c19f820 con 0x7f111c100480 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.181+0000 7f1119ffb640 1 -- 172.21.15.19:0/29533614 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c018890 con 0x7f111c100480 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.182+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a7100 con 0x7f111c100480 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.184+0000 7f1119ffb640 1 -- 172.21.15.19:0/29533614 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f110c018a70 con 0x7f111c100480 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.184+0000 7f112511a740 1 Processor -- start 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.184+0000 7f112511a740 1 -- start start 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.184+0000 7f112511a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.184+0000 7f112511a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee178a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f112511a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee185760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f112511a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557cee0de2f0 con 0x557cee179b40 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f112511a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557cee085cf0 con 0x557cee183370 2024-09-12T03:33:57.192 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f112511a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557cee0da2b0 con 0x557cee1785d0 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58102/0 (socket says 172.21.15.19:58102) 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 -- 172.21.15.19:0/386262514 learned_addr learned my addr 172.21.15.19:0/386262514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f1120add640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee178a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f1119ffb640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 0x7f11040a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f111bfff640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee185760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 -- 172.21.15.19:0/386262514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 msgr2=0x557cee178a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee178a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 -- 172.21.15.19:0/386262514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 msgr2=0x557cee185760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee185760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 -- 172.21.15.19:0/386262514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cee06a3a0 con 0x557cee179b40 2024-09-12T03:33:57.193 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f111bfff640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee185760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 0x7f11040a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f1119ffb640 1 -- 172.21.15.19:0/29533614 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f110c0cf1e0 con 0x7f111c100480 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.185+0000 7f11212de640 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f11140039f0 tx=0x7f1114003a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e696b6e0c7297d0e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f1102ffd640 1 -- 172.21.15.19:0/386262514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114050070 con 0x557cee179b40 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f1102ffd640 1 -- 172.21.15.19:0/386262514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1114032940 con 0x557cee179b40 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f1102ffd640 1 -- 172.21.15.19:0/386262514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114031990 con 0x557cee179b40 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 -- 172.21.15.19:0/386262514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 msgr2=0x557cee178090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f11212de640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 0x7f11040a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f111404d000 tx=0x7f111404a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f11140039f0 tx=0x7f1114003a20 comp rx=0 tx=0).stop 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 -- 172.21.15.19:0/386262514 shutdown_connections 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee185760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee178a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 --2- 172.21.15.19:0/386262514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee178090 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 -- 172.21.15.19:0/386262514 >> 172.21.15.19:0/386262514 conn(0x557cee141310 msgr2=0x557cee174c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 -- 172.21.15.19:0/386262514 shutdown_connections 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.186+0000 7f112511a740 1 -- 172.21.15.19:0/386262514 wait complete. 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 -- start start 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 Processor -- start 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee182da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 0x557cee180050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee1807e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x557cee186070 con 0x557cee183370 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x557cee099060 con 0x557cee179b40 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x557cee1861f0 con 0x557cee1785d0 2024-09-12T03:33:57.195 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f112511a740 10 client.0 _ll_register_callbacks cb 0x557ceddec8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.187+0000 7f11212de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee1807e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f111bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 0x557cee180050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f111bfff640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 0x557cee180050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.19:34884/0 (socket says 172.21.15.19:34884) 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f111bfff640 1 -- 172.21.15.19:0/3837081514 learned_addr learned my addr 172.21.15.19:0/3837081514 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f1120add640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee182da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f11212de640 1 -- 172.21.15.19:0/3837081514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 msgr2=0x557cee182da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f11212de640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee182da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f11212de640 1 -- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 msgr2=0x557cee180050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f11212de640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 0x557cee180050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f11212de640 1 -- 172.21.15.19:0/3837081514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cede4f3c0 con 0x557cee183370 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.188+0000 7f1120add640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee182da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f11212de640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee1807e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1114006010 tx=0x7f11140316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16010dd4e69b032 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.196 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114050040 con 0x557cee183370 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1114004250 con 0x557cee183370 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1114032940 con 0x557cee183370 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f111400a910 con 0x557cee183370 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f112511a740 10 client.18408 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557cee191ac0 con 0x557cee183370 2024-09-12T03:33:57.197 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1114032940 con 0x557cee183370 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f1114030a60 con 0x557cee183370 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.189+0000 7f112511a740 10 client.18408 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f112511a740 10 client.18408 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f112511a740 20 client.18408 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee192e10 0x557cee195200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f112511a740 4 client.18408 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cee0eb860 con 0x557cee192e10 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f1120add640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee192e10 0x557cee195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f10fbfff640 10 client.18408 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.190+0000 7f1120add640 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee192e10 0x557cee195200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9a6d9689b7b88bd7 server_cookie=78d6cacdd7dfcdbe in_seq=0 out_seq=0 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f10fbfff640 1 -- 172.21.15.19:0/3837081514 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557cee0eb860 con 0x557cee192e10 2024-09-12T03:33:57.199 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f10fbfff640 10 client.18408 handle_command_reply: tid=0 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 client.18408 shutdown 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 msgr2=0x557cee1807e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee1807e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1114006010 tx=0x7f11140316b0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 shutdown_connections 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee192e10 0x557cee195200 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee1807e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee179b40 0x557cee180050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 --2- 172.21.15.19:0/3837081514 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee182da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f10fbfff640 0 client.18408 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 >> 172.21.15.19:0/3837081514 conn(0x557cee141310 msgr2=0x557cee17a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 shutdown_connections 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.191+0000 7f112511a740 1 -- 172.21.15.19:0/3837081514 wait complete. 2024-09-12T03:33:57.200 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.192+0000 7f112511a740 20 client.18408 trim_cache size 0 max 16384 2024-09-12T03:33:57.205 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.204+0000 7f112511a740 1 Processor -- start 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.204+0000 7f112511a740 1 -- start start 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee17f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee17b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1af770 con 0x557cee179b40 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b14f0 con 0x557cee183370 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f112511a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1afe40 con 0x557cee1785d0 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58116/0 (socket says 172.21.15.19:58116) 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 -- 172.21.15.19:0/3403526964 learned_addr learned my addr 172.21.15.19:0/3403526964 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f1120add640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee17f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.206 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f11212de640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee17b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 -- 172.21.15.19:0/3403526964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 msgr2=0x557cee17f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee17f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 -- 172.21.15.19:0/3403526964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 msgr2=0x557cee17b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee17b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 -- 172.21.15.19:0/3403526964 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cee179760 con 0x557cee179b40 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f1120add640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee17f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f111bfff640 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1110004640 tx=0x7f111000bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea2a48f953cb67c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f11027fc640 1 -- 172.21.15.19:0/3403526964 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111000be30 con 0x557cee179b40 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.205+0000 7f11027fc640 1 -- 172.21.15.19:0/3403526964 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110012450 con 0x557cee179b40 2024-09-12T03:33:57.207 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f11027fc640 1 -- 172.21.15.19:0/3403526964 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100116a0 con 0x557cee179b40 2024-09-12T03:33:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- 172.21.15.19:0/3403526964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 msgr2=0x557cee181fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f1110004640 tx=0x7f111000bb20 comp rx=0 tx=0).stop 2024-09-12T03:33:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- 172.21.15.19:0/3403526964 shutdown_connections 2024-09-12T03:33:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee183370 0x557cee17b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.211 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee179b40 0x557cee181fc0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 --2- 172.21.15.19:0/3403526964 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee1785d0 0x557cee17f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- 172.21.15.19:0/3403526964 >> 172.21.15.19:0/3403526964 conn(0x557cee141310 msgr2=0x557cee1818b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- 172.21.15.19:0/3403526964 shutdown_connections 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- 172.21.15.19:0/3403526964 wait complete. 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.206+0000 7f112511a740 1 -- start start 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 Processor -- start 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee1785d0 0x557cee18bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee179b40 0x557cee17cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b0d60 con 0x557cee183370 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1b05d0 con 0x557cee1785d0 2024-09-12T03:33:57.212 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f111c1ad690 con 0x557cee179b40 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f112511a740 10 client.0 _ll_register_callbacks cb 0x7f111404e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58122/0 (socket says 172.21.15.19:58122) 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 -- 172.21.15.19:0/3600578532 learned_addr learned my addr 172.21.15.19:0/3600578532 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f1120add640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee1785d0 0x557cee18bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f111bfff640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee179b40 0x557cee17cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 -- 172.21.15.19:0/3600578532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee179b40 msgr2=0x557cee17cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee179b40 0x557cee17cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 -- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee1785d0 msgr2=0x557cee18bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.207+0000 7f11212de640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee1785d0 0x557cee18bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f11212de640 1 -- 172.21.15.19:0/3600578532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cee1fc880 con 0x557cee183370 2024-09-12T03:33:57.213 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f11212de640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1114031760 tx=0x7f1114032600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88850a060a984f50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f10fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111400a910 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f1114032950 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1110012450 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f11140319a0 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f112511a740 10 client.18414 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x557cee1fda10 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.208+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1110012450 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f1114004370 con 0x557cee183370 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 10 client.18414 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 10 client.18414 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.b 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 20 client.18414 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:57.214 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee1fe1e0 0x557cee200620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 4 client.18414 mds_command: new command op to 24467 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 --> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cee0e5d70 con 0x557cee1fe1e0 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f1120add640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee1fe1e0 0x557cee200620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f10fbfff640 10 client.18414 ms_handle_connect on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f1120add640 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee1fe1e0 0x557cee200620 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ba14dae47618068 server_cookie=44afff1aea6b9a36 in_seq=0 out_seq=0 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f10fbfff640 1 -- 172.21.15.19:0/3600578532 <== mds.1 v2:172.21.15.121:6838/1305484715 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cee0e5d70 con 0x557cee1fe1e0 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.209+0000 7f10fbfff640 10 client.18414 handle_command_reply: tid=0 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 client.18414 shutdown 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 msgr2=0x557cee17d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1114031760 tx=0x7f1114032600 comp rx=0 tx=0).stop 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 shutdown_connections 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:6838/1305484715,v1:172.21.15.121:6839/1305484715] conn(0x557cee1fe1e0 0x557cee200620 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.215 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x557cee183370 0x557cee17d6a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x557cee179b40 0x557cee17cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 --2- 172.21.15.19:0/3600578532 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x557cee1785d0 0x557cee18bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f10fbfff640 0 client.18414 ms_handle_reset on v2:172.21.15.121:6838/1305484715 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 >> 172.21.15.19:0/3600578532 conn(0x557cee141310 msgr2=0x557cee1820d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 shutdown_connections 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 1 -- 172.21.15.19:0/3600578532 wait complete. 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.210+0000 7f112511a740 20 client.18414 trim_cache size 0 max 16384 2024-09-12T03:33:57.216 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 msgr2=0x7f11040a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 0x7f11040a5040 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f111404d000 tx=0x7f111404a000 comp rx=0 tx=0).stop 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 msgr2=0x7f111c19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f110c00c920 tx=0x7f110c00cde0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 shutdown_connections 2024-09-12T03:33:57.217 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f111c1055b0 0x7f111c19ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f111c100e30 0x7f111c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f11040a2b80 0x7f11040a5040 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 --2- 172.21.15.19:0/29533614 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f111c100480 0x7f111c19a580 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 >> 172.21.15.19:0/29533614 conn(0x7f111c077210 msgr2=0x7f111c103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 shutdown_connections 2024-09-12T03:33:57.218 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:57.211+0000 7f1122d41640 1 -- 172.21.15.19:0/29533614 wait complete. 2024-09-12T03:33:57.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:57 smithi019 ceph-mon[30953]: pgmap v784: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:33:57.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:57 smithi135 ceph-mon[35474]: pgmap v784: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:33:57.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:57 smithi121 ceph-mon[34963]: pgmap v784: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:33:57.914 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:33:58.173 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 -- 172.21.15.19:0/2654374340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8100e20 msgr2=0x7f62a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 --2- 172.21.15.19:0/2654374340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8100e20 0x7f62a8101200 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6298009900 tx=0x7f629802a480 comp rx=0 tx=0).stop 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 -- 172.21.15.19:0/2654374340 shutdown_connections 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 --2- 172.21.15.19:0/2654374340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8102ef0 0x7f62a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 --2- 172.21.15.19:0/2654374340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102570 0x7f62a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 --2- 172.21.15.19:0/2654374340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8100e20 0x7f62a8101200 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 -- 172.21.15.19:0/2654374340 >> 172.21.15.19:0/2654374340 conn(0x7f62a80fc7d0 msgr2=0x7f62a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 -- 172.21.15.19:0/2654374340 shutdown_connections 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.765+0000 7f62ace83640 1 -- 172.21.15.19:0/2654374340 wait complete. 2024-09-12T03:33:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 Processor -- start 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 -- start start 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8100e20 0x7f62a8198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102ef0 0x7f62a819ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8110fa0 con 0x7f62a8102570 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8110e20 con 0x7f62a8100e20 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62ace83640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8111120 con 0x7f62a8102ef0 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58140/0 (socket says 172.21.15.19:58140) 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 -- 172.21.15.19:0/145704281 learned_addr learned my addr 172.21.15.19:0/145704281 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a659c640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8100e20 0x7f62a8198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102ef0 0x7f62a819ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 -- 172.21.15.19:0/145704281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102ef0 msgr2=0x7f62a819ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102ef0 0x7f62a819ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 -- 172.21.15.19:0/145704281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8100e20 msgr2=0x7f62a8198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8100e20 0x7f62a8198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.766+0000 7f62a5d9b640 1 -- 172.21.15.19:0/145704281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a819d520 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f629c00b4a0 tx=0x7f629c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62977fe640 1 -- 172.21.15.19:0/145704281 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c014070 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a819d810 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62977fe640 1 -- 172.21.15.19:0/145704281 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f629c010af0 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62977fe640 1 -- 172.21.15.19:0/145704281 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c017d10 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.767+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81a50f0 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.769+0000 7f62977fe640 1 -- 172.21.15.19:0/145704281 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f629c017070 con 0x7f62a8102570 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 Processor -- start 2024-09-12T03:33:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 -- start start 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c3cb40 0x561ff6c3b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c46370 0x561ff6c48760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561ff6ba12f0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ff6b48cf0 con 0x561ff6c3cb40 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62af25c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561ff6b9d2b0 con 0x561ff6c46370 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62977fe640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 0x7f62900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58142/0 (socket says 172.21.15.19:58142) 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62a659c640 1 -- 172.21.15.19:0/538864145 learned_addr learned my addr 172.21.15.19:0/538864145 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c46370 0x561ff6c48760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.770+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c3cb40 0x561ff6c3b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 0x7f62900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 -- 172.21.15.19:0/538864145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c46370 msgr2=0x561ff6c48760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62977fe640 1 -- 172.21.15.19:0/145704281 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f629c0ce4b0 con 0x7f62a8102570 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c46370 0x561ff6c48760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 -- 172.21.15.19:0/538864145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c3cb40 msgr2=0x561ff6c3b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c3cb40 0x561ff6c3b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 -- 172.21.15.19:0/538864145 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff6b2d3a0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a659c640 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f62980314e0 tx=0x7f6298004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a439a20cf69dd8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 0x7f62900a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f62a000bfd0 tx=0x7f62a0007220 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f6294ff9640 1 -- 172.21.15.19:0/538864145 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298035030 con 0x561ff6c3b5d0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f6294ff9640 1 -- 172.21.15.19:0/538864145 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.771+0000 7f6294ff9640 1 -- 172.21.15.19:0/538864145 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298031c80 con 0x561ff6c3b5d0 2024-09-12T03:33:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- 172.21.15.19:0/538864145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 msgr2=0x561ff6c3ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f62980314e0 tx=0x7f6298004170 comp rx=0 tx=0).stop 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- 172.21.15.19:0/538864145 shutdown_connections 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c46370 0x561ff6c48760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c3ba10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 --2- 172.21.15.19:0/538864145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c3cb40 0x561ff6c3b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- 172.21.15.19:0/538864145 >> 172.21.15.19:0/538864145 conn(0x561ff6c04310 msgr2=0x561ff6c37c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- 172.21.15.19:0/538864145 shutdown_connections 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- 172.21.15.19:0/538864145 wait complete. 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.772+0000 7f62af25c740 1 -- start start 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 Processor -- start 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c42fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x561ff6c49010 con 0x561ff6c3b5d0 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x561ff6b5c060 con 0x561ff6c46370 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x561ff6c49190 con 0x561ff6c3cb40 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62af25c740 10 client.0 _ll_register_callbacks cb 0x561ff68af8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58148/0 (socket says 172.21.15.19:58148) 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 -- 172.21.15.19:0/1364210765 learned_addr learned my addr 172.21.15.19:0/1364210765 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c42fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 -- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 msgr2=0x561ff6c42fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c42fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 -- 172.21.15.19:0/1364210765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 msgr2=0x561ff6c435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.773+0000 7f62a659c640 1 -- 172.21.15.19:0/1364210765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff69123c0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62a659c640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f629802aba0 tx=0x7f6298005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261afe4ad3ee4b6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298035070 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f62980035f0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6298003e90 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 10 client.18426 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x561ff6c54a70 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f629803c930 con 0x561ff6c3b5d0 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 10 client.18426 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 10 client.18426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 20 client.18426 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6c55dc0 0x561ff6c581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 4 client.18426 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.774+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ff6bae860 con 0x561ff6c55dc0 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.775+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6c55dc0 0x561ff6c581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.775+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6c55dc0 0x561ff6c581b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2af3953b0fc70369 server_cookie=c755a3827170a9e in_seq=0 out_seq=0 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.775+0000 7f628dffb640 10 client.18426 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f628dffb640 1 -- 172.21.15.19:0/1364210765 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ff6bae860 con 0x561ff6c55dc0 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f628dffb640 10 client.18426 handle_command_reply: tid=0 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 client.18426 shutdown 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 msgr2=0x561ff6c45d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f629802aba0 tx=0x7f6298005bd0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 shutdown_connections 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6c55dc0 0x561ff6c581b0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c42fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f628dffb640 0 client.18426 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 --2- 172.21.15.19:0/1364210765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c45d30 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 >> 172.21.15.19:0/1364210765 conn(0x561ff6c04310 msgr2=0x561ff6c3d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 shutdown_connections 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 1 -- 172.21.15.19:0/1364210765 wait complete. 2024-09-12T03:33:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.776+0000 7f62af25c740 20 client.18426 trim_cache size 0 max 16384 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 Processor -- start 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 -- start start 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c5d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81ab470 con 0x561ff6c3b5d0 2024-09-12T03:33:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81ad1f0 con 0x561ff6c46370 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62af25c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81abb40 con 0x561ff6c3cb40 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.789+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58158/0 (socket says 172.21.15.19:58158) 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 -- 172.21.15.19:0/158279295 learned_addr learned my addr 172.21.15.19:0/158279295 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c5d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 -- 172.21.15.19:0/158279295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 msgr2=0x561ff6c522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 -- 172.21.15.19:0/158279295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 msgr2=0x561ff6c5d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c5d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 -- 172.21.15.19:0/158279295 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ff6c46060 con 0x561ff6c3b5d0 2024-09-12T03:33:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f62a659c640 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6298000c00 tx=0x7f6298004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b220c882c417c2c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f628ffff640 1 -- 172.21.15.19:0/158279295 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629805d020 con 0x561ff6c3b5d0 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f628ffff640 1 -- 172.21.15.19:0/158279295 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.790+0000 7f628ffff640 1 -- 172.21.15.19:0/158279295 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298005020 con 0x561ff6c3b5d0 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- 172.21.15.19:0/158279295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 msgr2=0x561ff6c37e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f6298000c00 tx=0x7f6298004170 comp rx=0 tx=0).stop 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- 172.21.15.19:0/158279295 shutdown_connections 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c5d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 --2- 172.21.15.19:0/158279295 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c37e10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- 172.21.15.19:0/158279295 >> 172.21.15.19:0/158279295 conn(0x561ff6c04310 msgr2=0x561ff6c37700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- 172.21.15.19:0/158279295 shutdown_connections 2024-09-12T03:33:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- 172.21.15.19:0/158279295 wait complete. 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 -- start start 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.791+0000 7f62af25c740 1 Processor -- start 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c4a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c4aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81aca60 con 0x561ff6c3b5d0 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81ac2d0 con 0x561ff6c46370 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f62a81a9390 con 0x561ff6c3cb40 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62af25c740 10 client.0 _ll_register_callbacks cb 0x7f62a000bd10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58172/0 (socket says 172.21.15.19:58172) 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 -- 172.21.15.19:0/4069116051 learned_addr learned my addr 172.21.15.19:0/4069116051 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a6d9d640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c4aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c4a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 -- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 msgr2=0x561ff6c4a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c4a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 -- 172.21.15.19:0/4069116051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 msgr2=0x561ff6c4aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c4aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 -- 172.21.15.19:0/4069116051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ff6cbf7b0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f62a659c640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6298009900 tx=0x7f6298005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328a5553e5a75d6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.792+0000 7f628dffb640 10 client.0 ms_handle_connect on v2:172.21.15.19:3300/0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629805d040 con 0x561ff6c3b5d0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mon.0 v2:172.21.15.19:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f629804a070 con 0x561ff6c3b5d0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f6298003800 con 0x561ff6c3b5d0 2024-09-12T03:33:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 10 client.18432 fetch_fsmap learned FSMap version 63 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x561ff6cc0a30 con 0x561ff6c3b5d0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mon.0 v2:172.21.15.19:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6298003ad0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mon.0 v2:172.21.15.19:3300/0 5 ==== fsmap(e 63) ==== 8442+0+0 (secure 0 0 0) 0x7f62980594e0 con 0x561ff6c3b5d0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 10 client.18432 fetch_fsmap finished waiting for FSMap version 63 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 20 client.18432 populate_metadata read hostname 'smithi019' 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6cc11d0 0x561ff6cc35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 4 client.18432 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.793+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 --> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ff6ba8d70 con 0x561ff6cc11d0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6cc11d0 0x561ff6cc35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62a5d9b640 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6cc11d0 0x561ff6cc35f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f162203cfd04fe2 server_cookie=1fe4d427d0b62dee in_seq=0 out_seq=0 2024-09-12T03:33:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f628dffb640 10 client.18432 ms_handle_connect on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f628dffb640 1 -- 172.21.15.19:0/4069116051 <== mds.0 v2:172.21.15.135:6838/1449185077 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ff6ba8d70 con 0x561ff6cc11d0 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f628dffb640 10 client.18432 handle_command_reply: tid=0 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 client.18432 shutdown 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 msgr2=0x561ff6c4a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6298009900 tx=0x7f6298005930 comp rx=0 tx=0).stop 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 shutdown_connections 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:6838/1449185077,v1:172.21.15.135:6839/1449185077] conn(0x561ff6cc11d0 0x561ff6cc35f0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x561ff6c46370 0x561ff6c4aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x561ff6c3cb40 0x561ff6c4a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 --2- 172.21.15.19:0/4069116051 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x561ff6c3b5d0 0x561ff6c4a120 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 >> 172.21.15.19:0/4069116051 conn(0x561ff6c04310 msgr2=0x561ff6c364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f628dffb640 0 client.18432 ms_handle_reset on v2:172.21.15.135:6838/1449185077 2024-09-12T03:33:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 shutdown_connections 2024-09-12T03:33:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.794+0000 7f62af25c740 1 -- 172.21.15.19:0/4069116051 wait complete. 2024-09-12T03:33:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62af25c740 20 client.18432 trim_cache size 0 max 16384 2024-09-12T03:33:58.805 INFO:teuthology.orchestra.run.smithi019.stdout:[] 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 msgr2=0x7f62900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 0x7f62900a5040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f62a000bfd0 tx=0x7f62a0007220 comp rx=0 tx=0).stop 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 msgr2=0x7f62a8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f629c00b4a0 tx=0x7f629c00b960 comp rx=0 tx=0).stop 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 shutdown_connections 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f62a8102ef0 0x7f62a819ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f62900a2b80 0x7f62900a5040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f62a8102570 0x7f62a8198ab0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 --2- 172.21.15.19:0/145704281 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f62a8100e20 0x7f62a8198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 >> 172.21.15.19:0/145704281 conn(0x7f62a80fc7d0 msgr2=0x7f62a8069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 shutdown_connections 2024-09-12T03:33:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:58.795+0000 7f62ace83640 1 -- 172.21.15.19:0/145704281 wait complete. 2024-09-12T03:33:59.624 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T03:33:59.625 INFO:tasks.fwd_scrub:done joining 2024-09-12T03:33:59.626 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:33:59.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:33:59.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:33:59 smithi019 ceph-mon[30953]: pgmap v785: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:33:59.701 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:33:59 smithi121 ceph-mon[34963]: pgmap v785: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:33:59.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 -- 172.21.15.19:0/3606273082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 msgr2=0x7f3fb01049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:59.965 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3606273082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb01049a0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009900 tx=0x7f3fac02a410 comp rx=0 tx=0).stop 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 -- 172.21.15.19:0/3606273082 shutdown_connections 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3606273082 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 0x7f3fb0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3606273082 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb01072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3606273082 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb01049a0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 -- 172.21.15.19:0/3606273082 >> 172.21.15.19:0/3606273082 conn(0x7f3fb00fc1b0 msgr2=0x7f3fb00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 -- 172.21.15.19:0/3606273082 shutdown_connections 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.746+0000 7f3fb8766640 1 -- 172.21.15.19:0/3606273082 wait complete. 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 Processor -- start 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 -- start start 2024-09-12T03:33:59.966 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb0198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 0x7f3fb019f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb010d7f0 con 0x7f3fb01025b0 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb010d670 con 0x7f3fb0104ee0 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb8766640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3fb010d970 con 0x7f3fb0107810 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb6502640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb6502640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58186/0 (socket says 172.21.15.19:58186) 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb6502640 1 -- 172.21.15.19:0/3824817660 learned_addr learned my addr 172.21.15.19:0/3824817660 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.747+0000 7f3fb5d01640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb0198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6d03640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 0x7f3fb019f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 -- 172.21.15.19:0/3824817660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 msgr2=0x7f3fb019f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:59.967 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 0x7f3fb019f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 -- 172.21.15.19:0/3824817660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 msgr2=0x7f3fb0198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb0198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 -- 172.21.15.19:0/3824817660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fb019fa30 con 0x7f3fb01025b0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb5d01640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb6502640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009230 tx=0x7f3fac004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac03f030 con 0x7f3fb01025b0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fac003ce0 con 0x7f3fb01025b0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fb019fcc0 con 0x7f3fb01025b0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fac035070 con 0x7f3fb01025b0 2024-09-12T03:33:59.968 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.748+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fb01a01b0 con 0x7f3fb01025b0 2024-09-12T03:33:59.969 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:33:59 smithi135 ceph-mon[35474]: pgmap v785: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:33:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.749+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fb00fdca0 con 0x7f3fb01025b0 2024-09-12T03:33:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.750+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3fac042030 con 0x7f3fb01025b0 2024-09-12T03:33:59.970 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.751+0000 7f3fa77fe640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 0x7f3f9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.752+0000 7f3fb5d01640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 0x7f3f9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.752+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3fac0efa80 con 0x7f3fb01025b0 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.752+0000 7f3fb5d01640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 0x7f3f9c0a50d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005d00 tx=0x7f3fa0005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.756+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fac0bc9f0 con 0x7f3fb01025b0 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.934+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fb0062e90 con 0x7f3fb01025b0 2024-09-12T03:33:59.971 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.935+0000 7f3fa77fe640 1 -- 172.21.15.19:0/3824817660 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7f3fac0bc2f0 con 0x7f3fb01025b0 2024-09-12T03:33:59.972 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:33:59.972 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:33:59.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.937+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 msgr2=0x7f3f9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:59.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.937+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 0x7f3f9c0a50d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0005d00 tx=0x7f3fa0005c70 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.937+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 msgr2=0x7f3fb0197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.937+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009230 tx=0x7f3fac004360 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 shutdown_connections 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3fb0107810 0x7f3fb019f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3fb0104ee0 0x7f3fb0198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3f9c0a2c10 0x7f3f9c0a50d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 --2- 172.21.15.19:0/3824817660 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3fb01025b0 0x7f3fb0197e10 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 >> 172.21.15.19:0/3824817660 conn(0x7f3fb00fc1b0 msgr2=0x7f3fb0102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 shutdown_connections 2024-09-12T03:33:59.974 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:33:59.938+0000 7f3fb8766640 1 -- 172.21.15.19:0/3824817660 wait complete. 2024-09-12T03:33:59.975 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T03:12:25.696963+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T03:17:06.299019+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T03:17:08.893970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:33:59.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 -- 172.21.15.19:0/349220261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401025b0 msgr2=0x7f3c401049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 --2- 172.21.15.19:0/349220261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401025b0 0x7f3c401049a0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009900 tx=0x7f3c3002a410 comp rx=0 tx=0).stop 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 -- 172.21.15.19:0/349220261 shutdown_connections 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 --2- 172.21.15.19:0/349220261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40107810 0x7f3c40109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 --2- 172.21.15.19:0/349220261 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c40104ee0 0x7f3c401072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 --2- 172.21.15.19:0/349220261 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c401025b0 0x7f3c401049a0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 -- 172.21.15.19:0/349220261 >> 172.21.15.19:0/349220261 conn(0x7f3c400fc1b0 msgr2=0x7f3c400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 -- 172.21.15.19:0/349220261 shutdown_connections 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.132+0000 7f3c4692a640 1 -- 172.21.15.19:0/349220261 wait complete. 2024-09-12T03:34:00.135 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 Processor -- start 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 -- start start 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c401025b0 0x7f3c40197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40104ee0 0x7f3c40198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010d810 con 0x7f3c40107810 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010d690 con 0x7f3c40104ee0 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c4692a640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010d990 con 0x7f3c401025b0 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c44ec7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c44ec7640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:58206/0 (socket says 172.21.15.19:58206) 2024-09-12T03:34:00.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c44ec7640 1 -- 172.21.15.19:0/63330799 learned_addr learned my addr 172.21.15.19:0/63330799 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:00.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.133+0000 7f3c3ffff640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c401025b0 0x7f3c40197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 -- 172.21.15.19:0/63330799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c401025b0 msgr2=0x7f3c40197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c401025b0 0x7f3c40197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 -- 172.21.15.19:0/63330799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40104ee0 msgr2=0x7f3c40198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c3f7fe640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40104ee0 0x7f3c40198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40104ee0 0x7f3c40198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 -- 172.21.15.19:0/63330799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4019faf0 con 0x7f3c40107810 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c44ec7640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800ccb0 tx=0x7f3c3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c3800f450 con 0x7f3c40107810 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c3800fad0 con 0x7f3c40107810 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4019fde0 con 0x7f3c40107810 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c38018bf0 con 0x7f3c40107810 2024-09-12T03:34:00.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.134+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401a02a0 con 0x7f3c40107810 2024-09-12T03:34:00.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.135+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c400fdca0 con 0x7f3c40107810 2024-09-12T03:34:00.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.136+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c3800f5f0 con 0x7f3c40107810 2024-09-12T03:34:00.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.137+0000 7f3c3d7fa640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 0x7f3c280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.137+0000 7f3c3ffff640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 0x7f3c280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.137+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f3c380cf740 con 0x7f3c40107810 2024-09-12T03:34:00.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.138+0000 7f3c3ffff640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 0x7f3c280a50d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000c00 tx=0x7f3c300315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.141 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.141+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3c38014030 con 0x7f3c40107810 2024-09-12T03:34:00.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.318+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c4002da10 con 0x7f3c40107810 2024-09-12T03:34:00.319 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.319+0000 7f3c3d7fa640 1 -- 172.21.15.19:0/63330799 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f3c3809c7c0 con 0x7f3c40107810 2024-09-12T03:34:00.319 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-12T03:34:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.320+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 msgr2=0x7f3c280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.320+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 0x7f3c280a50d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3c30000c00 tx=0x7f3c300315b0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.320+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 msgr2=0x7f3c4019f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.320+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3c3800ccb0 tx=0x7f3c3800a620 comp rx=0 tx=0).stop 2024-09-12T03:34:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 shutdown_connections 2024-09-12T03:34:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f3c280a2c10 0x7f3c280a50d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f3c40107810 0x7f3c4019f480 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.321 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c40104ee0 0x7f3c40198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 --2- 172.21.15.19:0/63330799 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f3c401025b0 0x7f3c40197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 >> 172.21.15.19:0/63330799 conn(0x7f3c400fc1b0 msgr2=0x7f3c40102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 shutdown_connections 2024-09-12T03:34:00.322 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.321+0000 7f3c4692a640 1 -- 172.21.15.19:0/63330799 wait complete. 2024-09-12T03:34:00.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.440+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1016550765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 msgr2=0x7fc098109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.440+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1016550765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc098109f70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc090009900 tx=0x7fc09002a410 comp rx=0 tx=0).stop 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1016550765 shutdown_connections 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1016550765 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc098109f70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1016550765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098102d20 0x7fc0981031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1016550765 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098101b30 0x7fc098101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.443 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1016550765 >> 172.21.15.19:0/1016550765 conn(0x7fc0980fd290 msgr2=0x7fc0980ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1016550765 shutdown_connections 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1016550765 wait complete. 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 Processor -- start 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- start start 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098101b30 0x7fc098197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 0x7fc0981983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810d7b0 con 0x7fc0981036e0 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810d630 con 0x7fc098101b30 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.441+0000 7fc09cd4b640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fc09810d930 con 0x7fc098102d20 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.444 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42734/0 (socket says 172.21.15.19:42734) 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 -- 172.21.15.19:0/1206185340 learned_addr learned my addr 172.21.15.19:0/1206185340 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc09659c640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098101b30 0x7fc098197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc095d9b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 0x7fc0981983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 -- 172.21.15.19:0/1206185340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 msgr2=0x7fc0981983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 0x7fc0981983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 -- 172.21.15.19:0/1206185340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098101b30 msgr2=0x7fc098197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098101b30 0x7fc098197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 -- 172.21.15.19:0/1206185340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09819fa80 con 0x7fc0981036e0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc095d9b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 0x7fc0981983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc096d9d640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc0900314e0 tx=0x7fc090004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09003f030 con 0x7fc0981036e0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc090003ce0 con 0x7fc0981036e0 2024-09-12T03:34:00.445 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09819fd10 con 0x7fc0981036e0 2024-09-12T03:34:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc090035070 con 0x7fc0981036e0 2024-09-12T03:34:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.442+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0981a01f0 con 0x7fc0981036e0 2024-09-12T03:34:00.446 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.443+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc098108f20 con 0x7fc0981036e0 2024-09-12T03:34:00.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.445+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc090042030 con 0x7fc0981036e0 2024-09-12T03:34:00.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.445+0000 7fc08f7fe640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 0x7fc0840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.446+0000 7fc09659c640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 0x7fc0840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.446+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fc0900ef950 con 0x7fc0981036e0 2024-09-12T03:34:00.447 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.446+0000 7fc09659c640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 0x7fc0840a50d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc088005c40 tx=0x7fc08800a590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.448 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.448+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc0900bc9e0 con 0x7fc0981036e0 2024-09-12T03:34:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.624+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc098062e90 con 0x7fc0981036e0 2024-09-12T03:34:00.624 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.624+0000 7fc08f7fe640 1 -- 172.21.15.19:0/1206185340 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) ==== 104+0+11 (secure 0 0 0) 0x7fc0900bc2e0 con 0x7fc0981036e0 2024-09-12T03:34:00.625 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 32 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 msgr2=0x7fc0840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 0x7fc0840a50d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc088005c40 tx=0x7fc08800a590 comp rx=0 tx=0).stop 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 msgr2=0x7fc09819f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc0900314e0 tx=0x7fc090004360 comp rx=0 tx=0).stop 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 shutdown_connections 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fc0840a2c10 0x7fc0840a50d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fc0981036e0 0x7fc09819f410 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fc098102d20 0x7fc0981983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.627 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 --2- 172.21.15.19:0/1206185340 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc098101b30 0x7fc098197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 >> 172.21.15.19:0/1206185340 conn(0x7fc0980fd290 msgr2=0x7fc0981060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 shutdown_connections 2024-09-12T03:34:00.628 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.626+0000 7fc09cd4b640 1 -- 172.21.15.19:0/1206185340 wait complete. 2024-09-12T03:34:00.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:00 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3824817660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:00.636 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:00 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/63330799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:34:00.638 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:34:00.747 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 -- 172.21.15.19:0/1471733786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 msgr2=0x7f7d7c101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/1471733786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c101f10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7002a410 comp rx=0 tx=0).stop 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 -- 172.21.15.19:0/1471733786 shutdown_connections 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/1471733786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c1036c0 0x7f7d7c109f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/1471733786 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c102d00 0x7f7d7c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/1471733786 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c101f10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 -- 172.21.15.19:0/1471733786 >> 172.21.15.19:0/1471733786 conn(0x7f7d7c0fd2b0 msgr2=0x7f7d7c0ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 -- 172.21.15.19:0/1471733786 shutdown_connections 2024-09-12T03:34:00.748 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.744+0000 7f7d82c5c640 1 -- 172.21.15.19:0/1471733786 wait complete. 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 Processor -- start 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 -- start start 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c102d00 0x7f7d7c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c1036c0 0x7f7d7c19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10d7c0 con 0x7f7d7c101b10 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10d640 con 0x7f7d7c102d00 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d82c5c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c10d940 con 0x7f7d7c1036c0 2024-09-12T03:34:00.749 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d809f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d809f8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42756/0 (socket says 172.21.15.19:42756) 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d809f8640 1 -- 172.21.15.19:0/95853877 learned_addr learned my addr 172.21.15.19:0/95853877 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.745+0000 7f7d7bfff640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c102d00 0x7f7d7c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d811f9640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c1036c0 0x7f7d7c19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 -- 172.21.15.19:0/95853877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c1036c0 msgr2=0x7f7d7c19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c1036c0 0x7f7d7c19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 -- 172.21.15.19:0/95853877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c102d00 msgr2=0x7f7d7c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c102d00 0x7f7d7c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 -- 172.21.15.19:0/95853877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c19fa40 con 0x7f7d7c101b10 2024-09-12T03:34:00.750 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d809f8640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7d70031440 tx=0x7f7d70032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003f030 con 0x7f7d7c101b10 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d70003ce0 con 0x7f7d7c101b10 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70035070 con 0x7f7d7c101b10 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c19fcd0 con 0x7f7d7c101b10 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.746+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a01c0 con 0x7f7d7c101b10 2024-09-12T03:34:00.751 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.747+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c108f00 con 0x7f7d7c101b10 2024-09-12T03:34:00.752 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.748+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d70042030 con 0x7f7d7c101b10 2024-09-12T03:34:00.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.749+0000 7f7d79ffb640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 0x7f7d6c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:00.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.749+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f7d700ef750 con 0x7f7d7c101b10 2024-09-12T03:34:00.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.749+0000 7f7d7bfff640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 0x7f7d6c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:00.753 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.750+0000 7f7d7bfff640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 0x7f7d6c0a5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c199330 tx=0x7f7d6800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:00.754 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.753+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7d700bc660 con 0x7f7d7c101b10 2024-09-12T03:34:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:00 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3824817660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:00.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:00 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/63330799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:34:00.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:00 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3824817660' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:00.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:00 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/63330799' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:34:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.929+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d7c062df0 con 0x7f7d7c101b10 2024-09-12T03:34:00.929 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.929+0000 7f7d79ffb640 1 -- 172.21.15.19:0/95853877 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) ==== 100+0+11 (secure 0 0 0) 0x7f7d700bbf60 con 0x7f7d7c101b10 2024-09-12T03:34:00.929 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 64 2024-09-12T03:34:00.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 msgr2=0x7f7d6c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 0x7f7d6c0a5050 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c199330 tx=0x7f7d6800a2d0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 msgr2=0x7f7d7c197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:00.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f7d70031440 tx=0x7f7d70032e20 comp rx=0 tx=0).stop 2024-09-12T03:34:00.931 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 shutdown_connections 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7d7c1036c0 0x7f7d7c19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d7c102d00 0x7f7d7c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f7d6c0a2b90 0x7f7d6c0a5050 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 --2- 172.21.15.19:0/95853877 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7d7c101b10 0x7f7d7c197e20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 >> 172.21.15.19:0/95853877 conn(0x7f7d7c0fd2b0 msgr2=0x7f7d7c1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 shutdown_connections 2024-09-12T03:34:00.932 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:00.931+0000 7f7d82c5c640 1 -- 172.21.15.19:0/95853877 wait complete. 2024-09-12T03:34:00.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:34:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 -- 172.21.15.19:0/4028174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c100a30 msgr2=0x7fba2c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 --2- 172.21.15.19:0/4028174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c100a30 0x7fba2c100e30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02a410 comp rx=0 tx=0).stop 2024-09-12T03:34:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 -- 172.21.15.19:0/4028174781 shutdown_connections 2024-09-12T03:34:01.049 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 --2- 172.21.15.19:0/4028174781 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c1025e0 0x7fba2c108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 --2- 172.21.15.19:0/4028174781 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c101c20 0x7fba2c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 --2- 172.21.15.19:0/4028174781 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c100a30 0x7fba2c100e30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 -- 172.21.15.19:0/4028174781 >> 172.21.15.19:0/4028174781 conn(0x7fba2c0fc190 msgr2=0x7fba2c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 -- 172.21.15.19:0/4028174781 shutdown_connections 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.046+0000 7fba3290c640 1 -- 172.21.15.19:0/4028174781 wait complete. 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 Processor -- start 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 -- start start 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 0x7fba2c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 0x7fba2c19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10d820 con 0x7fba2c101c20 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10d6a0 con 0x7fba2c1025e0 2024-09-12T03:34:01.050 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba3290c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fba2c10d9a0 con 0x7fba2c100a30 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba2b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:42768/0 (socket says 172.21.15.19:42768) 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba2b7fe640 1 -- 172.21.15.19:0/4270478349 learned_addr learned my addr 172.21.15.19:0/4270478349 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.047+0000 7fba30ea9640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 0x7fba2c19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2bfff640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 0x7fba2c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 -- 172.21.15.19:0/4270478349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 msgr2=0x7fba2c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 0x7fba2c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 -- 172.21.15.19:0/4270478349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 msgr2=0x7fba2c19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 0x7fba2c19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:01.051 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 -- 172.21.15.19:0/4270478349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba2c19fb50 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba30ea9640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 0x7fba2c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2bfff640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 0x7fba2c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba2b7fe640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d950 tx=0x7fba2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20017070 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba20009c30 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba2c19fe40 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2000f6b0 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.048+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba2c1a0380 con 0x7fba2c101c20 2024-09-12T03:34:01.052 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.049+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2c107f10 con 0x7fba2c101c20 2024-09-12T03:34:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.050+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba200109d0 con 0x7fba2c101c20 2024-09-12T03:34:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.051+0000 7fba297fa640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 0x7fba140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.051+0000 7fba2bfff640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 0x7fba140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.051+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fba200cecb0 con 0x7fba2c101c20 2024-09-12T03:34:01.053 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.052+0000 7fba2bfff640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 0x7fba140a5050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000c00 tx=0x7fba1c009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:01.057 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.055+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fba20009de0 con 0x7fba2c101c20 2024-09-12T03:34:01.284 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.284+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fba2c02da00 con 0x7fba2c101c20 2024-09-12T03:34:01.292 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.292+0000 7fba297fa640 1 -- 172.21.15.19:0/4270478349 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740371 (secure 0 0 0) 0x7fba2009bc20 con 0x7fba2c101c20 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 18, 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "active_gid": 14150, 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "active_name": "x", 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addrs": { 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2264764467 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 2264764467 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "active_addr": "172.21.15.19:6801/2264764467", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "active_change": "2024-09-12T03:10:11.180177+0000", 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:34:01.294 INFO:teuthology.orchestra.run.smithi019.stdout: "available": true, 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "standbys": [ 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24106, 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "z", 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.300 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.301 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-12T03:34:01.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.307 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-12T03:34:01.308 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-12T03:34:01.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.316 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-12T03:34:01.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-12T03:34:01.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:01.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.324 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.339 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:34:01.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.342 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-12T03:34:01.345 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:34:01.346 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-12T03:34:01.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:34:01.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.352 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-12T03:34:01.353 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-12T03:34:01.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-12T03:34:01.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.361 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:34:01.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.366 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-12T03:34:01.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:34:01.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.370 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.371 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.374 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:34:01.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-12T03:34:01.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:34:01.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-12T03:34:01.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:34:01.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.395 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:34:01.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:34:01.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:34:01.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-12T03:34:01.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-12T03:34:01.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.404 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-12T03:34:01.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.408 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.409 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-12T03:34:01.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-12T03:34:01.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.417 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-12T03:34:01.421 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.423 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.424 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.428 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:34:01.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:34:01.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:34:01.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.436 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.446 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:34:01.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:34:01.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:34:01.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:34:01.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:34:01.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:34:01.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.462 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.462 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.462 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.464 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:34:01.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.475 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.478 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-12T03:34:01.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.484 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.485 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.487 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.489 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.490 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.493 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-12T03:34:01.495 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.496 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-12T03:34:01.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.503 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.504 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.505 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-12T03:34:01.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.507 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.508 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:34:01.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.520 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-12T03:34:01.521 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.522 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.524 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.525 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.526 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.530 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.532 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.538 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.539 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.540 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:01.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-12T03:34:01.546 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.547 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.550 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.551 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.552 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.553 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.554 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.556 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.558 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-12T03:34:01.559 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.560 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.562 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.566 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.567 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.569 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-12T03:34:01.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.577 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.578 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:34:01.579 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.580 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.583 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.589 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.590 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.591 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.593 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.596 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.597 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:01.598 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:34:01.599 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.600 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.602 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.603 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.606 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.610 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.622 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.625 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.626 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.627 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.629 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.630 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.631 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.637 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:34:01.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.641 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.643 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.644 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:34:01.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-12T03:34:01.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:01.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.649 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-12T03:34:01.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:01.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.655 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.658 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.660 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.667 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.670 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.676 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.679 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.681 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.690 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.693 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.696 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.700 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.703 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-12T03:34:01.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-12T03:34:01.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-12T03:34:01.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.712 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.713 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.715 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.719 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.725 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.727 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.728 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-12T03:34:01.729 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.730 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.731 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.732 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.733 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.734 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.736 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.737 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.738 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.739 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-12T03:34:01.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.741 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.742 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.743 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.744 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.745 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.746 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.747 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.748 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.749 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.750 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.751 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.752 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.753 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.754 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-12T03:34:01.755 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-12T03:34:01.756 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.758 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.760 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.761 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.762 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.763 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.764 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.766 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.767 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.768 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.769 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.770 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:34:01.771 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.772 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.773 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.774 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.775 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.776 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.777 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.778 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.779 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.780 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.781 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.782 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.783 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.784 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.785 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.786 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.787 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "gid": 24113, 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "y", 2024-09-12T03:34:01.788 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.789 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.790 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.791 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.792 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.793 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.794 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.796 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.797 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-12T03:34:01.798 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-12T03:34:01.799 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.800 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.801 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.802 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.803 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-12T03:34:01.804 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:34:01.805 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.806 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:01.807 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.808 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.810 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-12T03:34:01.811 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-12T03:34:01.812 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.814 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.815 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.816 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.817 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.818 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.819 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.820 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.821 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.822 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.823 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.824 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.826 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.827 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.828 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.829 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.830 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:01.832 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-12T03:34:01.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.834 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:34:01.835 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.836 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.837 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:34:01.838 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:34:01.839 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.840 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.841 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.842 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:34:01.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-12T03:34:01.844 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.846 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.847 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-12T03:34:01.848 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.849 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.850 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:34:01.851 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.852 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.853 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.855 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:34:01.856 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.857 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.859 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:34:01.860 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.861 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.862 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:34:01.863 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.864 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.865 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.866 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.867 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.868 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.869 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.869 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.869 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.869 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.872 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.873 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.874 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.875 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.876 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:34:01.877 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.878 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.879 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.880 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.881 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-12T03:34:01.882 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-12T03:34:01.883 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-12T03:34:01.884 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.885 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.887 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.888 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:34:01.889 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.890 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.891 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.892 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.893 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.894 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.895 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.896 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:34:01.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.899 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.901 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.902 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.903 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-12T03:34:01.904 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.906 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.908 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:34:01.909 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:34:01.910 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.911 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.913 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.914 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.915 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.916 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.917 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.918 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.919 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.920 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:34:01.921 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:34:01.922 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.923 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.925 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.926 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.927 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.928 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.929 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.930 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.931 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.932 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.933 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.934 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:34:01.935 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.939 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.940 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.941 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.942 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.943 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.944 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.945 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.946 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.947 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:34:01.948 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.949 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.951 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.952 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.953 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.954 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.955 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-12T03:34:01.957 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:01.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.959 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.960 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.961 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.962 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.963 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.964 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.965 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.966 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.967 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-12T03:34:01.968 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.969 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.970 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.973 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.974 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.977 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.978 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.979 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.980 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.981 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:34:01.982 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:34:01.983 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.984 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.985 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.986 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.987 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.988 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-12T03:34:01.989 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.990 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:01.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.992 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:01.993 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:01.994 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.995 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.996 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:01.997 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:01.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:01.999 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.000 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.001 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.003 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.004 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.005 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.006 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.007 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.009 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.010 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.011 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:34:02.012 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.013 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.014 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.015 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.016 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.019 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.020 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.021 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.022 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.023 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.024 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.025 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.026 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:34:02.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-12T03:34:02.028 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.029 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.030 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.031 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.032 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.033 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.034 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-12T03:34:02.035 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.036 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.037 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.038 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.040 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.041 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.042 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.043 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 msgr2=0x7fba140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 0x7fba140a5050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fba1c000c00 tx=0x7fba1c009560 comp rx=0 tx=0).stop 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 msgr2=0x7fba2c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fba2000d950 tx=0x7fba2000de10 comp rx=0 tx=0).stop 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 shutdown_connections 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba2c1025e0 0x7fba2c19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fba140a2b90 0x7fba140a5050 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fba2c101c20 0x7fba2c198470 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:02.045 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 --2- 172.21.15.19:0/4270478349 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fba2c100a30 0x7fba2c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:02.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 >> 172.21.15.19:0/4270478349 conn(0x7fba2c0fc190 msgr2=0x7fba2c1050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:02.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 shutdown_connections 2024-09-12T03:34:02.046 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:01.298+0000 7fba3290c640 1 -- 172.21.15.19:0/4270478349 wait complete. 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:01 smithi121 ceph-mon[34963]: pgmap v786: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/1206185340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/95853877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:01 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/4270478349' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:01 smithi135 ceph-mon[35474]: pgmap v786: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/1206185340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/95853877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.047 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:01 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/4270478349' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:34:02.050 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.051 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.052 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.053 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.054 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.055 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.057 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.060 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.062 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.063 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.064 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.065 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.068 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.069 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.070 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.071 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.072 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.073 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.074 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.075 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.075 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.075 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.077 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.078 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.079 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.080 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.081 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:02.082 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.083 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.084 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.085 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.086 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.087 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.088 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.089 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.090 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.091 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.092 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.093 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.094 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.095 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.097 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.098 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.099 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.100 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.101 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.102 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-12T03:34:02.103 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.104 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.105 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.106 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.107 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.108 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.109 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.110 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.111 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-12T03:34:02.112 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.113 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-12T03:34:02.114 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.115 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.116 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.117 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.119 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.120 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.121 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-12T03:34:02.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:34:02.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-12T03:34:02.124 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.125 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.128 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.129 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.130 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-12T03:34:02.131 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.132 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.134 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.135 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.136 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.137 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.139 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.141 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.142 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.143 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.144 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.145 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.146 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.147 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.148 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.150 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.152 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.153 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.154 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.155 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.156 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.157 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.158 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.159 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.160 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.162 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.163 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.164 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.165 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.166 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.167 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.168 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.171 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.172 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.173 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.174 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.175 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.176 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.177 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.178 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-12T03:34:02.179 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.181 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.182 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.183 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-12T03:34:02.184 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.185 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.186 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.187 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.188 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.189 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.190 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.191 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.192 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.193 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.194 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.195 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.196 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.197 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.198 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.199 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.200 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.202 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.203 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.204 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.205 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.206 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.209 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.210 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-12T03:34:02.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.212 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.214 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.215 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.216 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.217 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.218 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.219 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.220 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-12T03:34:02.221 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.222 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.223 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-12T03:34:02.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.225 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.226 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.227 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.228 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.229 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.230 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.231 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.232 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.233 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.234 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.235 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.236 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.237 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.238 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.239 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.240 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.241 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.242 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.243 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.244 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.245 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.246 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.247 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.248 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.249 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.250 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.251 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.252 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.253 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.254 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.255 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.257 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.259 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-12T03:34:02.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-12T03:34:02.261 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "modules": [ 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard", 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "iostat", 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "nfs", 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "restful" 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "available_modules": [ 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.263 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "alerts", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.264 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.265 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.267 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.268 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.269 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_destination": { 2024-09-12T03:34:02.270 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_destination", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.271 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_from_name": { 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_from_name", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "Ceph", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Email From: name", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_host": { 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_host", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP server", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.273 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_password": { 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_password", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Password to authenticate with", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.274 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_port": { 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_port", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "465", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP port", 2024-09-12T03:34:02.275 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_sender": { 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_sender", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.276 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_ssl": { 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_ssl", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.277 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "smtp_user": { 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "smtp_user", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.278 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "User to authenticate as", 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "balancer", 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.279 INFO:teuthology.orchestra.run.smithi019.stdout: "active": { 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "active", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_time": { 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_time", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0000", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "begin_weekday": { 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "begin_weekday", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:34:02.282 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_max_iterations": { 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "25", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "250", 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.283 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_metrics": { 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_metrics", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.284 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_compat_step": { 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crush_compat_step", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.5", 2024-09-12T03:34:02.285 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0.001", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "0.999", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "end_time": { 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_time", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.286 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2359", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "end_weekday": { 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "end_weekday", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "0", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "6", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.288 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.289 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.290 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.291 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.292 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min_score": { 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_score", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.294 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "upmap", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-compat", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "none", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "read", 2024-09-12T03:34:02.295 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap-read" 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Balancer mode", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_ids": { 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_ids", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.296 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_deviation": { 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_deviation", 2024-09-12T03:34:02.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.299 INFO:teuthology.orchestra.run.smithi019.stdout: "upmap_max_optimizations": { 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.300 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm", 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_down_multiplier": { 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_down_multiplier", 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.301 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_refresh_rate": { 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_refresh_rate", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.302 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "20", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "agent_starting_port": { 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "agent_starting_port", 2024-09-12T03:34:02.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4721", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_ptrace": { 2024-09-12T03:34:02.304 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_ptrace", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.305 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_interval": { 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_interval", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0.7", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.307 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_log_destination": { 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cephadm_log_destination", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "file", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "file,syslog", 2024-09-12T03:34:02.308 INFO:teuthology.orchestra.run.smithi019.stdout: "syslog" 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "cgroups_split": { 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cgroups_split", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.309 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "config_checks_enabled": { 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_checks_enabled", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.310 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "config_dashboard": { 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_dashboard", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.311 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_alertmanager": { 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_alertmanager", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.312 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_base": { 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_base", 2024-09-12T03:34:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_elasticsearch": { 2024-09-12T03:34:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "elasticsearch container image", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.315 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_grafana": { 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_grafana", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_haproxy": { 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_haproxy", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "HAproxy container image", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.317 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_agent": { 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.318 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_collector": { 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:34:02.319 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_jaeger_query": { 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.320 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Jaeger query container image", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_keepalived": { 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_keepalived", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.321 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Keepalived container image", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_loki": { 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_loki", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.322 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Loki container image", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_node_exporter": { 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_node_exporter", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:34:02.323 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_nvmeof": { 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_nvmeof", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Nvme-of container image", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_prometheus": { 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_prometheus", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.325 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Prometheus container image", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_promtail": { 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_promtail", 2024-09-12T03:34:02.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Promtail container image", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_samba": { 2024-09-12T03:34:02.327 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_samba", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image_snmp_gateway": { 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.329 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "container_init": { 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "container_init", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.330 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "daemon_cache_timeout": { 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:34:02.331 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "900", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.332 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_registry": { 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "default_registry", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "docker.io", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.333 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "device_cache_timeout": { 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_cache_timeout", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1800", 2024-09-12T03:34:02.334 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "device_enhanced_scan": { 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_enhanced_scan", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.335 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "facts_cache_timeout": { 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "facts_cache_timeout", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "grafana_dashboards_path": { 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.337 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "host_check_interval": { 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "host_check_interval", 2024-09-12T03:34:02.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "hw_monitoring": { 2024-09-12T03:34:02.339 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hw_monitoring", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.340 INFO:teuthology.orchestra.run.smithi019.stdout: "inventory_list_all": { 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "inventory_list_all", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.342 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "log_refresh_metadata": { 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_refresh_metadata", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.343 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.344 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.346 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.347 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.348 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "*", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "max_count_per_host": { 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_count_per_host", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.350 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd_draining_count": { 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_osd_draining_count", 2024-09-12T03:34:02.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "migration_current": { 2024-09-12T03:34:02.352 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "migration_current", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "internal - do not modify", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "mode": { 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mode", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "root", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm-package", 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "root" 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.354 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "oob_default_addr": { 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "oob_default_addr", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "169.254.1.1", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.355 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "prometheus_alerts_path": { 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:34:02.356 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_insecure": { 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_insecure", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.357 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_password": { 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_password", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.358 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_url": { 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_url", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.359 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "registry_username": { 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "registry_username", 2024-09-12T03:34:02.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "secure_monitoring_stack": { 2024-09-12T03:34:02.361 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.362 INFO:teuthology.orchestra.run.smithi019.stdout: "service_discovery_port": { 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "service_discovery_port", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8765", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "ssh_config_file": { 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssh_config_file", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.364 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "use_agent": { 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_agent", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.365 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "use_repo_digest": { 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "use_repo_digest", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:34:02.366 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_failed_host_check": { 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:34:02.367 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_daemons": { 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.368 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_on_stray_hosts": { 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.369 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crash", 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.370 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.371 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.373 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.374 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.375 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "retain_interval": { 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retain_interval", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "31536000", 2024-09-12T03:34:02.376 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_recent_interval": { 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_recent_interval", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1209600", 2024-09-12T03:34:02.377 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dashboard", 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.378 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.379 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.380 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.381 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.383 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.384 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.385 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.387 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:34:02.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:34:02.389 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.390 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.391 INFO:teuthology.orchestra.run.smithi019.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.393 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.394 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.395 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_URL": { 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.396 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.397 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.398 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.399 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.400 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.401 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.402 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:34:02.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:34:02.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:34:02.405 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.408 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.409 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.410 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.411 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.412 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:34:02.413 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.414 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.416 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "45", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:34:02.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:34:02.418 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "admin", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.419 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.421 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.422 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.423 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.424 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "cross_origin_url": { 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cross_origin_url", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.425 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "crt_file": { 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "crt_file", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.426 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "debug": { 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "debug", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.427 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "jwt_token_ttl": { 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "jwt_token_ttl", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "28800", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.429 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.431 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.434 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.435 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "motd": { 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "motd", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.436 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "The message of the day", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.437 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.438 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8080", 2024-09-12T03:34:02.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.440 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl_server_port": { 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl_server_port", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.441 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8443", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.442 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "redirect", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "redirect" 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.444 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "url_prefix": { 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url_prefix", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.445 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_monitoring": { 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_monitoring", 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.446 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "monitor device health metrics", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.447 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.448 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.449 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.450 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.451 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.452 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "mark_out_threshold": { 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mark_out_threshold", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "2419200", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.453 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": { 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pool_name", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "device_health_metrics", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.454 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "retention_period": { 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "retention_period", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15552000", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.455 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_frequency": { 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_frequency", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:02.456 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "self_heal": { 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "self_heal", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.457 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.458 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "warn_threshold": { 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "warn_threshold", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.459 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7257600", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "diskprediction_local", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.461 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.462 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.463 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.464 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.465 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "predict_interval": { 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predict_interval", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.466 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "86400", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "predictor_model": { 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "predictor_model", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.467 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "prophetstor", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "600", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "influx", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": false, 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "batch_size": { 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "batch_size", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5000", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.470 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "database": { 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "database", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "ceph", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.471 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "hostname": { 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "hostname", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.472 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.473 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "30", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "5", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.474 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.475 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.476 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.477 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.478 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "password": { 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "password", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.480 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "port": { 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "port", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "8086", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "InfluxDB server port", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.481 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "ssl": { 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ssl", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "false", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.482 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "threads": { 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threads", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "32", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:34:02.483 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "username": { 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "username", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.484 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "verify_ssl": { 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "verify_ssl", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "true", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.485 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "insights", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.488 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.489 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.490 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.491 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.492 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "iostat", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.494 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.495 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.496 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.497 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.498 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "k8sevents", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_event_retention_days": { 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "7", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.500 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "config_check_secs": { 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "config_check_secs", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "10", 2024-09-12T03:34:02.501 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.502 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.503 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.504 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.505 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "localpool", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "failure_domain": { 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "failure_domain", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "host", 2024-09-12T03:34:02.508 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.509 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.510 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.511 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.512 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.513 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": { 2024-09-12T03:34:02.514 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "min_size", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.515 INFO:teuthology.orchestra.run.smithi019.stdout: "num_rep": { 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "num_rep", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": { 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_num", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "128", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.517 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "prefix": { 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prefix", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.518 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "subtree": { 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "subtree", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "rack", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:34:02.519 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mds_autoscaler", 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.521 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.522 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.523 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.524 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.525 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirroring", 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.526 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.527 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.528 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.529 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.531 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "nfs", 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.532 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.533 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.534 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.535 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.537 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "fail_fs": { 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "fail_fs", 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.538 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.540 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.541 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.542 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.544 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator": { 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "orchestrator", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm", 2024-09-12T03:34:02.545 INFO:teuthology.orchestra.run.smithi019.stdout: "rook", 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "test_orchestrator" 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Orchestrator backend", 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_perf_query", 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.546 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.547 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.549 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.550 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.551 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "osd_support", 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.552 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.553 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.554 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.555 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.556 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.557 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "pg_autoscaler", 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.558 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.559 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.560 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.561 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.562 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.563 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.564 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "threshold": { 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "threshold", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "3.0", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1.0", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.565 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "scaling threshold", 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "progress", 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_pg_recovery_event": { 2024-09-12T03:34:02.566 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:02.567 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.569 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.570 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.571 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.572 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.573 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max_completed_events": { 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_completed_events", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "50", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.574 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "sleep_interval": { 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "sleep_interval", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "5", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.575 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "prometheus", 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.576 INFO:teuthology.orchestra.run.smithi019.stdout: "cache": { 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "cache", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.577 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "exclude_perf_counters": { 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "exclude_perf_counters", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:34:02.578 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.579 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.581 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.582 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.583 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools": { 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.584 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "300", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "scrape_interval": { 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "scrape_interval", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.586 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15.0", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.587 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "::", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:02.588 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "9283", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "stale_cache_strategy": { 2024-09-12T03:34:02.589 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stale_cache_strategy", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "log", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.590 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_behaviour": { 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_behaviour", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "default", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "default", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "error" 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.591 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "standby_error_status_code": { 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "standby_error_status_code", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "500", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "400", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "599", 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.592 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.593 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.594 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.595 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.596 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.597 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.598 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_snap_create": { 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.599 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.600 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "trash_purge_schedule": { 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "trash_purge_schedule", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.601 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "restful", 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.602 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_auth": { 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enable_auth", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.603 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "key_file": { 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "key_file", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.604 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.605 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.606 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.607 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.608 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.609 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "server_addr": { 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_addr", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.610 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "server_port": { 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "server_port", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.611 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.612 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rgw", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.613 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.614 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.615 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.616 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.617 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rook", 2024-09-12T03:34:02.618 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.619 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.620 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.621 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.622 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.623 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "storage_class": { 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "storage_class", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.624 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "local", 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.625 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "selftest", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.626 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.627 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.628 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.629 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.630 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "roption1": { 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption1", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.631 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "roption2": { 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "roption2", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.632 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "xyz", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption1": { 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption1", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.633 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption2": { 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption2", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.634 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption3": { 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption3", 2024-09-12T03:34:02.635 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "float", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption4": { 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption4", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption5": { 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption5", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption6": { 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption6", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.639 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rwoption7": { 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rwoption7", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "1", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "42", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.640 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "testkey": { 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testkey", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.641 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "testlkey": { 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testlkey", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.642 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "testnewline": { 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "testnewline", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.643 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snap_schedule", 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.644 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_m_granularity": { 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "allow_m_granularity", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.645 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "dump_on_update": { 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "dump_on_update", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.646 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.647 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.649 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.650 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.651 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "stats", 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.652 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.653 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.655 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.656 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.657 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "status", 2024-09-12T03:34:02.658 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.659 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.662 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.663 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telegraf", 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.664 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "address": { 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "address", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.665 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "15", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.666 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.667 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.668 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.670 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.671 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "telemetry", 2024-09-12T03:34:02.672 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_basic": { 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_basic", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:34:02.673 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_crash": { 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_crash", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.674 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_device": { 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_device", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.675 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_ident": { 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_ident", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.676 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "channel_perf": { 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "channel_perf", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.677 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "contact": { 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "contact", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.678 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "description": { 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "description", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.679 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "device_url": { 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "device_url", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.680 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "enabled": { 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "enabled", 2024-09-12T03:34:02.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.682 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "24", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "8", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.683 INFO:teuthology.orchestra.run.smithi019.stdout: "last_opt_revision": { 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "last_opt_revision", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "1", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.684 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard": { 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.685 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "leaderboard_description": { 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "leaderboard_description", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.686 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.687 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.688 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.689 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.690 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.691 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "organization": { 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "organization", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.692 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "proxy": { 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "proxy", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.693 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "url": { 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "url", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.694 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "test_orchestrator", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.696 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.697 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.699 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-12T03:34:02.701 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.702 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.703 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.704 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.705 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.706 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "max_concurrent_clones": { 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "max_concurrent_clones", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.707 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "4", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "periodic_async_work": { 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "periodic_async_work", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.708 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Periodically check for async work", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_delay": { 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:34:02.709 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "0", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:34:02.710 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "True", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.711 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "can_run": true, 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "error_string": "", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "module_options": { 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "discovery_interval": { 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "discovery_interval", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "uint", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "100", 2024-09-12T03:34:02.712 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "identifier": { 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "identifier", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.713 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "interval": { 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "interval", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "secs", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.714 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "60", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "log_level": { 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_level", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.715 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.716 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster": { 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.717 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_cluster_level": { 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_cluster_level", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "info", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.718 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [ 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "critical", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "debug", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "error", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "info", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "warning" 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": { 2024-09-12T03:34:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "log_to_file", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "bool", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "False", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.720 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_host": { 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_host", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_port": { 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_port", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "int", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "10051", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "zabbix_sender": { 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "zabbix_sender", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "str", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "level": "advanced", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 0, 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "min": "", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "max": "", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "enum_allowed": [], 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "desc": "", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "long_desc": "", 2024-09-12T03:34:02.723 INFO:teuthology.orchestra.run.smithi019.stdout: "tags": [], 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "see_also": [] 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "services": { 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "dashboard": "https://172.21.15.19:8443/" 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "always_on_modules": { 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus": [ 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-12T03:34:02.724 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific": [ 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-12T03:34:02.725 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy": [ 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-12T03:34:02.726 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "reef": [ 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-12T03:34:02.727 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "squid": [ 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "balancer", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "crash", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "devicehealth", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "orchestrator", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscaler", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "progress", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_support", 2024-09-12T03:34:02.728 INFO:teuthology.orchestra.run.smithi019.stdout: "status", 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "telemetry", 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "volumes" 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "last_failure_osd_epoch": 3, 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "active_clients": [ 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "devicehealth", 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1274098548 2024-09-12T03:34:02.729 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "libcephsqlite", 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1299949405 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "rbd_support", 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 4125437594 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "volumes", 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-12T03:34:02.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:0", 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3318220758 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-12T03:34:02.732 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:34:02.738 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:34:02.738 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:34:02.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:01 smithi019 ceph-mon[30953]: pgmap v786: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:02.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/1206185340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/95853877' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:34:02.739 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:01 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/4270478349' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:34:03.011 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:03.518 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1652, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1654, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 42198484, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 525337, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.187060035, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.049104438, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.070916481, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.052714429, 2024-09-12T03:34:03.519 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1652, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.097139179, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000058800 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.520 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1276, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1237, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 76500764, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 474110, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.186236719, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.041769223, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.114274457, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.028272111, 2024-09-12T03:34:03.521 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1236, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.110529336, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000089425 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 755648, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 170352 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.522 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 613, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 818, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 46298473, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 729284, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 7, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.091002868, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.028707747, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.041390550, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.015101803, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 817, 2024-09-12T03:34:03.523 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.046777283, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000057254 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 536576 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-12T03:34:03.524 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5370, 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.146665730, 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000027312 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:03.525 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.526 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 15797, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 0, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 15797, 2024-09-12T03:34:03.527 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 15797, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:03.528 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-09-12T03:34:03.529 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:03.530 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:03.531 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-12T03:34:03.532 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-12T03:34:03.533 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-12T03:34:03.534 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 80319, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 103, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 8, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-12T03:34:03.535 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 4194304, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 167528426, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 167527288, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 80319 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-12T03:34:03.536 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 13, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 17668, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 17655, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 11, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 850, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 839, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 15797, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 47433, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 31636, 2024-09-12T03:34:03.537 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 80388, 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215328 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-12T03:34:03.538 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.539 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.540 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.542 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.543 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.544 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.545 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.546 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.547 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.548 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.549 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.550 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1943, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-12T03:34:03.552 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:03.553 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:03.554 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 240239, 2024-09-12T03:34:03.555 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 80, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 154, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:03.556 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 11164680, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 63316, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-12T03:34:03.557 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 3213, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 3213, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3213, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 22.235795795, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.006920571 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3213, 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3214 2024-09-12T03:34:03.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 3213, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 3213, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 1865, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1346, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-12T03:34:03.559 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-12T03:34:03.560 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-12T03:34:03.561 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 82, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 11, 2024-09-12T03:34:03.562 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 11, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.563 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:03.564 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3537, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 164659997, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3537, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3537, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 164659997, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.565 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3213, 2024-09-12T03:34:03.566 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 163340521, 2024-09-12T03:34:03.567 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3213, 2024-09-12T03:34:03.567 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 163340521, 2024-09-12T03:34:03.567 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:03.567 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:03.568 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3213, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3213, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3213, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3213, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:03.569 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.570 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55d984a0eb40": { 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-12T03:34:03.571 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:03.572 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:34:03.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:03 smithi019 ceph-mon[30953]: pgmap v787: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:03.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:03 smithi135 ceph-mon[35474]: pgmap v787: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:03.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:03 smithi121 ceph-mon[34963]: pgmap v787: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:04.346 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-12T03:34:04.346 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-12T03:34:04.346 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-09-12T03:34:04.347 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-09-12T03:34:04.347 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-12T03:34:04.347 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:34:04.347 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:34:04.608 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 284, 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 281, 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 496736, 2024-09-12T03:34:05.112 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 173024, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.037433483, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018526845, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.020818738, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 280, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.026536089, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000094771 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:05.113 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 496736, 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 173024 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:05.114 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 6, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 216, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3344, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 195520, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-12T03:34:05.115 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.017655301, 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.012066234, 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.003498373, 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 215, 2024-09-12T03:34:05.116 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.013350958, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000062097 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 195520 2024-09-12T03:34:05.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 0, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 2, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 0, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.000221203, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.000129545, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:05.118 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.119 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:05.120 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:05.121 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:05.122 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:05.123 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:05.124 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:05.125 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 147938, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 28, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:05.127 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:05.128 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-12T03:34:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:05.129 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 286, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 456612, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 286, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:05.130 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 286, 2024-09-12T03:34:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 456612, 2024-09-12T03:34:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:05.131 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:05.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:05.132 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:34:05.753 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:05 smithi019 ceph-mon[30953]: pgmap v788: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:05.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:05 smithi135 ceph-mon[35474]: pgmap v788: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:05.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:05 smithi121 ceph-mon[34963]: pgmap v788: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:05.938 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:34:05.938 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:34:05.938 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-12T03:34:05.938 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-12T03:34:05.938 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T03:34:05.939 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:34:05.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:34:06.204 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:06.723 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:34:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:06.723 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1981, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1983, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 103523153, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 629959, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.249180843, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.065250116, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.118288137, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.050356538, 2024-09-12T03:34:06.724 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1981, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.081846247, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000041315 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:06.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 1304, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 1265, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 93292990, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 483663, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.188469459, 2024-09-12T03:34:06.726 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.038474445, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.127155644, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.025759319, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 1264, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.050751813, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000040151 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 756192, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 171648 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 547, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 754, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 63058068, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 622399, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 6, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:06.728 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.101297388, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.024659232, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.057079543, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.014264245, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 753, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.056963783, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000075649 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 450032 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 1, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-MDSRank": { 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 5675, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.144747453, 2024-09-12T03:34:06.730 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000025506 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "queue_len": 0, 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "complete_latency": { 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "mds": { 2024-09-12T03:34:06.731 INFO:teuthology.orchestra.run.smithi019.stdout: "request": 0, 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "reply": 0, 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "reply_latency": { 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.732 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_reply": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "forward": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_commit": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_split": 0, 2024-09-12T03:34:06.733 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_merge": 0, 2024-09-12T03:34:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes": 14809, 2024-09-12T03:34:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_top": 202, 2024-09-12T03:34:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_bottom": 14607, 2024-09-12T03:34:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pin_tail": 0, 2024-09-12T03:34:06.734 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_pinned": 14520, 2024-09-12T03:34:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_expired": 0, 2024-09-12T03:34:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "caps": 0, 2024-09-12T03:34:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "subtrees": 1, 2024-09-12T03:34:06.735 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse": 0, 2024-09-12T03:34:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_hit": 0, 2024-09-12T03:34:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_forward": 0, 2024-09-12T03:34:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_discover": 0, 2024-09-12T03:34:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:06.736 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "traverse_lock": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "load_cent": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "q": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "exported": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "exported_inodes": 0, 2024-09-12T03:34:06.737 INFO:teuthology.orchestra.run.smithi019.stdout: "imported": 0, 2024-09-12T03:34:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "imported_inodes": 0, 2024-09-12T03:34:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:34:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:06.738 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rfiles": 0, 2024-09-12T03:34:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rbytes": 0, 2024-09-12T03:34:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "root_rsnaps": 0, 2024-09-12T03:34:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:34:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:06.739 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "scrub_file_inodes": 0, 2024-09-12T03:34:06.740 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:34:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:34:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:34:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:06.741 INFO:teuthology.orchestra.run.smithi019.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps": 0, 2024-09-12T03:34:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:34:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_cap_release": 0, 2024-09-12T03:34:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: "process_request_cap_release": 0 2024-09-12T03:34:06.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache": { 2024-09-12T03:34:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update": 0, 2024-09-12T03:34:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_update_receipt": 0, 2024-09-12T03:34:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_try_discover": 0, 2024-09-12T03:34:06.743 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_send_discover": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "dir_handle_discover": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_created": 0, 2024-09-12T03:34:06.744 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_enqueued": 0, 2024-09-12T03:34:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "strays_migrated": 0, 2024-09-12T03:34:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:06.745 INFO:teuthology.orchestra.run.smithi019.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_started": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_completed": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:06.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_exportdir": 0, 2024-09-12T03:34:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_flush": 0, 2024-09-12T03:34:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:34:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: "ireq_inodestats": 0 2024-09-12T03:34:06.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log": { 2024-09-12T03:34:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "evlrg": 0, 2024-09-12T03:34:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "evadd": 0, 2024-09-12T03:34:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "evex": 0, 2024-09-12T03:34:06.748 INFO:teuthology.orchestra.run.smithi019.stdout: "evtrm": 0, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "ev": 118602, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "evexg": 0, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "evexd": 0, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "segadd": 0, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "segex": 0, 2024-09-12T03:34:06.749 INFO:teuthology.orchestra.run.smithi019.stdout: "segtrm": 0, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "seg": 134, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "segmjr": 11, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "segexg": 0, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "segexd": 0, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "expos": 29997662, 2024-09-12T03:34:06.750 INFO:teuthology.orchestra.run.smithi019.stdout: "wrpos": 262305975, 2024-09-12T03:34:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "rdpos": 262296446, 2024-09-12T03:34:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "jlat": { 2024-09-12T03:34:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.751 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "replayed": 118602 2024-09-12T03:34:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mem": { 2024-09-12T03:34:06.752 INFO:teuthology.orchestra.run.smithi019.stdout: "ino": 13, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "ino+": 30184, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "ino-": 30171, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dir": 11, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dir+": 1426, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dir-": 1415, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dn": 14809, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dn+": 69282, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "dn-": 54473, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "cap": 0, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "cap+": 0, 2024-09-12T03:34:06.753 INFO:teuthology.orchestra.run.smithi019.stdout: "cap-": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "rss": 92724, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "heap": 215328 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server": { 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_client_request": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "dispatch_server_request": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_request": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_client_session": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "handle_peer_request": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "req_create_latency": { 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.754 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getattr_latency": { 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "req_link_latency": { 2024-09-12T03:34:06.755 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookup_latency": { 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupino_latency": { 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupname_latency": { 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:06.757 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "req_lssnap_latency": { 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mkdir_latency": { 2024-09-12T03:34:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mknod_latency": { 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "req_mksnap_latency": { 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "req_open_latency": { 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.760 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "req_readdir_latency": { 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rename_latency": { 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmdir_latency": { 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.763 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setattr_latency": { 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.764 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setlayout_latency": { 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_setxattr_latency": { 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.765 INFO:teuthology.orchestra.run.smithi019.stdout: "req_symlink_latency": { 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "req_unlink_latency": { 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:06.766 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessions": { 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "session_count": 1, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "session_add": 1, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "session_remove": 0, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_open": 1, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "sessions_stale": 0, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "total_load": 0, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "average_load": 0, 2024-09-12T03:34:06.767 INFO:teuthology.orchestra.run.smithi019.stdout: "avg_session_uptime": 1944, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:06.768 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:06.769 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:06.770 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 238687, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 77, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:06.771 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 154, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 10469128, 2024-09-12T03:34:06.772 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 59364, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter": { 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "op_active": 0, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "op_laggy": 0, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send": 3504, 2024-09-12T03:34:06.773 INFO:teuthology.orchestra.run.smithi019.stdout: "op_send_bytes": 21, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_resend": 3, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_reply": 3504, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_latency": { 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3504, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 19.302822847, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.005508796 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_inflight": 0, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "oplen_avg": { 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3504, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 3505 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op": 3504, 2024-09-12T03:34:06.774 INFO:teuthology.orchestra.run.smithi019.stdout: "op_r": 3504, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_w": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_rmw": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "op_pg": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_stat": 2160, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_create": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_read": 1342, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_write": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writefull": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_writesame": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_append": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_zero": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_truncate": 0, 2024-09-12T03:34:06.775 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_delete": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_mapext": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_getxattr": 1, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_setxattr": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_call": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_watch": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_notify": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:06.776 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "osdop_other": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_active": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_send": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_resend": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "linger_ping": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_active": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_send": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolop_resend": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_active": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_send": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "poolstat_resend": 0, 2024-09-12T03:34:06.777 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_active": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_send": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "statfs_resend": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "command_active": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "command_send": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "command_resend": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "map_epoch": 82, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "map_full": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "map_inc": 2, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_sessions": 11, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_open": 11, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_session_close": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_laggy": 0, 2024-09-12T03:34:06.778 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_wr": 0, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_rd": 2, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_del": 0 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "oft": { 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_objs": 0, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_updates": 0, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "omap_total_removes": 0 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "osd-slow-ops": { 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "slow_ops_count": 0 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.779 INFO:teuthology.orchestra.run.smithi019.stdout: "purge_queue": { 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executing_high_water": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed_ops": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_executed": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:06.780 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 3828, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 259510236, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 3828, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3828, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 259510236, 2024-09-12T03:34:06.781 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:06.783 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:06.784 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3504, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 258139743, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3504, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 258139743, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 1024, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 3504, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 3504, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 3504, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 3504, 2024-09-12T03:34:06.786 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:06.787 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:06.788 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-write_buf_throttle-0x55d7b8780b40": { 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 3758096384, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 0, 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:06.789 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.790 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:06.790 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:34:07.568 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-12T03:34:07.568 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-12T03:34:07.568 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-09-12T03:34:07.568 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-09-12T03:34:07.569 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-12T03:34:07.569 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:34:07.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:34:07.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:07 smithi135 ceph-mon[35474]: pgmap v789: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:07.826 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:07.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:07 smithi121 ceph-mon[34963]: pgmap v789: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:07.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:07 smithi019 ceph-mon[30953]: pgmap v789: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:08.367 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-12T03:34:08.367 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 4, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 4, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 3024, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 448, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 0, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.003303488, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.000042311, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.002309162, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:08.368 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 3, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.004350315, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.001450105 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:08.369 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 286, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 283, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 497280, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 174304, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 2, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 1, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.039096901, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.018383208, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.024363634, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 282, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.013759559, 2024-09-12T03:34:08.370 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000048792 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 497280, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 174304 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_messages": 2, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages": 216, 2024-09-12T03:34:08.371 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_bytes": 320, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_bytes": 195424, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_created_connections": 4, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_active_connections": 2, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_total_time": 0.016641274, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_send_time": 0.013012736, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_recv_time": 0.001486096, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 214, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.018573215, 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000086790 2024-09-12T03:34:08.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: "msgr_send_encrypted_bytes": 195424 2024-09-12T03:34:08.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "cct": { 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "total_workers": 0, 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "unhealthy_workers": 0 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool": { 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:08.374 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:08.375 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:08.376 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:08.377 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_items": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:08.378 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_bytes": 148682, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_anon_items": 29, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bytes": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_items": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:08.379 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_bytes": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_items": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "pgmap_items": 0, 2024-09-12T03:34:08.380 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_bytes": 928, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_co_items": 2, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_1_items": 0, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "unittest_2_items": 0 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:08.381 INFO:teuthology.orchestra.run.smithi019.stdout: "val": 0, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "max": 104857600, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "get_started": 0, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "get": 288, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "get_sum": 456872, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "get_or_fail_success": 288, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "take": 0, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "take_sum": 0, 2024-09-12T03:34:08.382 INFO:teuthology.orchestra.run.smithi019.stdout: "put": 288, 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: "put_sum": 456872, 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: "wait": { 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: "avgcount": 0, 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: "sum": 0.000000000, 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: "avgtime": 0.000000000 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-12T03:34:08.383 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-12T03:34:09.094 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:34:09.094 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:34:09.095 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-12T03:34:09.095 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-12T03:34:09.095 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T03:34:09.095 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:34:09.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:34:09.661 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:34:09.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:09 smithi019 ceph-mon[30953]: pgmap v790: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:09.782 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:09 smithi121 ceph-mon[34963]: pgmap v790: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:09.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:09 smithi135 ceph-mon[35474]: pgmap v790: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:10.211 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-12T03:34:10.211 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 93442, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 110452, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 30945790, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 77066547, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 16, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 5.765935474, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 2.192296336, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 2.546182207, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.104570959, 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:10.212 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 110451, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 4.058275345, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000036742 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 113836, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.018680953, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000164 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 800272, 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 266736 2024-09-12T03:34:10.213 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 304000, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 402605, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 109807595, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 263921056, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 14, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 7, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 15.377118330, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 6.787485877, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 11.282389219, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.192018361, 2024-09-12T03:34:10.214 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 402604, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 20.059711750, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000049824 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 521889, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.088910429, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000170 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 454640 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.215 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 75485, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 88616, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 23601291, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 109556517, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 41, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 7, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 5.173356986, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 2.007577535, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 1.706844274, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.315921807, 2024-09-12T03:34:10.216 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 88614, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3.816322691, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000043066 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 61750, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.010367075, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000167 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:10.217 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 85250, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 63.821788179, 2024-09-12T03:34:10.218 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000748642 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 40121, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 7.839254979, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000195390 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 133408, 2024-09-12T03:34:10.219 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 121387, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 121387, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 2164.558855947, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.017831883 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 12019, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 11, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 434, 2024-09-12T03:34:10.220 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 47, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 36, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 12, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 12, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 12, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 15, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 13679, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 5, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 241192, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 184094, 2024-09-12T03:34:10.221 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 11924, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 1140, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 330, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 1000, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 1, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 287, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 8397, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 269, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 31359, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 109177, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:10.222 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 6674, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 147760442, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 109177, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 574, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 574, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 108603, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 309, 2024-09-12T03:34:10.223 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 47670, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 53422, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 58105, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 9316, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 940, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 38895 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 3848, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 9778, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 5443, 2024-09-12T03:34:10.224 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 7246, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 5213, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 21619, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 21619, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-12T03:34:10.225 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 12, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 86423, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 294, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 47, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 1, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 87375, 2024-09-12T03:34:10.226 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 87375, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 113, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 113, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 9, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-12T03:34:10.227 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 187856491, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 4193, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 216.590915545, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.051655357 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.228 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 43, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 26768, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 26725, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 39, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 5723, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 5684, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 36, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 70490, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 70454, 2024-09-12T03:34:10.229 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 37059, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 37059, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 137596, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 223520 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 149682, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 4131, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 133408, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 244, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 12471, 2024-09-12T03:34:10.230 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 8268, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 76.968128640, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.009309159 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 439, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 5.631077376, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.012827055 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.231 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 79857, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1685.380783873, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.021104984 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.232 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.233 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.234 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 54, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 4.360583565, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.080751547 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-12T03:34:10.235 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3075, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 8.020867470, 2024-09-12T03:34:10.236 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.002608412 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 872, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1.227775016, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001407998 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 8231, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 340.611608879, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.041381558 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.237 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 149, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 13.287936409, 2024-09-12T03:34:10.238 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.089180781 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-12T03:34:10.239 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.240 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-12T03:34:10.241 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 20442, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 29.070094719, 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001422076 2024-09-12T03:34:10.242 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2024-09-12T03:34:10.243 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 7551, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 7551, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 807, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:10.244 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:10.245 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:10.246 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 304183, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 108, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:10.247 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 154, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 173178, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 769, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:10.248 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 138444, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 195453104, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 28, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 138444, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 138444, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 463.928392313, 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.003351018 2024-09-12T03:34:10.249 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 138444, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 139546 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 138444, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 109880, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 28564, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 0, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 10, 2024-09-12T03:34:10.250 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 692, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 4885, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 395, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 47, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 22394, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 109179, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 20, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:10.251 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-12T03:34:10.252 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-12T03:34:10.253 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 82, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 12, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 12, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 1594, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 22, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 308 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-12T03:34:10.254 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 86276 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 14, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 6, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 22339, 2024-09-12T03:34:10.255 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 21619, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 472923, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 121768572, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 472923, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:10.256 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 472923, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 121768572, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:10.257 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 138444, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 197637167, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 138444, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 197637167, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.258 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 138444, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 138444, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 138444, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 138444, 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:10.259 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 21619, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 2184239, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 21619, 2024-09-12T03:34:10.260 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 692, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 2184239, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x564a3b7d2960": { 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:10.261 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 87375, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 183662187, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 87375, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 4193, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 183662187, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:10.262 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.263 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:10.263 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-12T03:34:11.144 INFO:tasks.check_counter:Daemon mds.b mds.exported=287 2024-09-12T03:34:11.144 INFO:tasks.check_counter:Daemon mds.b mds.imported=269 2024-09-12T03:34:11.144 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=3848 2024-09-12T03:34:11.144 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=9778 2024-09-12T03:34:11.144 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=47 2024-09-12T03:34:11.144 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:34:11.145 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:34:11.390 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:34:11.682 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:11 smithi121 ceph-mon[34963]: pgmap v791: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:11.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:11 smithi135 ceph-mon[35474]: pgmap v791: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:11.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:11 smithi019 ceph-mon[30953]: pgmap v791: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:11.933 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-12T03:34:11.933 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:11.933 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 749, 2024-09-12T03:34:11.933 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 749, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 63114081, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 237422, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.093672389, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.020815962, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.050775313, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.016060890, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 748, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.030050337, 2024-09-12T03:34:11.934 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000040174 2024-09-12T03:34:11.935 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.935 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:11.935 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.935 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.935 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.936 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.936 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:34:11.936 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:34:11.936 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.936 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:11.937 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1129, 2024-09-12T03:34:11.937 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1085, 2024-09-12T03:34:11.937 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 51316241, 2024-09-12T03:34:11.937 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 427981, 2024-09-12T03:34:11.937 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.131039051, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.035380843, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.074497595, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.022054736, 2024-09-12T03:34:11.938 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1084, 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.039276841, 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000036233 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.939 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 782944, 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 174848 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1845, 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2058, 2024-09-12T03:34:11.940 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 74137671, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 1038296, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 6, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.205391404, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.055948110, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.081869524, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.051255647, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2056, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.076266222, 2024-09-12T03:34:11.941 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000037094 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:34:11.942 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 451904 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 5580, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.135474593, 2024-09-12T03:34:11.943 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000024278 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.944 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.945 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 10967, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 40, 2024-09-12T03:34:11.946 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 10927, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 10909, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 1, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2024-09-12T03:34:11.947 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-12T03:34:11.948 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:34:11.949 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2024-09-12T03:34:11.951 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:11.952 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:34:11.953 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 86289, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-12T03:34:11.954 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 107, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 9, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 190897573, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 190896435, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.955 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 86289 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 14, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 19265, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 19251, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 19, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 1010, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 991, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 10967, 2024-09-12T03:34:11.956 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 45076, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 34109, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 80540, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 215328 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2024-09-12T03:34:11.957 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:11.958 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.959 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.960 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.961 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.962 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.963 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-12T03:34:11.963 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.963 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.963 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.963 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.964 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.965 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.966 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.967 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.968 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.969 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.970 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.971 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-12T03:34:11.972 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2181, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:11.973 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:11.974 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:11.975 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:11.976 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 232150, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 77, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:11.977 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 154, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 7777625, 2024-09-12T03:34:11.978 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 45115, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 3385, 2024-09-12T03:34:11.979 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2024-09-12T03:34:11.980 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2024-09-12T03:34:11.980 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 3385, 2024-09-12T03:34:11.980 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3385, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 16.350765716, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004830359 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3385, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3386 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 3385, 2024-09-12T03:34:11.981 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 3385, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 2053, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 1330, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2024-09-12T03:34:11.982 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:11.983 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-12T03:34:11.984 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-12T03:34:11.985 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 82, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 10, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 10, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.986 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:34:11.987 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3719, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 188213333, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:11.988 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3719, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3719, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 188213333, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:11.989 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3385, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 186837851, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3385, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 186837851, 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:11.990 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:11.991 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3385, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 3385, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3385, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 3385, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.992 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x55c1931f8b40": { 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:11.994 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:11.995 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-12T03:34:12.756 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:34:12.756 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:34:12.756 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-12T03:34:12.756 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-09-12T03:34:12.756 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T03:34:12.757 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:34:12.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:34:13.005 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:34:13.492 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-12T03:34:13.492 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:13.492 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 2232, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 2234, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 63569430, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 709783, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.202478221, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.056599986, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.077192034, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.053725446, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 2232, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.151315852, 2024-09-12T03:34:13.493 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000067793 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 488, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 698, 2024-09-12T03:34:13.494 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 62159650, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 607448, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 7, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.075463977, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.018117410, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.042412559, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.010308593, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 697, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.029849990, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000042826 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:13.495 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 453792 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 1047, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 1003, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 59690059, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 401256, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 5, 2024-09-12T03:34:13.496 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 0.112990494, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 0.029814889, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 0.069022002, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 0.016254763, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1002, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.034149478, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000034081 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.497 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 782400, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 173568 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 5704, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.125961489, 2024-09-12T03:34:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000022083 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 0, 2024-09-12T03:34:13.499 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 21634, 2024-09-12T03:34:13.500 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 21634, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 21633, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 3, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:13.501 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-12T03:34:13.502 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:13.503 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 0 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:13.504 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 0, 2024-09-12T03:34:13.505 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 87375, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 0, 2024-09-12T03:34:13.506 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 113, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 9, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 4194304, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 187856491, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 187846963, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.507 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 87375 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 18, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 26080, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 26062, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 16, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 1202, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 1186, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 21634, 2024-09-12T03:34:13.508 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 67656, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 46022, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 78896, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 215328 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 0, 2024-09-12T03:34:13.509 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 0, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.510 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.511 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.512 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.513 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.514 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-12T03:34:13.515 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.515 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.515 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.515 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.516 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.517 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.518 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.519 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:13.520 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.521 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.522 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.523 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.524 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 2, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 1, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 0, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 0, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 2184, 2024-09-12T03:34:13.525 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:13.526 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:13.527 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:13.528 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 239431, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 78, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-12T03:34:13.529 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 154, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 15291293, 2024-09-12T03:34:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 86732, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 3431, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 21, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 3, 2024-09-12T03:34:13.531 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 3431, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3431, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 12.164825867, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.003545562 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 3431, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 3432 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.532 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 3431, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 3431, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 2054, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 1375, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-12T03:34:13.533 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 1, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:13.534 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-12T03:34:13.535 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-12T03:34:13.536 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 82, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 11, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 11, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 0, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 2, 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 0 2024-09-12T03:34:13.537 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 0, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 0 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-12T03:34:13.538 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:13.539 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 3763, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 185060623, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 3763, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3763, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 185060623, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.540 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:13.541 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3431, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 183677407, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3431, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 183677407, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:13.542 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 3431, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 3431, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 3431, 2024-09-12T03:34:13.543 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 3431, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:13.544 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.545 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x556964072b40": { 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:13.546 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 0, 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 0, 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:13.547 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-12T03:34:13.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:13 smithi121 ceph-mon[34963]: pgmap v792: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:13.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:13 smithi135 ceph-mon[35474]: pgmap v792: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:13.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:13 smithi019 ceph-mon[30953]: pgmap v792: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:14.215 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-12T03:34:14.215 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-12T03:34:14.215 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-12T03:34:14.215 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-09-12T03:34:14.215 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T03:34:14.215 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:34:14.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:34:14.461 INFO:teuthology.orchestra.run.smithi121.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 338477, 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 463041, 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 110363552, 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 296018629, 2024-09-12T03:34:15.017 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 16, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 17.161645238, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 7.762550082, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 13.353380379, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.654669716, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 463038, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 22.707893243, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000049041 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:15.018 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 552122, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.099595859, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000180 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 147338, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 137141, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 55613805, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 120269376, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 9, 2024-09-12T03:34:15.019 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 5, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 7.895495503, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 2.948169510, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 3.581356499, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.652181299, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 137142, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 4.814109771, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000035103 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 137400, 2024-09-12T03:34:15.020 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.020632421, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000150 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 786992, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 250512 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_messages": 90725, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages": 87818, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_bytes": 26178266, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_bytes": 134396992, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_created_connections": 45, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_active_connections": 6, 2024-09-12T03:34:15.021 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_total_time": 5.196698397, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_send_time": 1.731547978, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_recv_time": 1.688820001, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_running_fast_dispatch_time": 1.623308136, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 87816, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 4.566914771, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000052005 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 43063, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.006328873, 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000146 2024-09-12T03:34:15.022 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "msgr_send_encrypted_bytes": 455328 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "cct": { 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "total_workers": 1, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "unhealthy_workers": 0 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-MDSRank": { 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 138511, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 98.701062058, 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000712586 2024-09-12T03:34:15.023 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "queue_len": 0, 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "complete_latency": { 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 27720, 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 4.410929698, 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000159124 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": { 2024-09-12T03:34:15.024 INFO:teuthology.orchestra.run.smithi121.stdout: "request": 109222, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "reply": 105901, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "reply_latency": { 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 105901, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1607.464334351, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.015178934 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_reply": 0, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "forward": 3321, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_complete": 37, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:15.025 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_commit": 950, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_split": 99, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_merge": 0, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes": 22, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_top": 4, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_bottom": 8, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pin_tail": 10, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_pinned": 13, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_expired": 39021, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "inodes_with_caps": 1, 2024-09-12T03:34:15.026 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": 1, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "subtrees": 4, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse": 190106, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_hit": 159789, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_forward": 3020, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_discover": 1391, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "traverse_lock": 836, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "load_cent": 1000, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "q": 1, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "exported": 327, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "exported_inodes": 28711, 2024-09-12T03:34:15.027 INFO:teuthology.orchestra.run.smithi121.stdout: "imported": 363, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "imported_inodes": 18875, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_backtrace_fetch": 178389, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rfiles": 6674, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rbytes": 147760442, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "root_rsnaps": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_fetch": 178389, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_inodes": 1438, 2024-09-12T03:34:15.028 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_dirfrag_rstats": 1438, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "scrub_file_inodes": 176951, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_inode_file_caps": 98, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_revoke": 76433, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_grant": 62763, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps": 96855, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_caps_dirty": 21704, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_cap_release": 2558, 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: "process_request_cap_release": 41553 2024-09-12T03:34:15.029 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_cache": { 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update": 4122, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_update_receipt": 8739, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_try_discover": 4932, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_send_discover": 6477, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "dir_handle_discover": 6584, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays": 0, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_created": 14799, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_enqueued": 14799, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "strays_migrated": 0, 2024-09-12T03:34:15.030 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_started": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "recovery_completed": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_path": 12, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_quiesce_inode": 158334, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_exportdir": 330, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_flush": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragmentdir": 100, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "ireq_inodestats": 0 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.031 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_log": { 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evlrg": 1, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evadd": 118602, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evex": 17466, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evtrm": 12291, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "ev": 106311, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evexg": 0, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "evexd": 5175, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "segadd": 150, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "segex": 22, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "segtrm": 16, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "seg": 134, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "segmjr": 9, 2024-09-12T03:34:15.032 INFO:teuthology.orchestra.run.smithi121.stdout: "segexg": 0, 2024-09-12T03:34:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "segexd": 6, 2024-09-12T03:34:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "expos": 29997662, 2024-09-12T03:34:15.033 INFO:teuthology.orchestra.run.smithi121.stdout: "wrpos": 262305975, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "rdpos": 0, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "jlat": { 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 7183, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 580.202078449, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.080774339 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "replayed": 0 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_mem": { 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "ino": 29, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "ino+": 31658, 2024-09-12T03:34:15.034 INFO:teuthology.orchestra.run.smithi121.stdout: "ino-": 31629, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dir": 26, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dir+": 5329, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dir-": 5303, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dn": 22, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dn+": 75879, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "dn-": 75857, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "cap": 1, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "cap+": 41431, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "cap-": 41430, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "rss": 167428, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "heap": 223520 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.035 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_server": { 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_client_request": 119221, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "dispatch_server_request": 5175, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_request": 109222, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_client_session": 320, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "handle_peer_request": 13727, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "req_create_latency": { 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 19146, 2024-09-12T03:34:15.036 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 47.562933307, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.002484222 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getattr_latency": { 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 49, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 44.450553267, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.907154148 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.037 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "req_link_latency": { 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookup_latency": { 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 45214, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 1076.829617173, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.023816287 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.038 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupino_latency": { 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 1, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000185548, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000185548 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupname_latency": { 2024-09-12T03:34:15.039 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.040 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "req_lssnap_latency": { 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mkdir_latency": { 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 136, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 5.838961684, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.042933541 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mknod_latency": { 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.041 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "req_mksnap_latency": { 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "req_open_latency": { 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 8326, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 59.775504800, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.007179378 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.042 INFO:teuthology.orchestra.run.smithi121.stdout: "req_readdir_latency": { 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 607, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.778331704, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001282259 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rename_latency": { 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 19202, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 345.345677558, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.017984880 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.043 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmdir_latency": { 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 80, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 8.425141908, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.105314273 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.044 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setattr_latency": { 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.045 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.046 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setlayout_latency": { 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "req_setxattr_latency": { 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "req_symlink_latency": { 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.047 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "req_unlink_latency": { 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 13140, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 18.457427402, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.001404674 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.048 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_sessions": { 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "session_count": 1, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "session_add": 1, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "session_remove": 0, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_open": 1, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "sessions_stale": 0, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "total_load": 18960, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "average_load": 18960, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "avg_session_uptime": 898, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.049 INFO:teuthology.orchestra.run.smithi121.stdout: "mempool": { 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:15.050 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:15.051 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_bytes": 303630, 2024-09-12T03:34:15.052 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_anon_items": 106, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_items": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_items": 154, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:15.053 INFO:teuthology.orchestra.run.smithi121.stdout: "pgmap_items": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_bytes": 116813, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_co_items": 581, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_1_items": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "unittest_2_items": 0 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "objecter": { 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "op_active": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "op_laggy": 0, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send": 205507, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "op_send_bytes": 280682811, 2024-09-12T03:34:15.054 INFO:teuthology.orchestra.run.smithi121.stdout: "op_resend": 29, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "op_reply": 205507, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "op_latency": { 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 205507, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 893.321352523, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.004346914 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "op_inflight": 0, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "oplen_avg": { 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 205507, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 211038 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "op": 205507, 2024-09-12T03:34:15.055 INFO:teuthology.orchestra.run.smithi121.stdout: "op_r": 178917, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "op_w": 26590, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "op_rmw": 0, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "op_pg": 0, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_stat": 46, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_create": 1874, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_read": 491, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_write": 7674, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writefull": 289, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_writesame": 0, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_append": 0, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_zero": 0, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_truncate": 99, 2024-09-12T03:34:15.056 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_delete": 15302, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_mapext": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_getxattr": 178389, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_setxattr": 3748, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_call": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_watch": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_notify": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:15.057 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "osdop_other": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_active": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_send": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_resend": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "linger_ping": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_active": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_send": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolop_resend": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_active": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_send": 0, 2024-09-12T03:34:15.058 INFO:teuthology.orchestra.run.smithi121.stdout: "poolstat_resend": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_active": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_send": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "statfs_resend": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "command_active": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "command_send": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "command_resend": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "map_epoch": 82, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "map_full": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "map_inc": 2, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_sessions": 12, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_open": 12, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_session_close": 0, 2024-09-12T03:34:15.059 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_laggy": 0, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_wr": 2634, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_rd": 74, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_del": 418 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "oft": { 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_objs": 1, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_updates": 0, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "omap_total_removes": 0 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "osd-slow-ops": { 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: "slow_ops_count": 158120 2024-09-12T03:34:15.060 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "purge_queue": { 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_ops_high_water": 14, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing": 0, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executing_high_water": 6, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed_ops": 15223, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_executed": 14799, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:15.061 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 576536, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 140244479, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 576536, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 576536, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 140244479, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.062 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 104857600, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 205507, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 282177342, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 205507, 2024-09-12T03:34:15.063 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 282177342, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 1024, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 0, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 0, 2024-09-12T03:34:15.064 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 205507, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 205507, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 205507, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 205507, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:15.065 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 14799, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 1495123, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 14799, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 491, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 1495123, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.066 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "throttle-write_buf_throttle-0x561c47388960": { 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "val": 0, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "max": 3758096384, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "get_started": 0, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "get": 118602, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "get_sum": 258111671, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "get_or_fail_success": 118602, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "take": 0, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "take_sum": 0, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "put": 7183, 2024-09-12T03:34:15.067 INFO:teuthology.orchestra.run.smithi121.stdout: "put_sum": 258111671, 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "wait": { 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgcount": 0, 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "sum": 0.000000000, 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: "avgtime": 0.000000000 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-09-12T03:34:15.068 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-09-12T03:34:15.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:15 smithi121 ceph-mon[34963]: pgmap v793: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:15.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:15 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:15 smithi135 ceph-mon[35474]: pgmap v793: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:15.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:15 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:15.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:15 smithi019 ceph-mon[30953]: pgmap v793: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:15.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:15 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:15.917 INFO:tasks.check_counter:Daemon mds.k mds.exported=327 2024-09-12T03:34:15.918 INFO:tasks.check_counter:Daemon mds.k mds.imported=363 2024-09-12T03:34:15.918 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=4122 2024-09-12T03:34:15.918 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=8739 2024-09-12T03:34:15.918 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=99 2024-09-12T03:34:15.918 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:34:15.918 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:34:16.256 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:34:16.640 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:16 smithi135 ceph-mon[35474]: pgmap v794: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:16.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:16 smithi121 ceph-mon[34963]: pgmap v794: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:16.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:16 smithi019 ceph-mon[30953]: pgmap v794: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 60940, 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 65330, 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 23328438, 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 127244716, 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 16, 2024-09-12T03:34:16.968 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 5.535696011, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 2.573603255, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 2.019915844, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.120461835, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 65332, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.371115702, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000066906 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 69268, 2024-09-12T03:34:16.969 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.013618922, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000196 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 805136, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 265040 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 76307, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 69855, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 22764677, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 76320267, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 11, 2024-09-12T03:34:16.970 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 5.080503186, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 2.166999916, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 2.099633288, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.948907023, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 69855, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3.336390325, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000047761 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 78691, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.012610149, 2024-09-12T03:34:16.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000160 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 454512 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 259429, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 350924, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 85649613, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 190104434, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 44, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 10, 2024-09-12T03:34:16.972 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 13.742114763, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 5.539908294, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 12.338641457, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.362962059, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 350918, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 28.156042773, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000080235 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 440567, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.105773869, 2024-09-12T03:34:16.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000240 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 1584, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 1440 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 62961, 2024-09-12T03:34:16.974 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 80.814895011, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001283570 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 20594, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.943678583, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000240054 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-09-12T03:34:16.975 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 95870, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 82165, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 82165, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1414.993301202, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.017221363 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 13705, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 686, 2024-09-12T03:34:16.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 75, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 36, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 14, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 11, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 11, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 14, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 22593, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 4, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 163731, 2024-09-12T03:34:16.977 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 127158, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 13502, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 1341, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 408, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 112, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 15713, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 157, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 9259, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:16.978 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 79931, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 6674, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 147760442, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 79931, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 630, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 630, 2024-09-12T03:34:16.979 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 79301, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 384, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 58831, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 47070, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 73745, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 15264, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 1615, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 28590 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-09-12T03:34:16.980 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 2508, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 10921, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 6126, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 7783, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 4741, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 1, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 10699, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 10698, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:16.981 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 12, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 73151, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 112, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 79, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.982 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 1, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 86289, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 86289, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 107, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 107, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 9, 2024-09-12T03:34:16.983 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 190897573, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4628, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 216.492773352, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.046778905 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.984 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 41, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 20278, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 20237, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 45, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 6086, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 6041, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 36, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 49403, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 49367, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 29101, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 29100, 2024-09-12T03:34:16.985 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 141524, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 223520 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 104687, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 5795, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 95870, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 256, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 12600, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 13867, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 157.704041665, 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.011372614 2024-09-12T03:34:16.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 93, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 6.631296062, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.071304258 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.987 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 39085, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 741.286952672, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.018966021 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000112599, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000112599 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.990 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.990 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:16.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.990 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 96, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 21.505314026, 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.224013687 2024-09-12T03:34:16.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-09-12T03:34:16.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 5514, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 33.721863639, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.006115680 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 426, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.517160868, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001213992 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 13896, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 394.814426113, 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.028412091 2024-09-12T03:34:16.993 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 81, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 47.624736698, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.587959712 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.995 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.996 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.997 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-09-12T03:34:16.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9106, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 11.187396860, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001228574 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:16.999 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 767, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 767, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 879, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.000 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:17.001 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:17.002 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 287734, 2024-09-12T03:34:17.003 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 104, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 154, 2024-09-12T03:34:17.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 178375, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 921, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-09-12T03:34:17.005 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 98274, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 202449910, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 33, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 98274, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 98274, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 336.832822608, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003427486 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 98274, 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 99632 2024-09-12T03:34:17.006 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 98274, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 80459, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 17815, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 10, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 528, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 5156, 2024-09-12T03:34:17.007 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 332, 2024-09-12T03:34:17.008 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-09-12T03:34:17.008 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-09-12T03:34:17.008 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-09-12T03:34:17.008 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 75, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 11186, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 79931, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 20, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-09-12T03:34:17.009 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-09-12T03:34:17.010 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 82, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-09-12T03:34:17.011 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 2048, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 346 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-09-12T03:34:17.012 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 73032 2024-09-12T03:34:17.013 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.013 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-09-12T03:34:17.013 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:17.013 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 14, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 6, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 11130, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 10698, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 396672, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 96019260, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:17.014 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 396672, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 396672, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 96019260, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:17.015 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 98274, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 203530840, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 98274, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 203530840, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:17.016 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 98274, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 98274, 2024-09-12T03:34:17.017 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 98274, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 98274, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 10698, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1080930, 2024-09-12T03:34:17.018 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 10698, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 528, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1080930, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55a38eb5c960": { 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:17.019 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 86289, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 186703269, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 86289, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4628, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 186703269, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:17.020 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:17.021 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.021 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:17.021 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-12T03:34:18.127 INFO:tasks.check_counter:Daemon mds.c mds.exported=112 2024-09-12T03:34:18.127 INFO:tasks.check_counter:Daemon mds.c mds.imported=157 2024-09-12T03:34:18.127 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=2508 2024-09-12T03:34:18.127 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=10921 2024-09-12T03:34:18.127 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=75 2024-09-12T03:34:18.127 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:34:18.128 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:34:18.385 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:34:18.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:18 smithi135 ceph-mon[35474]: pgmap v795: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 2055, 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 2057, 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 143229079, 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 653500, 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:18.959 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.251106327, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.050833035, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.128356145, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.056292220, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 2055, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.070255285, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000034187 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.960 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 568, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 782, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 106396034, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 723702, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 8, 2024-09-12T03:34:18.961 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 6, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.106959337, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.021778990, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.067197317, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.012494529, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 780, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.029323106, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000037593 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.962 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 543648 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 1519, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 1472, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 135320153, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 551019, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 5, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:18.963 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 0.197754487, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 0.043372592, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 0.123560004, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 0.029924454, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 1472, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.054701738, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000037161 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.964 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 793216, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 176096 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 6135, 2024-09-12T03:34:18.965 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.163217781, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000026604 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.966 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 0, 2024-09-12T03:34:18.967 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 21908, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 1019, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 20889, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 20451, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 3, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 0, 2024-09-12T03:34:18.968 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 0, 2024-09-12T03:34:18.969 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:34:18.970 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 0, 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 0 2024-09-12T03:34:18.971 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:18.972 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:18.973 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 165834, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-09-12T03:34:18.974 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 129, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 15, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 149756622, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 387256561, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 387252260, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 165834 2024-09-12T03:34:18.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 23, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 52259, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 52236, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 19, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 2465, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 2446, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 21908, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 120479, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 98571, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 0, 2024-09-12T03:34:18.976 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 0, 2024-09-12T03:34:18.977 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 82632, 2024-09-12T03:34:18.977 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 215328 2024-09-12T03:34:18.977 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-09-12T03:34:18.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.979 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.980 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.981 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.982 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.983 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.983 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.984 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.985 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-09-12T03:34:18.987 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.988 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.989 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.990 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.991 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:18.992 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 0, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 0, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 2190, 2024-09-12T03:34:18.993 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:18.994 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:18.995 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:18.996 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 239489, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 79, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:18.997 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 154, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 15499665, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 88134, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-09-12T03:34:18.998 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 3800, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 26, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 4, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 3800, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3800, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 15.092677751, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003971757 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-09-12T03:34:18.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3800, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 3801 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 3800, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 3800, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 0, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-09-12T03:34:19.000 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 2269, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 1528, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 2, 2024-09-12T03:34:19.001 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-09-12T03:34:19.002 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-09-12T03:34:19.003 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 82, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 11, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 11, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 0, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 2, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 0 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-09-12T03:34:19.004 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 0 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 0, 2024-09-12T03:34:19.005 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 0, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 4138, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 384552702, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 4138, 2024-09-12T03:34:19.006 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4138, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 384552702, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:19.007 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3800, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 383100756, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3800, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 383100756, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:19.008 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 3800, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 3800, 2024-09-12T03:34:19.009 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 3800, 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 3800, 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:19.010 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:19.011 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55e1a4254b40": { 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:19.012 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 0, 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 0, 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:19.013 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-12T03:34:19.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:18 smithi121 ceph-mon[34963]: pgmap v795: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:19.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:18 smithi019 ceph-mon[30953]: pgmap v795: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:20.097 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-09-12T03:34:20.097 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-09-12T03:34:20.097 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-09-12T03:34:20.097 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-09-12T03:34:20.097 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T03:34:20.097 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:34:20.098 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:34:20.357 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:34:20.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:20 smithi135 ceph-mon[35474]: pgmap v796: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:20.943 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 118175, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 112993, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 51731670, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 155600635, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 36, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 10.310018042, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 3.694459866, 2024-09-12T03:34:20.944 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 3.786572424, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 3.323041800, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 112992, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 6.077775145, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000053789 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 144398, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.023781536, 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000164 2024-09-12T03:34:20.945 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 815088, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 204608 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 410143, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 590500, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 129786480, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 459163565, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 17, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 4, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 26.408588584, 2024-09-12T03:34:20.946 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 9.660288141, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 21.516810404, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 6.322656473, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 590499, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 32.028829206, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000054240 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 734710, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.128840240, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000175 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.947 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 55312, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 494480 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 106627, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 105319, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 31303088, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 179363269, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 170, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 10, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 7.625959911, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 2.796629291, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 3.353647234, 2024-09-12T03:34:20.948 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.860044563, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 105317, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 5.579018488, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000052973 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 131037, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.023023768, 2024-09-12T03:34:20.949 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000175 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-09-12T03:34:20.950 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 89614, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 172.716261535, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001927335 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 40526, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 7.992851156, 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000197227 2024-09-12T03:34:20.951 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 219673, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 204433, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 204433, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 5335.282879217, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.026097953 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 15231, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 22, 2024-09-12T03:34:20.952 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 1, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 1216, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 129, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 6, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 37, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 13, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 13, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 11, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 18, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 72640, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 1, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 1, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 5, 2024-09-12T03:34:20.953 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 363411, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 296228, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 14774, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 1015, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 151, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 487, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 54964, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 500, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 36016, 2024-09-12T03:34:20.954 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 116790, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 4, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 234, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 116790, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 1665, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 17, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 1648, 2024-09-12T03:34:20.955 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 115125, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 291, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 65032, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 76571, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 7, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 91837, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 28600, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 2792, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 58440 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.956 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 5031, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 6647, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 3656, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 4730, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 11404, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 2, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 21855, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 21853, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-09-12T03:34:20.957 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 12, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 93833, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 17, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 487, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 138, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-09-12T03:34:20.958 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 2, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 165834, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 61904, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 61451, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 104383, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 12743, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 204, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 76, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 75, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 129, 2024-09-12T03:34:20.959 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 9, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 16, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 149756622, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 387256561, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 13955, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 796.068180544, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.057045373 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-09-12T03:34:20.960 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.963 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-09-12T03:34:20.963 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 41, 2024-09-12T03:34:20.963 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 53409, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 53368, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 35, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 5904, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 5869, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 37, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 129059, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 129022, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 1, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 56900, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 56899, 2024-09-12T03:34:20.964 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 151292, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 231712 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 226113, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 16469, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 219673, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 433, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 32990, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 33473, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 242.789037146, 2024-09-12T03:34:20.965 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.007253279 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 878, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 27.033093068, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.030789399 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.966 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 103126, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4469.711525692, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.043342236 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.967 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.968 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.969 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.970 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 290, 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 29.371437729, 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.101280819 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-09-12T03:34:20.971 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 12713, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 177.120717284, 2024-09-12T03:34:20.972 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.013932251 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 982, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 2.939833167, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002993720 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 33090, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 270.053705384, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.008161187 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.973 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-09-12T03:34:20.974 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 228, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 66.937438132, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.293585254 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.975 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 8, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.902527618, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.112815952 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.976 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 7, 2024-09-12T03:34:20.977 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.019022897, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002717556 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 3, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.001734521, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000578173 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 19635, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 48.402806579, 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.002465128 2024-09-12T03:34:20.978 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-09-12T03:34:20.979 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 2, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 1, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 10792, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 10792, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 906, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:20.980 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:20.981 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:20.982 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:20.983 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 319105, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 116, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 154, 2024-09-12T03:34:20.984 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 171211, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 1190, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-09-12T03:34:20.985 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 158750, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 411247422, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 37, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 158750, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 158750, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 690.217338762, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.004347825 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 158750, 2024-09-12T03:34:20.986 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 162451 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 158750, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 117464, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 41286, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 26, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 772, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 634, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 14573, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 379, 2024-09-12T03:34:20.987 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 129, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 23701, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 116825, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 1544, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:20.988 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 7, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-09-12T03:34:20.989 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-09-12T03:34:20.990 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 82, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 3356, 2024-09-12T03:34:20.991 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 46, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 459 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 86397 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.992 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 14, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 6, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 23033, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 21853, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:20.993 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 634941, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 155646779, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 634941, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 634941, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 155646779, 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:20.994 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:20.995 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 158750, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 413455419, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 158750, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 413455419, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.996 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 158750, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 158750, 2024-09-12T03:34:20.997 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 158750, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 158750, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:20.998 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 21853, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 2208353, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 21853, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 617, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 2208353, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:20.999 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55df2df6c960": { 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 165834, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 383062257, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 165834, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 13955, 2024-09-12T03:34:21.000 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 383062257, 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:21.001 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-12T03:34:21.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:20 smithi121 ceph-mon[34963]: pgmap v796: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:21.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:20 smithi019 ceph-mon[30953]: pgmap v796: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:34:21.657 INFO:tasks.check_counter:Daemon mds.i mds.exported=487 2024-09-12T03:34:21.657 INFO:tasks.check_counter:Daemon mds.i mds.imported=500 2024-09-12T03:34:21.657 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=5031 2024-09-12T03:34:21.657 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=6647 2024-09-12T03:34:21.657 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=129 2024-09-12T03:34:21.657 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:34:21.658 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:34:21.909 INFO:teuthology.orchestra.run.smithi135.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout:{ 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 94903, 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 94578, 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 38137594, 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 92325265, 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 10, 2024-09-12T03:34:22.456 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 3, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 7.788804616, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 3.555203942, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 2.718994558, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.604220194, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 94580, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 4.570508693, 2024-09-12T03:34:22.457 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000048324 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 92200, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.016661878, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000180 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.458 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 88076, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 86399, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 26762579, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 90083735, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 11, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 7, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 6.158028920, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 2.264687426, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 1.929652605, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.757847018, 2024-09-12T03:34:22.459 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 86399, 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 5.261131937, 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000060893 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 43469, 2024-09-12T03:34:22.460 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.007789486, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000179 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 802224, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 271776 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_messages": 300071, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages": 375136, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_bytes": 97342741, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_bytes": 205383171, 2024-09-12T03:34:22.461 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_created_connections": 49, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_active_connections": 9, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_total_time": 15.521710181, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_send_time": 6.444158810, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_recv_time": 11.668094975, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_running_fast_dispatch_time": 1.584591907, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 375130, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 27.688795535, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000073811 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.462 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 475875, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.099041843, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000208 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "msgr_send_encrypted_bytes": 459008 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "cct": { 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "total_workers": 1, 2024-09-12T03:34:22.463 INFO:teuthology.orchestra.run.smithi135.stdout: "unhealthy_workers": 0 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-MDSRank": { 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 103277, 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 72.850935686, 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000705393 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "finisher-PurgeQueue": { 2024-09-12T03:34:22.464 INFO:teuthology.orchestra.run.smithi135.stdout: "queue_len": 0, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "complete_latency": { 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 29695, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 5.428954131, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000182823 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "mds": { 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "request": 104538, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "reply": 91555, 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "reply_latency": { 2024-09-12T03:34:22.465 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 91555, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1748.928553749, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.019102490 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_reply": 0, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "forward": 12983, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_complete": 0, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_fetch_keys": 0, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_commit": 497, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_split": 54, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_merge": 1, 2024-09-12T03:34:22.466 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes": 49, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_top": 25, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_bottom": 14, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pin_tail": 10, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_pinned": 13, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_expired": 7958, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "inodes_with_caps": 0, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "caps": 0, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "subtrees": 6, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse": 186642, 2024-09-12T03:34:22.467 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_hit": 144766, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_forward": 12952, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_discover": 1119, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_remote_ino": 0, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "traverse_lock": 551, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "load_cent": 0, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "q": 0, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "exported": 226, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "exported_inodes": 2803, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "imported": 150, 2024-09-12T03:34:22.468 INFO:teuthology.orchestra.run.smithi135.stdout: "imported_inodes": 15079, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_dir_fetch": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_backtrace_fetch": 141790, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "openino_peer_discover": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rfiles": 6674, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rbytes": 147760442, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "root_rsnaps": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_fetch": 141790, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_set_tag": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_inodes": 902, 2024-09-12T03:34:22.469 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_dirfrag_rstats": 902, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "scrub_file_inodes": 140888, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_inode_file_caps": 68, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_revoke": 49471, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_grant": 47405, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps": 61374, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_caps_dirty": 12005, 2024-09-12T03:34:22.470 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_cap_release": 1061, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "process_request_cap_release": 34592 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_cache": { 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update": 3998, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_update_receipt": 8151, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_try_discover": 4572, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_send_discover": 6065, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "dir_handle_discover": 5164, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays": 0, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_delayed": 0, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:34:22.471 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_created": 15787, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_enqueued": 15787, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_reintegrated": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "strays_migrated": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_processing": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_started": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "recovery_completed": 0, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_path": 12, 2024-09-12T03:34:22.472 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_quiesce_inode": 114123, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_exportdir": 226, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_flush": 0, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragmentdir": 57, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_fragstats": 0, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "ireq_inodestats": 0 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_log": { 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "evlrg": 1, 2024-09-12T03:34:22.473 INFO:teuthology.orchestra.run.smithi135.stdout: "evadd": 80319, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "evex": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "evtrm": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "ev": 80319, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "evexg": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "evexd": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segadd": 103, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segex": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segtrm": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "seg": 103, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segmjr": 8, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segexg": 0, 2024-09-12T03:34:22.474 INFO:teuthology.orchestra.run.smithi135.stdout: "segexd": 0, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "expos": 4194304, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "wrpos": 167528426, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "rdpos": 0, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "jlat": { 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4477, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 303.865530008, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.067872577 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "replayed": 0 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.475 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_mem": { 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "ino": 56, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "ino+": 18326, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "ino-": 18270, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dir": 55, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dir+": 4637, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dir-": 4582, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dn": 49, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dn+": 51576, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "dn-": 51527, 2024-09-12T03:34:22.476 INFO:teuthology.orchestra.run.smithi135.stdout: "cap": 0, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "cap+": 31772, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "cap-": 31772, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "rss": 142196, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "heap": 223520 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_server": { 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_client_request": 116706, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "dispatch_server_request": 3978, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_request": 104538, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_client_session": 337, 2024-09-12T03:34:22.477 INFO:teuthology.orchestra.run.smithi135.stdout: "handle_peer_request": 12733, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_create_latency": { 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9384, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 146.216621117, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.015581481 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getattr_latency": { 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 11, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.049988426, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.004544402 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.478 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getfilelock_latency": { 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "req_link_latency": { 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookup_latency": { 2024-09-12T03:34:22.479 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 53326, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 1137.966718717, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.021339810 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookuphash_latency": { 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupino_latency": { 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.480 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupname_latency": { 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupparent_latency": { 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.481 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lookupsnap_latency": { 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "req_lssnap_latency": { 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.482 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mkdir_latency": { 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 52, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 35.618931556, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.684979453 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mknod_latency": { 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.483 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "req_mksnap_latency": { 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "req_open_latency": { 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 4233, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 38.805408189, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.009167353 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.484 INFO:teuthology.orchestra.run.smithi135.stdout: "req_readdir_latency": { 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 627, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.900447942, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001436121 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rename_latency": { 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 9341, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 354.256943349, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.037924948 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "req_renamesnap_latency": { 2024-09-12T03:34:22.485 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "req_snapdiff_latency": { 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmdir_latency": { 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 84, 2024-09-12T03:34:22.486 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 16.399720735, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.195234770 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmsnap_latency": { 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "req_rmxattr_latency": { 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.487 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setattr_latency": { 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setdirlayout_latency": { 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.488 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setfilelock_latency": { 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setlayout_latency": { 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.489 INFO:teuthology.orchestra.run.smithi135.stdout: "req_setxattr_latency": { 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "req_symlink_latency": { 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.490 INFO:teuthology.orchestra.run.smithi135.stdout: "req_unlink_latency": { 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 14497, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 18.713773718, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.001290872 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "req_getvxattr_latency": { 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.491 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_sessions": { 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "session_count": 1, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "session_add": 1, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "session_remove": 0, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_open": 1, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "sessions_stale": 0, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "total_load": 17910, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "average_load": 17910, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "avg_session_uptime": 822, 2024-09-12T03:34:22.492 INFO:teuthology.orchestra.run.smithi135.stdout: "mdthresh_evicted": 0 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "mempool": { 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bloom_filter_items": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:34:22.493 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_extent_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_blob_items": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:34:22.494 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_txc_items": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:34:22.495 INFO:teuthology.orchestra.run.smithi135.stdout: "bluestore_writing_items": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_bytes": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_items": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_bytes": 279692, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_anon_items": 99, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_bytes": 88, 2024-09-12T03:34:22.496 INFO:teuthology.orchestra.run.smithi135.stdout: "buffer_meta_items": 1, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_bytes": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_items": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_mapbl_items": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_pglog_items": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_bytes": 7496, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_items": 154, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:34:22.497 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_bytes": 0, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "pgmap_items": 0, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_bytes": 234861, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "mds_co_items": 944, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_bytes": 0, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_1_items": 0, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_bytes": 0, 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: "unittest_2_items": 0 2024-09-12T03:34:22.498 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.499 INFO:teuthology.orchestra.run.smithi135.stdout: "objecter": { 2024-09-12T03:34:22.499 INFO:teuthology.orchestra.run.smithi135.stdout: "op_active": 0, 2024-09-12T03:34:22.499 INFO:teuthology.orchestra.run.smithi135.stdout: "op_laggy": 0, 2024-09-12T03:34:22.499 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send": 165021, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "op_send_bytes": 176060542, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "op_resend": 33, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "op_reply": 165021, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "op_latency": { 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 165021, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 539.952518600, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.003272023 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "op_inflight": 0, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "oplen_avg": { 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 165021, 2024-09-12T03:34:22.500 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 166184 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "op": 165021, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "op_r": 142371, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "op_w": 22650, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "op_rmw": 0, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "op_pg": 0, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_stat": 0, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_create": 10, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_read": 581, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_write": 5062, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writefull": 356, 2024-09-12T03:34:22.501 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_writesame": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_append": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_zero": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_truncate": 54, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_delete": 16277, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_mapext": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_sparse_read": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_clonerange": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_getxattr": 141790, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_setxattr": 20, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_rmxattr": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:34:22.502 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_call": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_watch": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_notify": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "osdop_other": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_active": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_send": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_resend": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "linger_ping": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_active": 0, 2024-09-12T03:34:22.503 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_send": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "poolop_resend": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_active": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_send": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "poolstat_resend": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_active": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_send": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "statfs_resend": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "command_active": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "command_send": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "command_resend": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "map_epoch": 82, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "map_full": 0, 2024-09-12T03:34:22.504 INFO:teuthology.orchestra.run.smithi135.stdout: "map_inc": 2, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_sessions": 12, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_open": 12, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_session_close": 0, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "osd_laggy": 0, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_wr": 1704, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_rd": 0, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_del": 330 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "oft": { 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_objs": 1, 2024-09-12T03:34:22.505 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_updates": 0, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "omap_total_removes": 0 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "osd-slow-ops": { 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "slow_ops_count": 113980 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "purge_queue": { 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops": 0, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_ops_high_water": 14, 2024-09-12T03:34:22.506 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing": 0, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executing_high_water": 6, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed_ops": 16219, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_executed": 15787, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "pq_item_in_journal": 0 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 483046, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 118745200, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 483046, 2024-09-12T03:34:22.507 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 483046, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 118745200, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.508 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_bytes": { 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 104857600, 2024-09-12T03:34:22.509 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 165021, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 177655461, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 165021, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 177655461, 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:22.510 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-objecter_ops": { 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 1024, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 0, 2024-09-12T03:34:22.511 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 165021, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 165021, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 165021, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 165021, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:22.512 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 15787, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 1594919, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 15787, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 585, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 1594919, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.513 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: }, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "throttle-write_buf_throttle-0x55c5af440960": { 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "val": 0, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "max": 3758096384, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "get_started": 0, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "get": 80319, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "get_sum": 163334122, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_fail": 0, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "get_or_fail_success": 80319, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "take": 0, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "take_sum": 0, 2024-09-12T03:34:22.514 INFO:teuthology.orchestra.run.smithi135.stdout: "put": 4477, 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: "put_sum": 163334122, 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: "wait": { 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgcount": 0, 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: "sum": 0.000000000, 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: "avgtime": 0.000000000 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout: } 2024-09-12T03:34:22.515 INFO:teuthology.orchestra.run.smithi135.stdout:} 2024-09-12T03:34:23.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:22 smithi135 ceph-mon[35474]: pgmap v797: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:23.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:22 smithi121 ceph-mon[34963]: pgmap v797: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:23.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:22 smithi019 ceph-mon[30953]: pgmap v797: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:23.258 INFO:tasks.check_counter:Daemon mds.l mds.exported=226 2024-09-12T03:34:23.258 INFO:tasks.check_counter:Daemon mds.l mds.imported=150 2024-09-12T03:34:23.259 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=3998 2024-09-12T03:34:23.259 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=8151 2024-09-12T03:34:23.259 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=54 2024-09-12T03:34:23.259 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:34:23.271 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:34:23.271 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:34:23.271 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:34:23.294 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-12T03:34:23.295 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:34:23.356 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:34:23.667 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 -- 172.21.15.19:0/3890137446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 msgr2=0x7f882c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3890137446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c1029b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8820009900 tx=0x7f882002a480 comp rx=0 tx=0).stop 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 -- 172.21.15.19:0/3890137446 shutdown_connections 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3890137446 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 0x7f882c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3890137446 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c1029b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.237 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3890137446 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 0x7f882c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 -- 172.21.15.19:0/3890137446 >> 172.21.15.19:0/3890137446 conn(0x7f882c0fc7d0 msgr2=0x7f882c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 -- 172.21.15.19:0/3890137446 shutdown_connections 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.207+0000 7f8830cb2640 1 -- 172.21.15.19:0/3890137446 wait complete. 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.208+0000 7f8830cb2640 1 Processor -- start 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.208+0000 7f8830cb2640 1 -- start start 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.208+0000 7f8830cb2640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 0x7f882c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.208+0000 7f8830cb2640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 0x7f882c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f882c110fa0 con 0x7f882c102570 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f882c110e20 con 0x7f882c100e20 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f882c111120 con 0x7f882c102ef0 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56118/0 (socket says 172.21.15.19:56118) 2024-09-12T03:34:24.238 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 -- 172.21.15.19:0/3444855512 learned_addr learned my addr 172.21.15.19:0/3444855512 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f882ad9d640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 0x7f882c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f882a59c640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 0x7f882c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 -- 172.21.15.19:0/3444855512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 msgr2=0x7f882c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 0x7f882c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 -- 172.21.15.19:0/3444855512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 msgr2=0x7f882c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 0x7f882c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 -- 172.21.15.19:0/3444855512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c19d510 con 0x7f882c102570 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8829d9b640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f882002aa70 tx=0x7f8820003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882003f070 con 0x7f882c102570 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c19d7a0 con 0x7f882c102570 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8820031c20 con 0x7f882c102570 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820035030 con 0x7f882c102570 2024-09-12T03:34:24.239 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.209+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1a50e0 con 0x7f882c102570 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.210+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c101200 con 0x7f882c102570 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.212+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8820042070 con 0x7f882c102570 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.212+0000 7f881b7fe640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 0x7f88140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.212+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f88200efe50 con 0x7f882c102570 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.212+0000 7f882a59c640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 0x7f88140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.213+0000 7f882a59c640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 0x7f88140a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f881c005fd0 tx=0x7f881c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:24.240 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.216+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88200bcdc0 con 0x7f882c102570 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.404+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f882c1998c0 con 0x7f882c102570 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.404+0000 7f881b7fe640 1 -- 172.21.15.19:0/3444855512 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v82) ==== 81+0+590 (secure 0 0 0) 0x7f882c1998c0 con 0x7f882c102570 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3786063501 2024-09-13T03:10:11.180029+0000 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/262011260 2024-09-13T03:10:11.180029+0000 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/1406182514 2024-09-13T03:10:11.180029+0000 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/1268895810 2024-09-13T03:10:11.180029+0000 2024-09-12T03:34:24.405 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/1268895810 2024-09-13T03:10:11.180029+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/153701601 2024-09-13T03:09:47.038261+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6801/2981478510 2024-09-13T03:09:47.038261+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:6800/2981478510 2024-09-13T03:09:47.038261+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3143338534 2024-09-13T03:09:47.038261+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stdout:172.21.15.19:0/3087383304 2024-09-13T03:09:47.038261+0000 2024-09-12T03:34:24.406 INFO:teuthology.orchestra.run.smithi019.stderr:listed 10 entries 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 msgr2=0x7f88140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 0x7f88140a5040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f881c005fd0 tx=0x7f881c005850 comp rx=0 tx=0).stop 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 msgr2=0x7f882c198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f882002aa70 tx=0x7f8820003b60 comp rx=0 tx=0).stop 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 shutdown_connections 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f882c102ef0 0x7f882c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f88140a2b80 0x7f88140a5040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f882c102570 0x7f882c198aa0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.407 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 --2- 172.21.15.19:0/3444855512 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f882c100e20 0x7f882c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:24.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 >> 172.21.15.19:0/3444855512 conn(0x7f882c0fc7d0 msgr2=0x7f882c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:24.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 shutdown_connections 2024-09-12T03:34:24.408 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:24.407+0000 7f8830cb2640 1 -- 172.21.15.19:0/3444855512 wait complete. 2024-09-12T03:34:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:24 smithi135 ceph-mon[35474]: pgmap v798: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:25.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:24 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3444855512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:34:25.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:24 smithi121 ceph-mon[34963]: pgmap v798: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:25.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:24 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3444855512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:34:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:24 smithi019 ceph-mon[30953]: pgmap v798: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-09-12T03:34:25.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:24 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3444855512' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:34:25.226 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:34:25.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:34:25.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:34:25.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T03:34:25.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:34:25.436 DEBUG:teuthology.orchestra.run.smithi019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:34:25.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:34:25.494 DEBUG:teuthology.orchestra.run.smithi019:> ip netns list 2024-09-12T03:34:25.540 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:34:25.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:34:25.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:34:25.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:34:25.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo ip link delete ceph-brx 2024-09-12T03:34:25.716 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:34:25.727 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:34:25.728 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:34:25.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:34:25.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:25 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:26.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:25 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:26.069 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:26.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:25 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- 172.21.15.19:0/3597395387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0100a80 msgr2=0x7f10c0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 --2- 172.21.15.19:0/3597395387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0100a80 0x7f10c0100ee0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009930 tx=0x7f10b402a420 comp rx=0 tx=0).stop 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- 172.21.15.19:0/3597395387 shutdown_connections 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 --2- 172.21.15.19:0/3597395387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 0x7f10c010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 --2- 172.21.15.19:0/3597395387 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0100a80 0x7f10c0100ee0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 --2- 172.21.15.19:0/3597395387 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0106a80 0x7f10c0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- 172.21.15.19:0/3597395387 >> 172.21.15.19:0/3597395387 conn(0x7f10c00fc7a0 msgr2=0x7f10c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- 172.21.15.19:0/3597395387 shutdown_connections 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- 172.21.15.19:0/3597395387 wait complete. 2024-09-12T03:34:26.683 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 Processor -- start 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.651+0000 7f10c6e33640 1 -- start start 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 0x7f10c00fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 0x7f10c00fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0110ea0 con 0x7f10c0106a80 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0110d20 con 0x7f10c0101420 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c6e33640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f10c0111020 con 0x7f10c0100a80 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56140/0 (socket says 172.21.15.19:56140) 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 -- 172.21.15.19:0/837083504 learned_addr learned my addr 172.21.15.19:0/837083504 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10bffff640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 0x7f10c00fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:26.684 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c4bcf640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 0x7f10c00fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 -- 172.21.15.19:0/837083504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 msgr2=0x7f10c00fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 0x7f10c00fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 -- 172.21.15.19:0/837083504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 msgr2=0x7f10c00fd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 0x7f10c00fd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c53d0640 1 -- 172.21.15.19:0/837083504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c01a6990 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.652+0000 7f10c4bcf640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 0x7f10c00fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10c53d0640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009fd0 tx=0x7f10b800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b8017070 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10b8003e50 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c01a6c80 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b80106c0 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.653+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c01a7190 con 0x7f10c0106a80 2024-09-12T03:34:26.685 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.654+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c0100ee0 con 0x7f10c0106a80 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.655+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f10b8003970 con 0x7f10c0106a80 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.656+0000 7f10bdffb640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 0x7f10a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.656+0000 7f10c4bcf640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 0x7f10a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.656+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f10b80ceda0 con 0x7f10c0106a80 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.656+0000 7f10c4bcf640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 0x7f10a80a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009930 tx=0x7f10b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:26.686 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.660+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10b809bcb0 con 0x7f10c0106a80 2024-09-12T03:34:26.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:26 smithi019 ceph-mon[30953]: pgmap v799: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:26.733 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:26 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:26.872 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.872+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10c010a020 con 0x7f10c0106a80 2024-09-12T03:34:26.873 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.873+0000 7f10bdffb640 1 -- 172.21.15.19:0/837083504 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+12003 (secure 0 0 0) 0x7f10b809b5b0 con 0x7f10c0106a80 2024-09-12T03:34:26.873 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:34:26.874 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":66,"btime":"2024-09-12T03:34:25:708515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:34:25.523672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:34:26.874 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 66 2024-09-12T03:34:26.875 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 msgr2=0x7f10a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 0x7f10a80a50c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009930 tx=0x7f10b0008040 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 msgr2=0x7f10c00fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f10b8009fd0 tx=0x7f10b800b260 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 shutdown_connections 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f10a80a2c00 0x7f10a80a50c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f10c0106a80 0x7f10c00fd850 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c0101420 0x7f10c00fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 --2- 172.21.15.19:0/837083504 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f10c0100a80 0x7f10c00fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 >> 172.21.15.19:0/837083504 conn(0x7f10c00fc7a0 msgr2=0x7f10c006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 shutdown_connections 2024-09-12T03:34:26.876 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:26.875+0000 7f10c6e33640 1 -- 172.21.15.19:0/837083504 wait complete. 2024-09-12T03:34:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:26 smithi135 ceph-mon[35474]: pgmap v799: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:27.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:26 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:27.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:26 smithi121 ceph-mon[34963]: pgmap v799: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s 2024-09-12T03:34:27.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:26 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:27.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:34:27.895 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:27.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:27 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/837083504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:28.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:27 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/837083504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:28.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:27 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/837083504' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968766618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc100a80 msgr2=0x7f12dc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968766618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc100a80 0x7f12dc100ee0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009930 tx=0x7f12cc02a420 comp rx=0 tx=0).stop 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968766618 shutdown_connections 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968766618 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc101420 0x7f12dc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968766618 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc100a80 0x7f12dc100ee0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968766618 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc106a80 0x7f12dc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968766618 >> 172.21.15.19:0/2968766618 conn(0x7f12dc0fc7a0 msgr2=0x7f12dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968766618 shutdown_connections 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968766618 wait complete. 2024-09-12T03:34:28.377 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 Processor -- start 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.350+0000 7f12e2f93640 1 -- start start 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 0x7f12dc1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc101420 0x7f12dc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc110ed0 con 0x7f12dc106a80 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc110d50 con 0x7f12dc100a80 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e2f93640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f12dc111050 con 0x7f12dc101420 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56170/0 (socket says 172.21.15.19:56170) 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 -- 172.21.15.19:0/2968074085 learned_addr learned my addr 172.21.15.19:0/2968074085 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12dbfff640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc101420 0x7f12dc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e0d2f640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 0x7f12dc1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:28.378 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 -- 172.21.15.19:0/2968074085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc101420 msgr2=0x7f12dc197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc101420 0x7f12dc197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 -- 172.21.15.19:0/2968074085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 msgr2=0x7f12dc1972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 0x7f12dc1972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 -- 172.21.15.19:0/2968074085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12dc19c2a0 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e0d2f640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 0x7f12dc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.351+0000 7f12e1530640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f12d400cce0 tx=0x7f12d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.352+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d400f450 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.352+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12dc19c590 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.352+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f12d400fad0 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.352+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d4018c10 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.352+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12dc1a3e70 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.353+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12dc100ee0 con 0x7f12dc106a80 2024-09-12T03:34:28.379 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.354+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12d400f5f0 con 0x7f12dc106a80 2024-09-12T03:34:28.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.355+0000 7f12d9ffb640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 0x7f12c80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:28.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.355+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f12d40cf800 con 0x7f12dc106a80 2024-09-12T03:34:28.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.355+0000 7f12e0d2f640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 0x7f12c80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:28.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.355+0000 7f12e0d2f640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 0x7f12c80a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009ad0 tx=0x7f12c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:28.380 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.359+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f12d4014030 con 0x7f12dc106a80 2024-09-12T03:34:28.570 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.570+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12dc1a40d0 con 0x7f12dc106a80 2024-09-12T03:34:28.571 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.571+0000 7f12d9ffb640 1 -- 172.21.15.19:0/2968074085 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+12003 (secure 0 0 0) 0x7f12d409c710 con 0x7f12dc106a80 2024-09-12T03:34:28.571 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:34:28.571 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":66,"btime":"2024-09-12T03:34:25:708515+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14511,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6835/1760992520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6834","nonce":1760992520},{"type":"v1","addr":"172.21.15.19:6835","nonce":1760992520}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14523,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.19:6837/2911685711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6836","nonce":2911685711},{"type":"v1","addr":"172.21.15.19:6837","nonce":2911685711}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":66,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T03:17:10.527386+0000","modified":"2024-09-12T03:34:25.523672+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24467,"mds_2":24463,"mds_3":24421,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14577":{"gid":14577,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6839/2689902455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6838","nonce":2689902455},{"type":"v1","addr":"172.21.15.19:6839","nonce":2689902455}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14589":{"gid":14589,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.19:6841/9500015","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6840","nonce":9500015},{"type":"v1","addr":"172.21.15.19:6841","nonce":9500015}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":4,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.121:6833/3300883065","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":3300883065},{"type":"v1","addr":"172.21.15.121:6833","nonce":3300883065}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"l","rank":3,"incarnation":23,"state":"up:active","state_seq":15,"addr":"172.21.15.135:6833/990973415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6832","nonce":990973415},{"type":"v1","addr":"172.21.15.135:6833","nonce":990973415}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6835/3114415604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":3114415604},{"type":"v1","addr":"172.21.15.121:6835","nonce":3114415604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.121:6837/826498871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":826498871},{"type":"v1","addr":"172.21.15.121:6837","nonce":826498871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24457":{"gid":24457,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.135:6835/3494935988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6834","nonce":3494935988},{"type":"v1","addr":"172.21.15.135:6835","nonce":3494935988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24463":{"gid":24463,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":10,"addr":"172.21.15.135:6837/1827195054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6836","nonce":1827195054},{"type":"v1","addr":"172.21.15.135:6837","nonce":1827195054}]},"join_fscid":-1,"export_targets":[0,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24467":{"gid":24467,"name":"b","rank":1,"incarnation":19,"state":"up:active","state_seq":6,"addr":"172.21.15.121:6839/1305484715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6838","nonce":1305484715},{"type":"v1","addr":"172.21.15.121:6839","nonce":1305484715}]},"join_fscid":-1,"export_targets":[0,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.135:6839/1449185077","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6838","nonce":1449185077},{"type":"v1","addr":"172.21.15.135:6839","nonce":1449185077}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24407,24421,24463,24467,24487]},"id":1}]} 2024-09-12T03:34:28.572 INFO:teuthology.orchestra.run.smithi019.stderr:dumped fsmap epoch 66 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.573+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 msgr2=0x7f12c80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.573+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 0x7f12c80a5040 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009ad0 tx=0x7f12c4005f90 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.573+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 msgr2=0x7f12dc19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.573+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f12d400cce0 tx=0x7f12d400a620 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 shutdown_connections 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f12c80a2b80 0x7f12c80a5040 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f12dc106a80 0x7f12dc19bbc0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f12dc101420 0x7f12dc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 --2- 172.21.15.19:0/2968074085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f12dc100a80 0x7f12dc1972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:28.574 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 >> 172.21.15.19:0/2968074085 conn(0x7f12dc0fc7a0 msgr2=0x7f12dc0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 shutdown_connections 2024-09-12T03:34:28.575 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:28.574+0000 7f12e2f93640 1 -- 172.21.15.19:0/2968074085 wait complete. 2024-09-12T03:34:28.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:28 smithi019 ceph-mon[30953]: pgmap v800: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T03:34:28.914 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:28 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2968074085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:28 smithi135 ceph-mon[35474]: pgmap v800: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T03:34:29.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:28 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2968074085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:29.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:28 smithi121 ceph-mon[34963]: pgmap v800: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-09-12T03:34:29.051 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:28 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2968074085' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:34:29.368 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:34:29.633 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:30.136 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 -- 172.21.15.19:0/166421889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7570102d40 msgr2=0x7f7570103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 --2- 172.21.15.19:0/166421889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7570102d40 0x7f7570103120 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7560009900 tx=0x7f756002a480 comp rx=0 tx=0).stop 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 -- 172.21.15.19:0/166421889 shutdown_connections 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 --2- 172.21.15.19:0/166421889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f757010c170 0x7f757010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 --2- 172.21.15.19:0/166421889 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570103660 0x7f757010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 --2- 172.21.15.19:0/166421889 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f7570102d40 0x7f7570103120 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 -- 172.21.15.19:0/166421889 >> 172.21.15.19:0/166421889 conn(0x7f75700fc7a0 msgr2=0x7f75700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 -- 172.21.15.19:0/166421889 shutdown_connections 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.110+0000 7f7575c0c640 1 -- 172.21.15.19:0/166421889 wait complete. 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 Processor -- start 2024-09-12T03:34:30.137 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 -- start start 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570102d40 0x7f75701983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 0x7f7570198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111fc0 con 0x7f757010c170 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111e40 con 0x7f7570103660 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f7575c0c640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112140 con 0x7f7570102d40 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:56190/0 (socket says 172.21.15.19:56190) 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 -- 172.21.15.19:0/2408339756 learned_addr learned my addr 172.21.15.19:0/2408339756 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756effd640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 0x7f7570198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:30.138 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756f7fe640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570102d40 0x7f75701983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 -- 172.21.15.19:0/2408339756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570102d40 msgr2=0x7f75701983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570102d40 0x7f75701983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 -- 172.21.15.19:0/2408339756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 msgr2=0x7f7570198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.111+0000 7f756ffff640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 0x7f7570198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756ffff640 1 -- 172.21.15.19:0/2408339756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019d370 con 0x7f757010c170 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756effd640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 0x7f7570198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756ffff640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f756800cce0 tx=0x7f756800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800f450 con 0x7f757010c170 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756800fad0 con 0x7f757010c170 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019d660 con 0x7f757010c170 2024-09-12T03:34:30.139 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568018c10 con 0x7f757010c170 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.112+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a4f40 con 0x7f757010c170 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.113+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7570103120 con 0x7f757010c170 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.114+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f756800f5f0 con 0x7f757010c170 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.115+0000 7f756cff9640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 0x7f75580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.115+0000 7f756f7fe640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 0x7f75580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.115+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7f75680cf740 con 0x7f757010c170 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.115+0000 7f756f7fe640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 0x7f75580a50c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7560000c00 tx=0x7f7560005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:30.140 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.118+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7568014030 con 0x7f757010c170 2024-09-12T03:34:30.294 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.294+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7570199720 con 0x7f757010c170 2024-09-12T03:34:30.296 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.296+0000 7f756cff9640 1 -- 172.21.15.19:0/2408339756 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22735 (secure 0 0 0) 0x7f7570199720 con 0x7f757010c170 2024-09-12T03:34:30.296 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-12T03:34:30.297 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":82,"fsid":"45fb56a2-70b4-11ef-bcea-c7b262605968","created":"2024-09-12T03:09:21.564122+0000","modified":"2024-09-12T03:18:13.217217+0000","last_up_change":"2024-09-12T03:16:15.566074+0000","last_in_change":"2024-09-12T03:15:52.071159+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T03:12:25.696963+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T03:17:06.299019+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T03:17:08.893970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c60c7b8-10fe-4d61-8f59-09d8bb612f89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6803","nonce":3926650955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6805","nonce":3926650955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6809","nonce":3926650955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3926650955},{"type":"v1","addr":"172.21.15.19:6807","nonce":3926650955}]},"public_addr":"172.21.15.19:6803/3926650955","cluster_addr":"172.21.15.19:6805/3926650955","heartbeat_back_addr":"172.21.15.19:6809/3926650955","heartbeat_front_addr":"172.21.15.19:6807/3926650955","state":["exists","up"]},{"osd":1,"uuid":"963d3ee4-bad4-4896-8b30-38c5d33c0333","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6801","nonce":4159912273}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6803","nonce":4159912273}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6807","nonce":4159912273}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":4159912273},{"type":"v1","addr":"172.21.15.121:6805","nonce":4159912273}]},"public_addr":"172.21.15.121:6801/4159912273","cluster_addr":"172.21.15.121:6803/4159912273","heartbeat_back_addr":"172.21.15.121:6807/4159912273","heartbeat_front_addr":"172.21.15.121:6805/4159912273","state":["exists","up"]},{"osd":2,"uuid":"aca96f86-8aca-451b-8be2-0d18ed46afae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6800","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6801","nonce":2430507446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6802","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6803","nonce":2430507446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6806","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6807","nonce":2430507446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6804","nonce":2430507446},{"type":"v1","addr":"172.21.15.135:6805","nonce":2430507446}]},"public_addr":"172.21.15.135:6801/2430507446","cluster_addr":"172.21.15.135:6803/2430507446","heartbeat_back_addr":"172.21.15.135:6807/2430507446","heartbeat_front_addr":"172.21.15.135:6805/2430507446","state":["exists","up"]},{"osd":3,"uuid":"4eccbc69-1f18-4b12-be71-e115af1f187f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6811","nonce":3191237589}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6813","nonce":3191237589}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6817","nonce":3191237589}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":3191237589},{"type":"v1","addr":"172.21.15.19:6815","nonce":3191237589}]},"public_addr":"172.21.15.19:6811/3191237589","cluster_addr":"172.21.15.19:6813/3191237589","heartbeat_back_addr":"172.21.15.19:6817/3191237589","heartbeat_front_addr":"172.21.15.19:6815/3191237589","state":["exists","up"]},{"osd":4,"uuid":"bf3b091d-b06c-466a-b160-49acb3bb2917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6809","nonce":1363833707}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6811","nonce":1363833707}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6815","nonce":1363833707}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1363833707},{"type":"v1","addr":"172.21.15.121:6813","nonce":1363833707}]},"public_addr":"172.21.15.121:6809/1363833707","cluster_addr":"172.21.15.121:6811/1363833707","heartbeat_back_addr":"172.21.15.121:6815/1363833707","heartbeat_front_addr":"172.21.15.121:6813/1363833707","state":["exists","up"]},{"osd":5,"uuid":"3ac0c634-1484-497b-bee3-58e37c79b212","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6808","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6809","nonce":3468418134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6810","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6811","nonce":3468418134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6814","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6815","nonce":3468418134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6812","nonce":3468418134},{"type":"v1","addr":"172.21.15.135:6813","nonce":3468418134}]},"public_addr":"172.21.15.135:6809/3468418134","cluster_addr":"172.21.15.135:6811/3468418134","heartbeat_back_addr":"172.21.15.135:6815/3468418134","heartbeat_front_addr":"172.21.15.135:6813/3468418134","state":["exists","up"]},{"osd":6,"uuid":"10489254-1ec9-4c5d-acf1-abc03c7e1167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6819","nonce":2104639727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6821","nonce":2104639727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6825","nonce":2104639727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":2104639727},{"type":"v1","addr":"172.21.15.19:6823","nonce":2104639727}]},"public_addr":"172.21.15.19:6819/2104639727","cluster_addr":"172.21.15.19:6821/2104639727","heartbeat_back_addr":"172.21.15.19:6825/2104639727","heartbeat_front_addr":"172.21.15.19:6823/2104639727","state":["exists","up"]},{"osd":7,"uuid":"17cbf654-fee4-4cfd-a5f3-32a8e312fabc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6817","nonce":1039653795}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6819","nonce":1039653795}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6823","nonce":1039653795}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":1039653795},{"type":"v1","addr":"172.21.15.121:6821","nonce":1039653795}]},"public_addr":"172.21.15.121:6817/1039653795","cluster_addr":"172.21.15.121:6819/1039653795","heartbeat_back_addr":"172.21.15.121:6823/1039653795","heartbeat_front_addr":"172.21.15.121:6821/1039653795","state":["exists","up"]},{"osd":8,"uuid":"d7263722-4663-4725-8f31-833f02d73b8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6816","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6817","nonce":693996615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6818","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6819","nonce":693996615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6822","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6823","nonce":693996615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6820","nonce":693996615},{"type":"v1","addr":"172.21.15.135:6821","nonce":693996615}]},"public_addr":"172.21.15.135:6817/693996615","cluster_addr":"172.21.15.135:6819/693996615","heartbeat_back_addr":"172.21.15.135:6823/693996615","heartbeat_front_addr":"172.21.15.135:6821/693996615","state":["exists","up"]},{"osd":9,"uuid":"93a2fba3-453f-42ab-ac18-23faec974ede","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6827","nonce":892182666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6829","nonce":892182666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6832","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6833","nonce":892182666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":892182666},{"type":"v1","addr":"172.21.15.19:6831","nonce":892182666}]},"public_addr":"172.21.15.19:6827/892182666","cluster_addr":"172.21.15.19:6829/892182666","heartbeat_back_addr":"172.21.15.19:6833/892182666","heartbeat_front_addr":"172.21.15.19:6831/892182666","state":["exists","up"]},{"osd":10,"uuid":"17c4e2a4-2369-4f99-b966-e9deb821b99d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6825","nonce":1462491703}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6826","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6827","nonce":1462491703}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6831","nonce":1462491703}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1462491703},{"type":"v1","addr":"172.21.15.121:6829","nonce":1462491703}]},"public_addr":"172.21.15.121:6825/1462491703","cluster_addr":"172.21.15.121:6827/1462491703","heartbeat_back_addr":"172.21.15.121:6831/1462491703","heartbeat_front_addr":"172.21.15.121:6829/1462491703","state":["exists","up"]},{"osd":11,"uuid":"a58b445d-ac43-4862-9926-1bcd0a63537d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":74,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6824","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6825","nonce":3648701186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6826","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6827","nonce":3648701186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6830","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6831","nonce":3648701186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.135:6828","nonce":3648701186},{"type":"v1","addr":"172.21.15.135:6829","nonce":3648701186}]},"public_addr":"172.21.15.135:6825/3648701186","cluster_addr":"172.21.15.135:6827/3648701186","heartbeat_back_addr":"172.21.15.135:6831/3648701186","heartbeat_front_addr":"172.21.15.135:6829/3648701186","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:33.319795+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:11:59.195436+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:23.217846+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:12:48.589075+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:14.592686+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:13:40.320645+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:05.210056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:30.816539+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:14:56.559163+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:22.214911+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:15:48.147623+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T03:16:13.749150+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.19:0/3786063501":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/262011260":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/1406182514":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6801/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:6800/1268895810":"2024-09-13T03:10:11.180029+0000","172.21.15.19:0/153701601":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6801/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:6800/2981478510":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3143338534":"2024-09-13T03:09:47.038261+0000","172.21.15.19:0/3087383304":"2024-09-13T03:09:47.038261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:34:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 msgr2=0x7f75580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 0x7f75580a50c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f7560000c00 tx=0x7f7560005a20 comp rx=0 tx=0).stop 2024-09-12T03:34:30.298 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 msgr2=0x7f757019cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f756800cce0 tx=0x7f756800a620 comp rx=0 tx=0).stop 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 shutdown_connections 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f75580a2c00 0x7f75580a50c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f757010c170 0x7f757019cc90 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7570103660 0x7f7570198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 --2- 172.21.15.19:0/2408339756 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f7570102d40 0x7f75701983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 >> 172.21.15.19:0/2408339756 conn(0x7f75700fc7a0 msgr2=0x7f75700fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 shutdown_connections 2024-09-12T03:34:30.299 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:30.298+0000 7f7575c0c640 1 -- 172.21.15.19:0/2408339756 wait complete. 2024-09-12T03:34:30.899 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:30 smithi121 ceph-mon[34963]: pgmap v801: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:34:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:30 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:30.900 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:30 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2408339756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:30 smithi135 ceph-mon[35474]: pgmap v801: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:34:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:30 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:31.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:30 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2408339756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:30 smithi019 ceph-mon[30953]: pgmap v801: 97 pgs: 97 active+clean; 969 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:34:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:30 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:31.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:30 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2408339756' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:34:31.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:34:31.620 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:32.013 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:31 smithi019 ceph-mon[30953]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:32.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:31 smithi135 ceph-mon[35474]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2102625167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48100a80 msgr2=0x7efe48100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2102625167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48100a80 0x7efe48100ee0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7efe38009930 tx=0x7efe3802a420 comp rx=0 tx=0).stop 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2102625167 shutdown_connections 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2102625167 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48101420 0x7efe4810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2102625167 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48100a80 0x7efe48100ee0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2102625167 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48106a80 0x7efe48106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2102625167 >> 172.21.15.19:0/2102625167 conn(0x7efe480fc7a0 msgr2=0x7efe480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2102625167 shutdown_connections 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.096+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2102625167 wait complete. 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 Processor -- start 2024-09-12T03:34:32.122 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 -- start start 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48100a80 0x7efe481992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48101420 0x7efe48199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7efe48110e80 con 0x7efe48106a80 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe48110d00 con 0x7efe48101420 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4f5e5640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7efe48111000 con 0x7efe48100a80 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4db82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4db82640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37832/0 (socket says 172.21.15.19:37832) 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4d381640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48100a80 0x7efe481992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4db82640 1 -- 172.21.15.19:0/2599244038 learned_addr learned my addr 172.21.15.19:0/2599244038 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.097+0000 7efe4cb80640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48101420 0x7efe48199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:32.123 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 -- 172.21.15.19:0/2599244038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48100a80 msgr2=0x7efe481992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48100a80 0x7efe481992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 -- 172.21.15.19:0/2599244038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48101420 msgr2=0x7efe48199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48101420 0x7efe48199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 -- 172.21.15.19:0/2599244038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe481a4890 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4db82640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7efe4000eaa0 tx=0x7efe4000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4000cdb0 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe40005020 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe481a4a60 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe400116b0 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.098+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481a4f20 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.099+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe481021c0 con 0x7efe48106a80 2024-09-12T03:34:32.124 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.100+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe40005560 con 0x7efe48106a80 2024-09-12T03:34:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.101+0000 7efe3e7fc640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 0x7efe340a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.101+0000 7efe4d381640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 0x7efe340a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.101+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7efe400cf210 con 0x7efe48106a80 2024-09-12T03:34:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.102+0000 7efe4d381640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 0x7efe340a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efe4400bb60 tx=0x7efe4400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:32.125 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.105+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe40016030 con 0x7efe48106a80 2024-09-12T03:34:32.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:31 smithi121 ceph-mon[34963]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:34:32.318 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.317+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efe48109010 con 0x7efe48106a80 2024-09-12T03:34:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.320+0000 7efe3e7fc640 1 -- 172.21.15.19:0/2599244038 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) ==== 161+0+0 (secure 0 0 0) 0x7efe4009bad0 con 0x7efe48106a80 2024-09-12T03:34:32.320 INFO:teuthology.orchestra.run.smithi019.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:34:32.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.323+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 msgr2=0x7efe340a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:32.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.323+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 0x7efe340a5040 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efe4400bb60 tx=0x7efe4400b390 comp rx=0 tx=0).stop 2024-09-12T03:34:32.324 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.323+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 msgr2=0x7efe481a4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.323+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7efe4000eaa0 tx=0x7efe4000ef60 comp rx=0 tx=0).stop 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 shutdown_connections 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7efe340a2b80 0x7efe340a5040 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7efe48106a80 0x7efe481a4350 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe48101420 0x7efe48199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 --2- 172.21.15.19:0/2599244038 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7efe48100a80 0x7efe481992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:32.325 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 >> 172.21.15.19:0/2599244038 conn(0x7efe480fc7a0 msgr2=0x7efe480fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 shutdown_connections 2024-09-12T03:34:32.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:32.324+0000 7efe4f5e5640 1 -- 172.21.15.19:0/2599244038 wait complete. 2024-09-12T03:34:32.544 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.544 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.544 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mds[66016]: -- 172.21.15.135:0/890332020 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5574fb965860 con 0x5574fb943400 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: starting mds.i at 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f[65399]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f[65399]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f[65399]: starting mds.f at 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.545 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mds[65702]: -- 172.21.15.135:0/576534746 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563d398db860 con 0x563d398b9000 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: starting mds.c at 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mds[65100]: -- 172.21.15.135:0/1915207935 <== mon.2 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55f0dd13c680 con 0x55f0dd111400 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mds[65100]: -- 172.21.15.135:0/1915207935 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0dd133860 con 0x55f0dd111400 2024-09-12T03:34:32.546 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: starting mds.l at 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mds[71596]: -- 172.21.15.19:0/3384762165 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560666145860 con 0x560666123400 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: starting mds.a at 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.665 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mds[71186]: -- 172.21.15.19:0/4076212852 <== mon.1 v2:172.21.15.135:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561235a69860 con 0x561235a46800 2024-09-12T03:34:32.666 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: starting mds.g at 2024-09-12T03:34:32.666 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a[30949]: 2024-09-12T03:34:32.318+0000 7f5166834640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:34:32.762 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.762 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.762 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mds[65149]: -- 172.21.15.121:0/1638520574 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f84bb63860 con 0x55f84bb40800 2024-09-12T03:34:32.762 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: starting mds.h at 2024-09-12T03:34:32.762 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mds[64851]: -- 172.21.15.121:0/2929794681 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1ae079860 con 0x55d1ae056800 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: starting mds.e at 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mds[64535]: -- 172.21.15.121:0/34288273 <== mon.0 v2:172.21.15.19:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c514c77860 con 0x55c514c54800 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: starting mds.k at 2024-09-12T03:34:32.763 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: ignoring --setuser ceph since I am not root 2024-09-12T03:34:32.764 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: ignoring --setgroup ceph since I am not root 2024-09-12T03:34:32.764 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mds[65464]: -- 172.21.15.121:0/1607749955 <== mon.2 v2:172.21.15.121:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b01a8cbc20 con 0x55b01a8a9000 2024-09-12T03:34:32.764 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: starting mds.b at 2024-09-12T03:34:33.019 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: pgmap v802: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:34:33.020 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:32 smithi019 ceph-mon[30953]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: pgmap v802: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:34:33.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:34:33.045 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:32 smithi135 ceph-mon[35474]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:34:33.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:34:33.149 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: pgmap v802: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2599244038' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T03:34:33.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:32 smithi121 ceph-mon[34963]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:34:33.311 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:33.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 -- 172.21.15.19:0/2841161574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 msgr2=0x7fa7c01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/2841161574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c01067e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009900 tx=0x7fa7b402a3f0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 -- 172.21.15.19:0/2841161574 shutdown_connections 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/2841161574 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0106f10 0x7fa7c010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/2841161574 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c01067e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/2841161574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0101de0 0x7fa7c01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 -- 172.21.15.19:0/2841161574 >> 172.21.15.19:0/2841161574 conn(0x7fa7c00fd8c0 msgr2=0x7fa7c00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 -- 172.21.15.19:0/2841161574 shutdown_connections 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.800+0000 7fa7c64c8640 1 -- 172.21.15.19:0/2841161574 wait complete. 2024-09-12T03:34:33.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.801+0000 7fa7c64c8640 1 Processor -- start 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.801+0000 7fa7c64c8640 1 -- start start 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0101de0 0x7fa7c0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 0x7fa7c019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0111f30 con 0x7fa7c0102790 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0111db0 con 0x7fa7c0106f10 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c64c8640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c01120b0 con 0x7fa7c0101de0 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37894/0 (socket says 172.21.15.19:37894) 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 -- 172.21.15.19:0/421632448 learned_addr learned my addr 172.21.15.19:0/421632448 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bffff640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0101de0 0x7fa7c0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.817 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c4a65640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 0x7fa7c019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 -- 172.21.15.19:0/421632448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0101de0 msgr2=0x7fa7c0198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0101de0 0x7fa7c0198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 -- 172.21.15.19:0/421632448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 msgr2=0x7fa7c019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 0x7fa7c019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 -- 172.21.15.19:0/421632448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c019d3c0 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7c4a65640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 0x7fa7c019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bf7fe640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005ab0 tx=0x7fa7b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.802+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b403f030 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.803+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c019d650 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.803+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b4003ce0 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.803+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4035070 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.803+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01a4f90 con 0x7fa7c0102790 2024-09-12T03:34:33.818 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.804+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c01021c0 con 0x7fa7c0102790 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.805+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7b4042030 con 0x7fa7c0102790 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.805+0000 7fa7bd7fa640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.805+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8641+0+0 (secure 0 0 0) 0x7fa7b40efb80 con 0x7fa7c0102790 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.805+0000 7fa7bffff640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.806+0000 7fa7bffff640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 0x7fa7a80a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005e10 tx=0x7fa7b0005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:33.819 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:33.809+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa7b40bc940 con 0x7fa7c0102790 2024-09-12T03:34:34.022 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.021+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7c010a0f0 con 0x7fa7c0102790 2024-09-12T03:34:34.326 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.326+0000 7fa7bd7fa640 1 -- 172.21.15.19:0/421632448 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) ==== 104+0+0 (secure 0 0 0) 0x7fa7b40bc240 con 0x7fa7c0102790 2024-09-12T03:34:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 msgr2=0x7fa7a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 0x7fa7a80a5040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0005e10 tx=0x7fa7b0005da0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 msgr2=0x7fa7c0198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4005ab0 tx=0x7fa7b40042f0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.329 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 shutdown_connections 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa7c0106f10 0x7fa7c019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7fa7a80a2b80 0x7fa7a80a5040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7fa7c0102790 0x7fa7c0198950 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 --2- 172.21.15.19:0/421632448 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7fa7c0101de0 0x7fa7c0198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 >> 172.21.15.19:0/421632448 conn(0x7fa7c00fd8c0 msgr2=0x7fa7c00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 shutdown_connections 2024-09-12T03:34:34.330 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:34.329+0000 7fa7c64c8640 1 -- 172.21.15.19:0/421632448 wait complete. 2024-09-12T03:34:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6838/3679509562,v1:172.21.15.19:6839/3679509562] up:boot 2024-09-12T03:34:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.19:6840/3893847284,v1:172.21.15.19:6841/3893847284] up:boot 2024-09-12T03:34:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6833/4249437562,v1:172.21.15.121:6835/4249437562] up:boot 2024-09-12T03:34:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6832/1599461005,v1:172.21.15.121:6834/1599461005] up:boot 2024-09-12T03:34:34.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6833/2982557500,v1:172.21.15.135:6836/2982557500] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6832/2693331598,v1:172.21.15.135:6837/2693331598] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6835/515623788,v1:172.21.15.135:6839/515623788] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.135:6834/2088583741,v1:172.21.15.135:6838/2088583741] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6836/2090886336,v1:172.21.15.121:6838/2090886336] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: mds.? [v2:172.21.15.121:6837/3415875653,v1:172.21.15.121:6839/3415875653] up:boot 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:34:34.651 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:34:34.652 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:34 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:34:34.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6838/3679509562,v1:172.21.15.19:6839/3679509562] up:boot 2024-09-12T03:34:34.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.19:6840/3893847284,v1:172.21.15.19:6841/3893847284] up:boot 2024-09-12T03:34:34.662 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6833/4249437562,v1:172.21.15.121:6835/4249437562] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6832/1599461005,v1:172.21.15.121:6834/1599461005] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6833/2982557500,v1:172.21.15.135:6836/2982557500] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6832/2693331598,v1:172.21.15.135:6837/2693331598] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6835/515623788,v1:172.21.15.135:6839/515623788] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.135:6834/2088583741,v1:172.21.15.135:6838/2088583741] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6836/2090886336,v1:172.21.15.121:6838/2090886336] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: mds.? [v2:172.21.15.121:6837/3415875653,v1:172.21.15.121:6839/3415875653] up:boot 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:34:34.663 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:34:34.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:34 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:34:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6838/3679509562,v1:172.21.15.19:6839/3679509562] up:boot 2024-09-12T03:34:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.19:6840/3893847284,v1:172.21.15.19:6841/3893847284] up:boot 2024-09-12T03:34:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6833/4249437562,v1:172.21.15.121:6835/4249437562] up:boot 2024-09-12T03:34:34.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6832/1599461005,v1:172.21.15.121:6834/1599461005] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6833/2982557500,v1:172.21.15.135:6836/2982557500] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6832/2693331598,v1:172.21.15.135:6837/2693331598] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6835/515623788,v1:172.21.15.135:6839/515623788] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.135:6834/2088583741,v1:172.21.15.135:6838/2088583741] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6836/2090886336,v1:172.21.15.121:6838/2090886336] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: mds.? [v2:172.21.15.121:6837/3415875653,v1:172.21.15.121:6839/3415875653] up:boot 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:34:34.795 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='mgr.14150 172.21.15.19:0/3645850271' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:34:34.796 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:34 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:34:35.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:34:35.290 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: pgmap v804: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: Cluster is now healthy 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:34:35.585 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:35 smithi019 ceph-mon[30953]: fsmap 12 up:standby 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: pgmap v804: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: Cluster is now healthy 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:34:35.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:35 smithi121 ceph-mon[34963]: fsmap 12 up:standby 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: pgmap v804: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: Cluster is now healthy 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/421632448' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:34:35.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:35 smithi135 ceph-mon[35474]: fsmap 12 up:standby 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 -- 172.21.15.19:0/3340904313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d010c2f0 msgr2=0x7f32d010e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3340904313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d010c2f0 0x7f32d010e790 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009ca0 tx=0x7f32c802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 -- 172.21.15.19:0/3340904313 shutdown_connections 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3340904313 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d010c2f0 0x7f32d010e790 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3340904313 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d01037a0 0x7f32d010bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3340904313 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d0102df0 0x7f32d01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 -- 172.21.15.19:0/3340904313 >> 172.21.15.19:0/3340904313 conn(0x7f32d00fc7d0 msgr2=0x7f32d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 -- 172.21.15.19:0/3340904313 shutdown_connections 2024-09-12T03:34:35.795 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 -- 172.21.15.19:0/3340904313 wait complete. 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.788+0000 7f32d6f87640 1 Processor -- start 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 -- start start 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d0102df0 0x7f32d01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d010c2f0 0x7f32d019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7f32d0112170 con 0x7f32d01037a0 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f32d0111ff0 con 0x7f32d0102df0 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d6f87640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7f32d01122f0 con 0x7f32d010c2f0 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37920/0 (socket says 172.21.15.19:37920) 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 -- 172.21.15.19:0/3393722474 learned_addr learned my addr 172.21.15.19:0/3393722474 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:35.796 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d4d23640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d0102df0 0x7f32d01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32d5524640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d010c2f0 0x7f32d019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 -- 172.21.15.19:0/3393722474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d010c2f0 msgr2=0x7f32d019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d010c2f0 0x7f32d019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 -- 172.21.15.19:0/3393722474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d0102df0 msgr2=0x7f32d01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d0102df0 0x7f32d01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 -- 172.21.15.19:0/3393722474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d019d5a0 con 0x7f32d01037a0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.789+0000 7f32cffff640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c970 tx=0x7f32c400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.790+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c4007bb0 con 0x7f32d01037a0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.790+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d019d890 con 0x7f32d01037a0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.790+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32c4007d50 con 0x7f32d01037a0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.790+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c400f450 con 0x7f32d01037a0 2024-09-12T03:34:35.797 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.790+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d01a5170 con 0x7f32d01037a0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.791+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32d01031d0 con 0x7f32d01037a0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.792+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32c4004eb0 con 0x7f32d01037a0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.792+0000 7f32cdffb640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.793+0000 7f32d4d23640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.793+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8641+0+0 (secure 0 0 0) 0x7f32c40ceee0 con 0x7f32d01037a0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.793+0000 7f32d4d23640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:35.798 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.797+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f32c4014030 con 0x7f32d01037a0 2024-09-12T03:34:35.973 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:35.973+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f32d01072c0 con 0x7f32d01037a0 2024-09-12T03:34:36.334 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.334+0000 7f32cdffb640 1 -- 172.21.15.19:0/3393722474 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v84) ==== 181+0+0 (secure 0 0 0) 0x7f32c409bc40 con 0x7f32d01037a0 2024-09-12T03:34:36.334 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.337+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 msgr2=0x7f32b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.337+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c0009110 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.337+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 msgr2=0x7f32d0198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.337+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f32c400c970 tx=0x7f32c400cd70 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 shutdown_connections 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7f32d010c2f0 0x7f32d019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7f32b80a2b80 0x7f32b80a5040 secure :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f32c0009990 tx=0x7f32c0009110 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7f32d01037a0 0x7f32d0198b30 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 --2- 172.21.15.19:0/3393722474 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f32d0102df0 0x7f32d01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:36.338 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 >> 172.21.15.19:0/3393722474 conn(0x7f32d00fc7d0 msgr2=0x7f32d00fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 shutdown_connections 2024-09-12T03:34:36.339 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:36.338+0000 7f32d6f87640 1 -- 172.21.15.19:0/3393722474 wait complete. 2024-09-12T03:34:36.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:36 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:36.664 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:36 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:36.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:36 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:37.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:34:37.303 INFO:teuthology.orchestra.run.smithi019.stderr:Inferring config /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:34:37.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:37 smithi019 ceph-mon[30953]: pgmap v805: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T03:34:37.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:37 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:37.588 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:37 smithi019 ceph-mon[30953]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:34:37.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:37 smithi121 ceph-mon[34963]: pgmap v805: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T03:34:37.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:37 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:37.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:37 smithi121 ceph-mon[34963]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:34:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:37 smithi135 ceph-mon[35474]: pgmap v805: 97 pgs: 97 active+clean; 969 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-09-12T03:34:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:37 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/3393722474' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:37.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:37 smithi135 ceph-mon[35474]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:34:37.812 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 -- 172.21.15.19:0/2679402066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c101de0 msgr2=0x7eff9c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 --2- 172.21.15.19:0/2679402066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c101de0 0x7eff9c1021c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009900 tx=0x7eff8c02a410 comp rx=0 tx=0).stop 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 -- 172.21.15.19:0/2679402066 shutdown_connections 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 --2- 172.21.15.19:0/2679402066 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c106f10 0x7eff9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 --2- 172.21.15.19:0/2679402066 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 0x7eff9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 --2- 172.21.15.19:0/2679402066 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c101de0 0x7eff9c1021c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 -- 172.21.15.19:0/2679402066 >> 172.21.15.19:0/2679402066 conn(0x7eff9c0fd8c0 msgr2=0x7eff9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 -- 172.21.15.19:0/2679402066 shutdown_connections 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.802+0000 7effa1896640 1 -- 172.21.15.19:0/2679402066 wait complete. 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 Processor -- start 2024-09-12T03:34:37.813 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 -- start start 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 0x7eff9c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 --2- >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 0x7eff9c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 -- --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c112010 con 0x7eff9c106f10 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c111e90 con 0x7eff9c101de0 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7effa1896640 1 -- --> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] -- mon_getmap magic: 0 -- 0x7eff9c112190 con 0x7eff9c102790 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.803+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 --2- >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.19:3300/0 says I am v2:172.21.15.19:37944/0 (socket says 172.21.15.19:37944) 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 -- 172.21.15.19:0/2320850929 learned_addr learned my addr 172.21.15.19:0/2320850929 (peer_addr_for_me v2:172.21.15.19:0/0) 2024-09-12T03:34:37.814 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff927fc640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 0x7eff9c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9affd640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 0x7eff9c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 -- 172.21.15.19:0/2320850929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 msgr2=0x7eff9c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 0x7eff9c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 -- 172.21.15.19:0/2320850929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 msgr2=0x7eff9c1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 0x7eff9c1985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 -- 172.21.15.19:0/2320850929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9c19d560 con 0x7eff9c106f10 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9affd640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 0x7eff9c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7eff9b7fe640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7eff9400b970 tx=0x7eff9400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94019070 con 0x7eff9c106f10 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff94014440 con 0x7eff9c106f10 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff9c19d850 con 0x7eff9c106f10 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff940075d0 con 0x7eff9c106f10 2024-09-12T03:34:37.815 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.804+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff9c1a5130 con 0x7eff9c106f10 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.805+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff9c102240 con 0x7eff9c106f10 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.807+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff9401c070 con 0x7eff9c106f10 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.807+0000 7effa0894640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 0x7eff880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.808+0000 7eff9affd640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 0x7eff880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.808+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8087+0+0 (secure 0 0 0) 0x7eff940ce530 con 0x7eff9c106f10 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.808+0000 7eff9affd640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 0x7eff880a50c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7eff8c000c00 tx=0x7eff8c032d50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:34:37.816 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.811+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7eff9409b570 con 0x7eff9c106f10 2024-09-12T03:34:37.991 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:37.990+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 --> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7eff9c109b60 con 0x7eff9c106f10 2024-09-12T03:34:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.343+0000 7effa0894640 1 -- 172.21.15.19:0/2320850929 <== mon.0 v2:172.21.15.19:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v85) ==== 169+0+0 (secure 0 0 0) 0x7eff9409ae70 con 0x7eff9c106f10 2024-09-12T03:34:38.343 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'cephfs_data' removed 2024-09-12T03:34:38.347 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 msgr2=0x7eff880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 0x7eff880a50c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7eff8c000c00 tx=0x7eff8c032d50 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 msgr2=0x7eff9c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7eff9400b970 tx=0x7eff9400bd70 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 shutdown_connections 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:6800/2264764467,v1:172.21.15.19:6801/2264764467] conn(0x7eff880a2c00 0x7eff880a50c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] conn(0x7eff9c106f10 0x7eff9c19ce80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.135:3300/0,v1:172.21.15.135:6789/0] conn(0x7eff9c102790 0x7eff9c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 --2- 172.21.15.19:0/2320850929 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7eff9c101de0 0x7eff9c1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 >> 172.21.15.19:0/2320850929 conn(0x7eff9c0fd8c0 msgr2=0x7eff9c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:34:38.348 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 shutdown_connections 2024-09-12T03:34:38.349 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-12T03:34:38.347+0000 7effa1896640 1 -- 172.21.15.19:0/2320850929 wait complete. 2024-09-12T03:34:38.641 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:38 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:38.649 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:38 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:38.794 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:38 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:34:39.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:34:39.109 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:34:39.149 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:34:39.197 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:34:39.197 DEBUG:teuthology.orchestra.run.smithi019:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:34:39.213 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:34:39.229 DEBUG:teuthology.orchestra.run.smithi135:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:34:39.245 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:34:39.245 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:34:39.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:34:39.345 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 systemd[1]: Stopping Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:39 smithi121 ceph-mon[34963]: pgmap v807: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:34:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:39 smithi121 ceph-mon[34963]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:39.400 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:39 smithi121 ceph-mon[34963]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:34:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:39 smithi135 ceph-mon[35474]: pgmap v807: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:34:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:39 smithi135 ceph-mon[35474]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:39.544 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:39 smithi135 ceph-mon[35474]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:34:39.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 ceph-mon[30953]: pgmap v807: 65 pgs: 65 active+clean; 577 KiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:34:39.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 ceph-mon[30953]: from='client.? 172.21.15.19:0/2320850929' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:34:39.598 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 ceph-mon[30953]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:34:39.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a[30949]: 2024-09-12T03:34:39.597+0000 7f516c03f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:39.864 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a[30949]: 2024-09-12T03:34:39.597+0000 7f516c03f640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:34:40.164 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:39 smithi019 podman[120335]: 2024-09-12 03:34:39.864516415 +0000 UTC m=+0.376863908 container died 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T03:34:40.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 podman[120335]: 2024-09-12 03:34:40.298347138 +0000 UTC m=+0.810694630 container cleanup 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-12T03:34:40.491 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 bash[120335]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:34:40.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:34:40.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 podman[120348]: 2024-09-12 03:34:40.491523433 +0000 UTC m=+0.628970595 container remove 4b7bab35e51ba613f50fd6a9568cf14f018838e252dc8ee92b12a12e614eb6f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:34:40.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T03:34:40.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 systemd[1]: Stopped Ceph mon.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:34:40.762 INFO:journalctl@ceph.mon.a.smithi019.stdout:Sep 12 03:34:40 smithi019 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.a.service: Consumed 24.391s CPU time. 2024-09-12T03:34:40.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:40.788 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:34:40.788 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:34:40.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:34:41.150 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:40 smithi121 systemd[1]: Stopping Ceph mon.b for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:41.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-b[34959]: 2024-09-12T03:34:41.176+0000 7f8bd1cea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:41.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-b[34959]: 2024-09-12T03:34:41.176+0000 7f8bd1cea640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:34:41.650 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:41 smithi121 podman[70781]: 2024-09-12 03:34:41.349721483 +0000 UTC m=+0.294610459 container died 9d07ff7117278aa67d51cb99cc006a278a4f4932ed3948103734dfa2f735b92a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:34:42.011 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:41 smithi121 podman[70781]: 2024-09-12 03:34:41.759904939 +0000 UTC m=+0.704793915 container cleanup 9d07ff7117278aa67d51cb99cc006a278a4f4932ed3948103734dfa2f735b92a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:34:42.011 INFO:journalctl@ceph.mon.b.smithi121.stdout:Sep 12 03:34:41 smithi121 bash[70781]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:34:42.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:34:42.279 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:42.279 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:34:42.279 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:34:42.279 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:34:42.644 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:42 smithi135 systemd[1]: Stopping Ceph mon.c for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-c[35469]: 2024-09-12T03:34:42.644+0000 7f998eb3b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-c[35469]: 2024-09-12T03:34:42.644+0000 7f998eb3b640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:34:43.044 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:42 smithi135 podman[71247]: 2024-09-12 03:34:42.807472609 +0000 UTC m=+0.285079620 container died a1047c0d1c5fb10626920d3d77913ba99deb3aa7adcda7e42e426e7097b0ff2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:34:43.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:43 smithi135 podman[71247]: 2024-09-12 03:34:43.219621889 +0000 UTC m=+0.697228898 container cleanup a1047c0d1c5fb10626920d3d77913ba99deb3aa7adcda7e42e426e7097b0ff2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:34:43.462 INFO:journalctl@ceph.mon.c.smithi135.stdout:Sep 12 03:34:43 smithi135 bash[71247]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:34:43.640 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:34:43.703 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:43.703 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:34:43.703 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:34:43.703 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:34:44.028 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:43 smithi019 systemd[1]: Stopping Ceph mgr.x for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:44.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x[31179]: 2024-09-12T03:34:44.027+0000 7faa7344a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:44.395 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:44 smithi019 podman[120457]: 2024-09-12 03:34:44.059019566 +0000 UTC m=+0.154938272 container died ec30d3fdcb6fed6fdba540e20baf94f5eb2a1071cf8d576336d0f0c1f1075fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:34:44.664 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:44 smithi019 podman[120457]: 2024-09-12 03:34:44.503654278 +0000 UTC m=+0.599572981 container cleanup ec30d3fdcb6fed6fdba540e20baf94f5eb2a1071cf8d576336d0f0c1f1075fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:34:44.664 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:44 smithi019 bash[120457]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:34:44.973 INFO:journalctl@ceph.mgr.x.smithi019.stdout:Sep 12 03:34:44 smithi019 podman[120472]: 2024-09-12 03:34:44.722126901 +0000 UTC m=+0.655854041 container remove ec30d3fdcb6fed6fdba540e20baf94f5eb2a1071cf8d576336d0f0c1f1075fa7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:34:44.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:34:45.014 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:45.015 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:34:45.015 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:34:45.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:34:45.360 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:34:45 smithi121 systemd[1]: Stopping Ceph mgr.y for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:45.650 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:34:45 smithi121 podman[70907]: 2024-09-12 03:34:45.361037017 +0000 UTC m=+0.146198760 container died 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:34:46.067 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:34:45 smithi121 podman[70907]: 2024-09-12 03:34:45.77415577 +0000 UTC m=+0.559317512 container cleanup 9db0429741c2e3c8f7d83635c478d04eb6b63045e95cfa6c66b78a278b64d6d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:34:46.067 INFO:journalctl@ceph.mgr.y.smithi121.stdout:Sep 12 03:34:45 smithi121 bash[70907]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:34:46.195 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:34:46.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:46.230 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:34:46.230 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:34:46.230 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:34:46.544 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:34:46 smithi135 systemd[1]: Stopping Ceph mgr.z for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:46.842 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:34:46 smithi135 podman[71369]: 2024-09-12 03:34:46.571167572 +0000 UTC m=+0.147277616 container died f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:34:47.285 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:34:46 smithi135 podman[71369]: 2024-09-12 03:34:46.950811469 +0000 UTC m=+0.526921502 container cleanup f0b2323b8e0d0837391f30d74baddfe1a241e53d34e921a798db4f8b6d2ec36b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:34:47.285 INFO:journalctl@ceph.mgr.z.smithi135.stdout:Sep 12 03:34:46 smithi135 bash[71369]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:34:47.413 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:34:47.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:47.448 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:34:47.448 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:34:47.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:34:47.776 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:47 smithi019 systemd[1]: Stopping Ceph osd.0 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:48.164 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:34:47.775+0000 7f68e113d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:48.164 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:34:47.775+0000 7f68e113d640 -1 osd.0 85 *** Got signal Terminated *** 2024-09-12T03:34:48.164 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0[44882]: 2024-09-12T03:34:47.775+0000 7f68e113d640 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:34:53.664 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:53 smithi019 podman[120579]: 2024-09-12 03:34:53.344231079 +0000 UTC m=+5.687880577 container died d0e1b6c98c0e51dc676d4169dcab784474a037be35a83e753c8beedde23cd265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:34:54.119 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:53 smithi019 podman[120579]: 2024-09-12 03:34:53.867860206 +0000 UTC m=+6.211509698 container cleanup d0e1b6c98c0e51dc676d4169dcab784474a037be35a83e753c8beedde23cd265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:34:54.119 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:53 smithi019 bash[120579]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:34:54.414 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120594]: 2024-09-12 03:34:54.119943043 +0000 UTC m=+0.765997546 container remove d0e1b6c98c0e51dc676d4169dcab784474a037be35a83e753c8beedde23cd265 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:34:54.814 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120686]: 2024-09-12 03:34:54.454671451 +0000 UTC m=+0.016647974 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:34:54.814 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120686]: 2024-09-12 03:34:54.579196274 +0000 UTC m=+0.141172784 container create 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:34:55.164 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120686]: 2024-09-12 03:34:54.815428516 +0000 UTC m=+0.377405026 container init 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:34:55.165 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120686]: 2024-09-12 03:34:54.819148173 +0000 UTC m=+0.381124684 container start 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3) 2024-09-12T03:34:55.165 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:54 smithi019 podman[120686]: 2024-09-12 03:34:54.91539633 +0000 UTC m=+0.477372839 container attach 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:34:55.165 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:55 smithi019 podman[120686]: 2024-09-12 03:34:55.016709791 +0000 UTC m=+0.578686303 container died 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:34:55.834 INFO:journalctl@ceph.osd.0.smithi019.stdout:Sep 12 03:34:55 smithi019 podman[120709]: 2024-09-12 03:34:55.82069646 +0000 UTC m=+0.793499281 container remove 5cc334e2ba2945202a935f24c1c87fb4c4060187c9329cfa5a1c51d9cacb5d60 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:34:55.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:34:55.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:34:55.881 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:34:55.881 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:34:55.881 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:34:56.261 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:34:55 smithi121 systemd[1]: Stopping Ceph osd.1 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:34:56.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:34:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:34:56.260+0000 7f87b2841640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:34:56.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:34:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:34:56.260+0000 7f87b2841640 -1 osd.1 85 *** Got signal Terminated *** 2024-09-12T03:34:56.650 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:34:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1[39434]: 2024-09-12T03:34:56.260+0000 7f87b2841640 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:02.139 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:01 smithi121 podman[71033]: 2024-09-12 03:35:01.821571534 +0000 UTC m=+5.746327202 container died 30fcce2823ad4a79c8803cda5e7e681f04e725bd10c546f6dfda66a3e2dbf91f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:35:02.400 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:02 smithi121 podman[71033]: 2024-09-12 03:35:02.244717748 +0000 UTC m=+6.169473415 container cleanup 30fcce2823ad4a79c8803cda5e7e681f04e725bd10c546f6dfda66a3e2dbf91f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-12T03:35:02.400 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:02 smithi121 bash[71033]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:35:02.899 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:02 smithi121 podman[71048]: 2024-09-12 03:35:02.512976888 +0000 UTC m=+0.686968720 container remove 30fcce2823ad4a79c8803cda5e7e681f04e725bd10c546f6dfda66a3e2dbf91f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:35:03.207 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:02 smithi121 podman[71138]: 2024-09-12 03:35:02.831770772 +0000 UTC m=+0.015829982 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:03.207 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:02 smithi121 podman[71138]: 2024-09-12 03:35:02.94709453 +0000 UTC m=+0.131153738 container create 384a1d9c6d5a8171c8ff6f46394dabb7780f9e57016d4cb0862c6fc9924a0f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T03:35:03.207 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:03 smithi121 podman[71138]: 2024-09-12 03:35:03.207890427 +0000 UTC m=+0.391949641 container init 384a1d9c6d5a8171c8ff6f46394dabb7780f9e57016d4cb0862c6fc9924a0f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:35:03.481 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:03 smithi121 podman[71138]: 2024-09-12 03:35:03.211817788 +0000 UTC m=+0.395876999 container start 384a1d9c6d5a8171c8ff6f46394dabb7780f9e57016d4cb0862c6fc9924a0f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:03.481 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:03 smithi121 podman[71138]: 2024-09-12 03:35:03.316544813 +0000 UTC m=+0.500604027 container attach 384a1d9c6d5a8171c8ff6f46394dabb7780f9e57016d4cb0862c6fc9924a0f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:35:03.844 INFO:journalctl@ceph.osd.1.smithi121.stdout:Sep 12 03:35:03 smithi121 podman[71138]: 2024-09-12 03:35:03.482407854 +0000 UTC m=+0.666467076 container died 384a1d9c6d5a8171c8ff6f46394dabb7780f9e57016d4cb0862c6fc9924a0f66 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:35:04.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:35:04.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:04.194 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:35:04.194 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:35:04.194 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:35:04.544 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:04 smithi135 systemd[1]: Stopping Ceph osd.2 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:05.294 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:35:04.829+0000 7f3b917df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:05.294 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:35:04.829+0000 7f3b917df640 -1 osd.2 85 *** Got signal Terminated *** 2024-09-12T03:35:05.294 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2[40429]: 2024-09-12T03:35:04.829+0000 7f3b917df640 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:10.490 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:10 smithi135 podman[71508]: 2024-09-12 03:35:10.18695447 +0000 UTC m=+5.795280115 container died 52fac6e3f03b74b15c96c84c295bffb811b57ecf44fa4911529632e8ea2bab37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:35:10.794 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:10 smithi135 podman[71508]: 2024-09-12 03:35:10.589008333 +0000 UTC m=+6.197333983 container cleanup 52fac6e3f03b74b15c96c84c295bffb811b57ecf44fa4911529632e8ea2bab37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:35:10.794 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:10 smithi135 bash[71508]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:35:11.150 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:10 smithi135 podman[71526]: 2024-09-12 03:35:10.840758229 +0000 UTC m=+0.649418647 container remove 52fac6e3f03b74b15c96c84c295bffb811b57ecf44fa4911529632e8ea2bab37 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:11.461 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.05371016 +0000 UTC m=+0.016194042 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:11.461 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.166491633 +0000 UTC m=+0.128975509 container create a06df38037319c35cd491de8f4910a38b8c99ed81fbfa486fc1ee3127d45cd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:11.461 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.46113006 +0000 UTC m=+0.423613936 container init a06df38037319c35cd491de8f4910a38b8c99ed81fbfa486fc1ee3127d45cd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:11.794 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.464774077 +0000 UTC m=+0.427257954 container start a06df38037319c35cd491de8f4910a38b8c99ed81fbfa486fc1ee3127d45cd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:35:11.794 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.569546157 +0000 UTC m=+0.532030053 container attach a06df38037319c35cd491de8f4910a38b8c99ed81fbfa486fc1ee3127d45cd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:35:12.222 INFO:journalctl@ceph.osd.2.smithi135.stdout:Sep 12 03:35:11 smithi135 podman[71615]: 2024-09-12 03:35:11.86972549 +0000 UTC m=+0.832209376 container died a06df38037319c35cd491de8f4910a38b8c99ed81fbfa486fc1ee3127d45cd1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:12.549 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:35:12.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:12.590 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:35:12.590 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:35:12.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:35:12.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:12 smithi019 systemd[1]: Stopping Ceph osd.3 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:13.414 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:12.923+0000 7fac47802640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:13.414 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:12.923+0000 7fac47802640 -1 osd.3 85 *** Got signal Terminated *** 2024-09-12T03:35:13.415 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:12 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:12.923+0000 7fac47802640 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:14.339+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:15.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:14 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:14.902+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:14 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:14.931+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:15.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:15 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:15.337+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:15.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:15 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:15.638+0000 7fac43e18640 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.220326+0000 front 2024-09-12T03:34:52.220220+0000 (oldest deadline 2024-09-12T03:35:15.120407+0000) 2024-09-12T03:35:16.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:15.913+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:15.952+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:16.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:16 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:16.291+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:16.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:16 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:16.608+0000 7fac43e18640 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.220326+0000 front 2024-09-12T03:34:52.220220+0000 (oldest deadline 2024-09-12T03:35:15.120407+0000) 2024-09-12T03:35:17.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:16.921+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:17.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:16.990+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:17.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:17 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:17.296+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:17.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:17 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:17.218+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:17.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:17 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3[51738]: 2024-09-12T03:35:17.603+0000 7fac43e18640 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.220326+0000 front 2024-09-12T03:34:52.220220+0000 (oldest deadline 2024-09-12T03:35:15.120407+0000) 2024-09-12T03:35:17.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:17 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:17.622+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:18.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:17 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:17.968+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:18.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:17 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:17.954+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:18.649 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:18 smithi019 podman[120800]: 2024-09-12 03:35:18.357447877 +0000 UTC m=+5.563216055 container died 36572ebd5aab4c446d5bccefd13ea2b84bd0f2ec31617e485a8051678d41af09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:35:18.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:18 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:18.231+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:18.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:18 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:18.291+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:18.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:18.457+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:18.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:18 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:18.648+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:18.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:18 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:18.650+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:18.930+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:19.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:18.968+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:19.342 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:18 smithi019 podman[120800]: 2024-09-12 03:35:18.957131907 +0000 UTC m=+6.162900116 container cleanup 36572ebd5aab4c446d5bccefd13ea2b84bd0f2ec31617e485a8051678d41af09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:35:19.342 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:18 smithi019 bash[120800]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:35:19.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:19 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:19.242+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:19.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:19 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:19.219+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:19.652 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:19 smithi019 podman[120828]: 2024-09-12 03:35:19.342175358 +0000 UTC m=+0.981334271 container remove 36572ebd5aab4c446d5bccefd13ea2b84bd0f2ec31617e485a8051678d41af09 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:19.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:19.496+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:19.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:19 smithi019 podman[120915]: 2024-09-12 03:35:19.655545188 +0000 UTC m=+0.016641481 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:19.915 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:19 smithi019 podman[120915]: 2024-09-12 03:35:19.776572639 +0000 UTC m=+0.137668919 container create 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:19.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:19 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:19.683+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:19.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:19 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:19.651+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:20.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:19.949+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:20.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:19.975+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:20.414 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:20 smithi019 podman[120915]: 2024-09-12 03:35:20.029202967 +0000 UTC m=+0.390299257 container init 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:35:20.414 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:20 smithi019 podman[120915]: 2024-09-12 03:35:20.032990811 +0000 UTC m=+0.394087097 container start 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:35:20.415 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:20 smithi019 podman[120915]: 2024-09-12 03:35:20.112425978 +0000 UTC m=+0.473522270 container attach 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:35:20.415 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:20 smithi019 podman[120915]: 2024-09-12 03:35:20.228969655 +0000 UTC m=+0.590065936 container died 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:20.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:20 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:20.174+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:20.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:20 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:20.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:20.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:20.476+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:20.914 INFO:journalctl@ceph.osd.3.smithi019.stdout:Sep 12 03:35:20 smithi019 podman[120939]: 2024-09-12 03:35:20.908848856 +0000 UTC m=+0.673818251 container remove 6321a7de70fad9131bed63453d1a78b5f9bfbbe2d447ed3a2152fa3e3debb640 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:20.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:20 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:20.697+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:20.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:20 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:20.661+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:20.937 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:35:20.983 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:20.983 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:35:20.983 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:35:20.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:35:21.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:20.947+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:21.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:20.985+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:21.400 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:21 smithi121 systemd[1]: Stopping Ceph osd.4 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:21.400 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:21 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:21.130+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:21.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:21 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:21.236+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:21.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:21.507+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:21.899 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:21 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:21.422+0000 7f8ea3d64640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:21.900 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:21 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:21.422+0000 7f8ea3d64640 -1 osd.4 85 *** Got signal Terminated *** 2024-09-12T03:35:21.900 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:21 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:21.422+0000 7f8ea3d64640 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:22.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:21 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:21.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:22.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:21 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:21.695+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:22.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:21.965+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:21.997+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:22.399 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:22 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:22.110+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:22.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:22 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:22.212+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:22.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:22.510+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:22.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:22 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:22.660+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:22.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:22 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:22.681+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:23.269 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:22.967+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:23.269 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:23.005+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:23.270 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:23.005+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:23.400 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:23 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:23.113+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:23.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:23 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:23.168+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:23.544 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:23.501+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:23.544 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:23 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:23.501+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:23.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:23 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:23.565+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:24.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:23.671+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:24.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:23.671+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:24.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:23.716+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:24.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:23 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:23.716+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:24.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:24.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:24.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:24.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:24.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:24.008+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:24.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:24.008+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:24.399 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:24 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:24.095+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:24.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:24 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:24.190+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:24.563 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:24.540+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:24.564 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:24.540+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:24.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:24 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:24.591+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:25.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:24 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:24.686+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:25.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:24 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:24.686+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:25.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:24 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:24.727+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:25.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:24 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:24.727+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:25.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:24.981+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:25.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:24 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:24.981+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:25.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:25.031+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:25.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:25.031+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:25.400 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:25 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:25.047+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:25.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:25 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:25.211+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:25.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:25.590+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:25.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:25.590+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:25.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:25 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:25.626+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:26.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:25.712+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:26.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:25.712+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:26.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:25.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:26.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:25.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:26.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:25.997+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:26.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:25.997+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:26.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:25.943+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:26.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:25 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:25.943+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:26.399 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:26 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4[46041]: 2024-09-12T03:35:26.047+0000 7f8ea037a640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.131392+0000 front 2024-09-12T03:34:51.131230+0000 (oldest deadline 2024-09-12T03:35:17.031108+0000) 2024-09-12T03:35:26.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:26 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:26.172+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:26.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:26 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:26.630+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:26.919 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:26 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:26.589+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:26.919 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:26 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:26.589+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:27.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:26 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:26.782+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:27.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:26 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:26.782+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:27.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:26 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:26.724+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:27.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:26 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:26.724+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:27.255 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:26 smithi121 podman[71253]: 2024-09-12 03:35:26.953811904 +0000 UTC m=+5.773154094 container died 6f6641e7bbe04f0b4a6f75c486a81f67451b44792103052bae702d7bee150d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-12T03:35:27.256 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:27 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:27.162+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:27.256 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:27 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:27.162+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:27.295 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:27.003+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:27.382 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:27.003+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:27.382 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:26 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:26.919+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:27.382 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:26 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:26.919+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:27.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:27 smithi121 podman[71253]: 2024-09-12 03:35:27.355510513 +0000 UTC m=+6.174852704 container cleanup 6f6641e7bbe04f0b4a6f75c486a81f67451b44792103052bae702d7bee150d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:35:27.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:27 smithi121 bash[71253]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:35:27.944 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:27.598+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:27.944 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:27.598+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:27.966 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:27 smithi121 podman[71268]: 2024-09-12 03:35:27.657085315 +0000 UTC m=+0.686874245 container remove 6f6641e7bbe04f0b4a6f75c486a81f67451b44792103052bae702d7bee150d07 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:35:27.966 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:27 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:27.671+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:28.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:27 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:27.752+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:28.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:27 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:27.752+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:28.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:27 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:27.781+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:28.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:27 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:27.781+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:28.251 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:28 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:28.130+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:28.251 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:28 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:28.130+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:28.251 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:27 smithi121 podman[71358]: 2024-09-12 03:35:27.86950735 +0000 UTC m=+0.015900862 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:28.251 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:28 smithi121 podman[71358]: 2024-09-12 03:35:28.024568849 +0000 UTC m=+0.170962351 container create 698dc133d7ee5c439fd6d4338bd12f0e425238ab89810c4bd761a8a79f40adf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:35:28.251 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:28 smithi121 podman[71358]: 2024-09-12 03:35:28.252060413 +0000 UTC m=+0.398453921 container init 698dc133d7ee5c439fd6d4338bd12f0e425238ab89810c4bd761a8a79f40adf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:35:28.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:27.984+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:28.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:27.984+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:28.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:27.944+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:28.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:27 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:27.944+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:28.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:28 smithi121 podman[71358]: 2024-09-12 03:35:28.255356656 +0000 UTC m=+0.401750160 container start 698dc133d7ee5c439fd6d4338bd12f0e425238ab89810c4bd761a8a79f40adf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-12T03:35:28.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:28 smithi121 podman[71358]: 2024-09-12 03:35:28.352131088 +0000 UTC m=+0.498524609 container attach 698dc133d7ee5c439fd6d4338bd12f0e425238ab89810c4bd761a8a79f40adf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T03:35:28.650 INFO:journalctl@ceph.osd.4.smithi121.stdout:Sep 12 03:35:28 smithi121 podman[71358]: 2024-09-12 03:35:28.445544956 +0000 UTC m=+0.591938473 container died 698dc133d7ee5c439fd6d4338bd12f0e425238ab89810c4bd761a8a79f40adf5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:35:28.941 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:28.608+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:28.941 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:28.608+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:29.083 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:28 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:28.701+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:29.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:28 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:28.739+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:29.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:28 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:28.739+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:29.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:28 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:28.784+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:29.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:28 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:28.784+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:29.172 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:35:29.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:29.213 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:35:29.213 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:35:29.213 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:35:29.242 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:28.993+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:29.243 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:28.993+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:29.243 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:28.941+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:29.243 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:28 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:28.941+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:29.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:29 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:29.083+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:29.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:29 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:29.083+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:29.535 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 systemd[1]: Stopping Ceph osd.5 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:29.794 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:29.535+0000 7f3048fb7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:29.794 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:29.535+0000 7f3048fb7640 -1 osd.5 85 *** Got signal Terminated *** 2024-09-12T03:35:29.794 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:29.535+0000 7f3048fb7640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:29.795 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:29.651+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:29.795 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:29.651+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:30.036 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:29 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:29.656+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:30.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:29.709+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:30.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:29.709+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:30.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:29.753+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:30.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:29.753+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:30.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:29.997+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:30.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:29.997+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:30.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:29.957+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:30.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:29 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:29.957+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:30.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:30 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:30.036+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:30.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:30 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:30.036+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:30.962 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:30.613+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:30.962 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:30.613+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:31.069 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:30 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:30.652+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:31.069 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:30 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:30.652+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:31.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:30 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:30.704+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:31.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:30 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:30.704+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:31.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:30 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:30.787+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:31.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:30 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:30.787+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:31.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:30.972+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:31.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:30.972+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:31.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:30.961+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:31.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:30 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:30.961+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:31.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:31.069+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:31.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:31.069+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:31.930 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:31.618+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:31.931 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:31.618+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:32.060 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:31.685+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:32.060 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:31 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:31.685+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:32.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:31.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:32.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:31.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:32.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:31.791+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:32.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:31.791+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:32.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:31 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:31.791+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:32.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:31.930+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:32.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:31.930+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:32.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:31.949+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:32.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:31.949+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:32.295 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:31 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:31.949+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.068354+0000 front 2024-09-12T03:35:09.068382+0000 (oldest deadline 2024-09-12T03:35:31.269049+0000) 2024-09-12T03:35:32.329 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:32.060+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:32.329 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:32.060+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:32.329 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:32.060+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:32.915 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:32.631+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:32.915 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:32.631+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:32.915 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:32.631+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:33.015 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:32.650+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:33.015 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:32 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:32.650+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:33.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:32.700+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:33.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:32.700+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:33.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:32.813+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:33.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:32.813+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:33.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:32.813+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:33.285 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:33 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:33.014+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:33.285 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:33 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:33.014+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:33.285 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:33 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:33.014+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:32.989+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:32.989+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:32.989+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.068354+0000 front 2024-09-12T03:35:09.068382+0000 (oldest deadline 2024-09-12T03:35:31.269049+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:32.915+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:32.915+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:33.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:32 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:32.915+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:33.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:33 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:33.639+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:33.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:33 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:33.639+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:33.950 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:33.673+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:33.950 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:33.673+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:33.950 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:33.673+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:34.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:33.706+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:34.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:33.706+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:34.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:33.769+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:34.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:33.769+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:34.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:33 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:33.769+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:34.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:34.012+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.865391+0000 front 2024-09-12T03:34:49.865186+0000 (oldest deadline 2024-09-12T03:35:13.965317+0000) 2024-09-12T03:35:34.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:34.012+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.567201+0000 front 2024-09-12T03:34:58.567043+0000 (oldest deadline 2024-09-12T03:35:23.266980+0000) 2024-09-12T03:35:34.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5[47039]: 2024-09-12T03:35:34.012+0000 7f30455cd640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.068354+0000 front 2024-09-12T03:35:09.068382+0000 (oldest deadline 2024-09-12T03:35:31.269049+0000) 2024-09-12T03:35:34.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:33.948+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:34.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:33.948+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:34.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:33 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:33.948+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:34.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:34 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:34.049+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:34.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:34 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:34.049+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:34.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:34 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:34.049+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:34.959 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:34.636+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:34.959 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:34.636+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:34.959 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:34.636+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:35.046 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:34 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:34.670+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:35.047 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:34 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:34.670+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:35.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:34.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:35.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:34.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:35.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:34.664+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:35.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:34.759+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:35.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:34.759+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:35.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:34 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:34.759+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:35.294 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:35 smithi135 podman[71741]: 2024-09-12 03:35:35.099166462 +0000 UTC m=+5.693133479 container died 85706fda8d605db15c01ba1cca5a83445d4c1eca6d6fab1ae825ee8358cdb6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-12T03:35:35.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:34.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:35.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:34.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:35.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:34 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:34.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:35.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:35 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:35.047+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:35.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:35 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:35.047+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:35.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:35 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:35.047+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:35.681 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:35 smithi135 podman[71741]: 2024-09-12 03:35:35.501742409 +0000 UTC m=+6.095709415 container cleanup 85706fda8d605db15c01ba1cca5a83445d4c1eca6d6fab1ae825ee8358cdb6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:35.681 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:35 smithi135 bash[71741]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:35:35.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:35 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:35.628+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:35.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:35 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:35.628+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:35.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:35.631+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:35.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:35.631+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:35.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:35.631+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:35.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:35.716+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:35.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:35.716+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:35.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:35 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:35.716+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:35.962 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:35 smithi135 podman[71758]: 2024-09-12 03:35:35.778503222 +0000 UTC m=+0.675440799 container remove 85706fda8d605db15c01ba1cca5a83445d4c1eca6d6fab1ae825ee8358cdb6e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:35:35.963 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:35.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:35.963 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:35.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:35.963 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:35.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:36.277 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.024970837 +0000 UTC m=+0.015676587 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:36.277 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.146043932 +0000 UTC m=+0.136749671 container create 9bf3fa4e01020046bce3afe8f16875cca1b91b2b10544304cad7acf82b6f7eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:35:36.277 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:35.962+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:36.278 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:35.962+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:36.278 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:35 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:35.962+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:36.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:36 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:36.087+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:36.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:36 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:36.087+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:36.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:36 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:36.087+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:36.533 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.432286357 +0000 UTC m=+0.422992105 container init 9bf3fa4e01020046bce3afe8f16875cca1b91b2b10544304cad7acf82b6f7eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:35:36.534 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.435728 +0000 UTC m=+0.426433740 container start 9bf3fa4e01020046bce3afe8f16875cca1b91b2b10544304cad7acf82b6f7eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:35:36.534 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.534255712 +0000 UTC m=+0.524961466 container attach 9bf3fa4e01020046bce3afe8f16875cca1b91b2b10544304cad7acf82b6f7eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:35:36.794 INFO:journalctl@ceph.osd.5.smithi135.stdout:Sep 12 03:35:36 smithi135 podman[71849]: 2024-09-12 03:35:36.623993214 +0000 UTC m=+0.614698956 container died 9bf3fa4e01020046bce3afe8f16875cca1b91b2b10544304cad7acf82b6f7eec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:35:36.794 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:36.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:36.795 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:36.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:36.795 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:36.680+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:36.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:36.613+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:36.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:36.613+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:36.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:36.613+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:36.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:36.764+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:36.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:36.764+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:36.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:36 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:36.764+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:37.112 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:36 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:36.659+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:37.112 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:36 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:36.659+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:37.287 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:36.997+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:37.287 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:36.997+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:37.288 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:36 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:36.997+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:37.308 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:35:37.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:37.389 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:35:37.389 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:35:37.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:35:37.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:37.111+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:37.399 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:37.111+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:37.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:37.111+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:37.724 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 systemd[1]: Stopping Ceph osd.6 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:37.724 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.563+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:37.724 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.563+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:37.724 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.563+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:38.008 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:37 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:37.729+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:38.008 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:37 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:37.729+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:38.008 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:37 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:37.729+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:38.149 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:37.659+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:38.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:37 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:37.659+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:38.164 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.727+0000 7f35f298c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:38.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.727+0000 7f35f298c640 -1 osd.6 85 *** Got signal Terminated *** 2024-09-12T03:35:38.165 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:37.727+0000 7f35f298c640 -1 osd.6 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:38.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:37.723+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:38.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:37.723+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:38.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:37.723+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:38.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:38.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:38.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:38.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:38.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:38.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:38.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:38.151+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:38.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:38.151+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:38.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:38.151+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:38.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:38.546+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:38.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:38.546+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:38.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:38.546+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:38.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:38.737+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:38.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:38.737+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:38.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:38.737+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:38.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:38 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:38.737+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:39.034 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:38.687+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:39.034 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:38.687+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:39.034 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:38 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:38.687+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:39.149 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:38.688+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:39.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:38.688+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:39.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:38 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:38.688+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:39.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:39.034+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:39.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:39.034+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:39.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:39.034+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:39.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:39.187+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:39.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:39.187+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:39.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:39.187+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:39.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:39.187+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:39.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:39.542+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:39.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:39.542+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:39.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:39.542+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:39.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:39.773+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:39.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:39.773+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:39.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:39.773+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:39.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:39 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:39.773+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:40.026 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:39.716+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:40.026 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:39.716+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:40.026 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:39 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:39.716+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:40.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:39.716+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:40.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:39.716+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:40.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:39 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:39.716+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:40.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:40.026+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:40.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:40.026+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:40.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:40.026+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:40.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:40.207+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:40.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:40.207+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:40.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:40.207+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:40.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:40.207+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:40.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:40.557+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:40.557+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:40.557+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:40.789+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:40.789+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:40.789+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:40.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:40 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:40.789+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:41.043 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:40.734+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:41.043 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:40.734+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:41.043 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:40 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:40.734+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:41.149 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:40.678+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:41.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:40.678+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:41.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:40.678+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:41.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:41.043+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:41.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:41.043+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:41.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:41.043+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:41.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:41.247+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:41.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:41.247+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:41.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:41.247+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:41.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:41.247+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:41.808 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:41.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:41.808 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:41.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:41.808 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:41.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:41.808 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:41.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.780329+0000 front 2024-09-12T03:35:16.780309+0000 (oldest deadline 2024-09-12T03:35:40.880448+0000) 2024-09-12T03:35:42.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:42.018+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:42.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:42.018+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:42.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:42.018+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:42.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:41.783+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:42.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:41.783+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:42.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:41.783+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:42.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:41 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:41.783+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:42.149 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:41.675+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:42.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:41.675+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:42.150 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:41 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:41.675+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:42.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:41.808+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:42.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:41.808+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:42.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:41.808+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:42.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:41 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:41.808+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:42.634 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:42.258+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:42.634 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:42.258+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:42.634 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:42.258+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:42.634 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:42.258+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:42.791 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:42.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.077787+0000 front 2024-09-12T03:34:52.077817+0000 (oldest deadline 2024-09-12T03:35:17.378021+0000) 2024-09-12T03:35:42.791 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:42.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.378855+0000 front 2024-09-12T03:34:57.378773+0000 (oldest deadline 2024-09-12T03:35:23.278627+0000) 2024-09-12T03:35:42.791 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:42.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.179968+0000 front 2024-09-12T03:35:09.179729+0000 (oldest deadline 2024-09-12T03:35:34.479536+0000) 2024-09-12T03:35:42.791 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6[58202]: 2024-09-12T03:35:42.522+0000 7f35eefa2640 -1 osd.6 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.780329+0000 front 2024-09-12T03:35:16.780309+0000 (oldest deadline 2024-09-12T03:35:40.880448+0000) 2024-09-12T03:35:42.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:42.634+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:42.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:42.634+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:42.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:42 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:42.634+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:43.019 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:42.739+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:43.019 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:42.739+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:43.019 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:42.739+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:43.019 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:42 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:42.739+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:43.071 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:42.790+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:43.071 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:42.790+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:43.071 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:42.790+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:43.072 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:42 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:42.790+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:43.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:43.019+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:43.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:43.019+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:43.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:43.019+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:43.414 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:43 smithi019 podman[121019]: 2024-09-12 03:35:43.231401761 +0000 UTC m=+5.630750292 container died e6dc5fe2895da7c81813326fff6f1feee80bb86178f0474701e5bd835d35d0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:43.591 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:43.243+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:43.591 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:43.243+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:43.591 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:43.243+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:43.591 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:43.243+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:43.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:43.591+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:43.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:43.591+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:43.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:43 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:43.591+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:43.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:43 smithi019 podman[121019]: 2024-09-12 03:35:43.736509522 +0000 UTC m=+6.135858053 container cleanup e6dc5fe2895da7c81813326fff6f1feee80bb86178f0474701e5bd835d35d0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:35:43.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:43 smithi019 bash[121019]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:35:43.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:43.746+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:43.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:43.746+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:43.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:43.746+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:43.916 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:43 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:43.746+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:44.024 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:43.746+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:44.024 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:43.746+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:44.024 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:43.746+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:44.024 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:43 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:43.746+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:44.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:44.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:44.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:44.316 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:43 smithi019 podman[121038]: 2024-09-12 03:35:43.987968991 +0000 UTC m=+0.751499806 container remove e6dc5fe2895da7c81813326fff6f1feee80bb86178f0474701e5bd835d35d0b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:35:44.573 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:44.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:44.573 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:44.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:44.573 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:44.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:44.573 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:44.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:44.591 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.219937096 +0000 UTC m=+0.016720920 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:44.591 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.364318349 +0000 UTC m=+0.161102173 container create 46504fc49d9242dc289f3af28e7819f2a744ba05ce02af5f25cc60f2f58eddbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:44.591 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.592028053 +0000 UTC m=+0.388811893 container init 46504fc49d9242dc289f3af28e7819f2a744ba05ce02af5f25cc60f2f58eddbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-12T03:35:44.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:44.573+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:44.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:44.573+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:44.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:44 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:44.573+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:44.914 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.595936819 +0000 UTC m=+0.392720648 container start 46504fc49d9242dc289f3af28e7819f2a744ba05ce02af5f25cc60f2f58eddbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.683484274 +0000 UTC m=+0.480268112 container attach 46504fc49d9242dc289f3af28e7819f2a744ba05ce02af5f25cc60f2f58eddbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.6.smithi019.stdout:Sep 12 03:35:44 smithi019 podman[121136]: 2024-09-12 03:35:44.788468869 +0000 UTC m=+0.585252702 container died 46504fc49d9242dc289f3af28e7819f2a744ba05ce02af5f25cc60f2f58eddbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:44.751+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:44.751+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:44.751+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:44.915 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:44 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:44.751+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:45.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.983+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:45.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.983+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:45.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:44.983+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:45.044 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:44.788+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:45.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:44.788+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:45.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:44.788+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:45.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:44 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:44.788+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:45.527 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:35:45.562 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:45.235+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:45.562 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:45.235+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:45.562 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:45.235+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:45.562 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:45.235+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:45.570 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:45.570 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:35:45.571 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:35:45.571 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.562+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.562+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.562+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 systemd[1]: Stopping Ceph osd.7 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.891+0000 7f695161d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.891+0000 7f695161d640 -1 osd.7 85 *** Got signal Terminated *** 2024-09-12T03:35:45.892 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:45.891+0000 7f695161d640 -1 osd.7 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:46.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:45.730+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:46.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:45.730+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:46.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:45.730+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:46.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:45 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:45.730+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:46.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:45.946+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:46.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:45.946+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:46.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:45.946+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:46.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:45.806+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:46.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:45.806+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:46.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:45.806+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:46.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:45 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:45.806+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:46.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:46.225+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:46.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:46.225+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:46.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:46.225+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:46.552 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:46.225+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:46.899 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:46.550+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:46.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:46.550+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:46.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:46 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:46.550+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:47.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:46.931+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:47.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:46.931+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:47.044 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:46.931+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:47.044 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:46.766+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:47.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:46.766+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:47.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:46.766+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:47.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:46 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:46.766+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:47.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:46.760+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:47.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:46.760+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:47.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:46.760+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:47.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:46 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:46.760+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:47.525 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:47.256+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:47.525 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:47.256+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:47.525 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:47.256+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:47.526 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:47.256+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:47.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:47.525+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:47.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:47.525+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:47.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:47 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:47.525+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:48.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:47.745+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:48.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:47.745+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:48.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:47.745+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:48.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:47 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:47.745+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:47.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:47.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:47.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:47.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:47.815+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:48.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:47.815+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:48.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:47.815+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:48.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:47 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:47.815+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:48.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:48.477+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:48.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:48.477+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:48.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:48.477+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:48.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:48.243+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:48.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:48.244+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:48.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:48.244+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:48.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:48.244+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:48.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:48 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:48.244+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:49.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:48.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:49.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:48.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:49.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:48.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:49.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:48 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:48.733+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:48.813+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:48.813+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:48.813+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:48.813+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:48.955+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:49.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:48.955+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:49.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:48.955+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:49.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:48 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:48.955+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:49.433+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:49.433+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:49.433+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:49.433+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:26.028898+0000 front 2024-09-12T03:35:26.028932+0000 (oldest deadline 2024-09-12T03:35:48.929034+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:49.272+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:49.272+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:49.272+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:49.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:49.272+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:49.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:49.272+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:50.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:49.767+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:50.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:49.767+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:50.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:49.767+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:50.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:49 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:49.767+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:50.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:49.855+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:50.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:49.855+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:50.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:49.855+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:50.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:49.855+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:50.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:49.918+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:50.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:49.918+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:50.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:49.918+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:50.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:49 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:49.918+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:50.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:50.438+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:57.524269+0000 front 2024-09-12T03:34:57.524223+0000 (oldest deadline 2024-09-12T03:35:22.824405+0000) 2024-09-12T03:35:50.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:50.438+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:09.225916+0000 front 2024-09-12T03:35:09.226088+0000 (oldest deadline 2024-09-12T03:35:30.325976+0000) 2024-09-12T03:35:50.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:50.438+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:12.527549+0000 front 2024-09-12T03:35:12.527470+0000 (oldest deadline 2024-09-12T03:35:38.427487+0000) 2024-09-12T03:35:50.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7[52971]: 2024-09-12T03:35:50.438+0000 7f6949c2b640 -1 osd.7 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:26.028898+0000 front 2024-09-12T03:35:26.028932+0000 (oldest deadline 2024-09-12T03:35:48.929034+0000) 2024-09-12T03:35:50.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:50.323+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:50.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:50.323+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:50.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:50.323+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:50.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:50.323+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:50.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:50 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:50.323+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:51.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:50 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:50.768+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:51.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:50 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:50.768+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:51.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:50 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:50.768+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:51.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:50 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:50.768+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:51.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:50.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:51.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:50.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:51.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:50.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:51.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:50.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:51.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:50.917+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:51.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:50.917+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:51.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:50.917+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:51.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:50 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:50.917+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:51 smithi121 podman[71473]: 2024-09-12 03:35:51.401243317 +0000 UTC m=+5.635352998 container died d9950e174d18b80ee477a8e40041e38b6d47845422d9d4ce5c100811ede93022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:51 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:51.353+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:51 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:51.353+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:51 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:51.353+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:51 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:51.353+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:51.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:51 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:51.353+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:52.083 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:51 smithi121 podman[71473]: 2024-09-12 03:35:51.782641949 +0000 UTC m=+6.016751619 container cleanup d9950e174d18b80ee477a8e40041e38b6d47845422d9d4ce5c100811ede93022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3) 2024-09-12T03:35:52.083 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:51 smithi121 bash[71473]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:35:52.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:51.788+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:52.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:51.788+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:52.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:51.788+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:52.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:51.788+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:52.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:51 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:51.788+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:52.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:51.868+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:52.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:51.868+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:52.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:51.868+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:52.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:51.868+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:52.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:51.937+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:52.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:51.937+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:52.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:51.937+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:52.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:51 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:51.937+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:52.367 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71488]: 2024-09-12 03:35:52.083992114 +0000 UTC m=+0.684693158 container remove d9950e174d18b80ee477a8e40041e38b6d47845422d9d4ce5c100811ede93022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:52 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:52.367+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:52 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:52.367+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:52 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:52.367+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:52 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:52.367+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:52 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:52.367+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:52.629 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.296819873 +0000 UTC m=+0.015959408 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:35:52.630 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.401668553 +0000 UTC m=+0.120808133 container create e53734d49e1f5a57a367842b0c1e5aea8d1d86b32dde0133ca8c1384ecaa8850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:35:52.630 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.629556857 +0000 UTC m=+0.348696391 container init e53734d49e1f5a57a367842b0c1e5aea8d1d86b32dde0133ca8c1384ecaa8850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True) 2024-09-12T03:35:52.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.632996452 +0000 UTC m=+0.352135993 container start e53734d49e1f5a57a367842b0c1e5aea8d1d86b32dde0133ca8c1384ecaa8850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:35:52.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.737789116 +0000 UTC m=+0.456928663 container attach e53734d49e1f5a57a367842b0c1e5aea8d1d86b32dde0133ca8c1384ecaa8850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T03:35:52.900 INFO:journalctl@ceph.osd.7.smithi121.stdout:Sep 12 03:35:52 smithi121 podman[71579]: 2024-09-12 03:35:52.816155432 +0000 UTC m=+0.535294968 container died e53734d49e1f5a57a367842b0c1e5aea8d1d86b32dde0133ca8c1384ecaa8850 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-12T03:35:53.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:52.827+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:53.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:52.827+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:53.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:52.827+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:53.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:52.827+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:53.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:52 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:52.827+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:52.900+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:52.900+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:52.900+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:52.900+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:52.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:53.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:52.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:53.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:52.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:53.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:52 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:52.968+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:53.449 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:53.375+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:53.449 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:53.375+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:53.449 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:53.375+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:53.449 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:53.375+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:53.449 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:53.375+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:53.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:35:53.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:35:53.521 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:35:53.521 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:35:53.521 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:35:53.898 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 systemd[1]: Stopping Ceph osd.8 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:35:53.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:53 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:53.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:53.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:53 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:53.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:53.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:53 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:53.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:53.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:53 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:53.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:53.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:53 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:53.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:53.988+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:53.988+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:53.988+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:53.988+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.897+0000 7efd7fd57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.897+0000 7efd7fd57640 -1 osd.8 85 *** Got signal Terminated *** 2024-09-12T03:35:54.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.897+0000 7efd7fd57640 -1 osd.8 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:35:54.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.945+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:54.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.945+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:54.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.945+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:54.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:53 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:53.945+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:54.701 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:54 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:54.409+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:54.701 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:54 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:54.409+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:54.701 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:54 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:54.409+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:54.701 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:54 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:54.409+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:54.701 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:54 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:54.409+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:55.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:54 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:54.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:55.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:54 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:54.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:55.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:54 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:54.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:55.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:54 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:54.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:55.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:54 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:54.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:55.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:54.967+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:55.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:54.967+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:55.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:54.967+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:55.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:54.967+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:55.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:54.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:55.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:54.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:55.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:54.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:55.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:54 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:54.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:55.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:55 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:55.387+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:55.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:55 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:55.387+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:55.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:55 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:55.387+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:55.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:55 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:55.387+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:55.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:55 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:55.387+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:56.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:55.844+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:56.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:55.844+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:56.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:55.844+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:56.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:55.844+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:56.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:55 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:55.844+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:56.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:55 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:55.955+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:55 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:55.955+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:55 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:55.955+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:55 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:55.955+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:56.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:56.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:56.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:56.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:56.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:56.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:35:56.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:56.350+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:56.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:56.350+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:56.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:56.350+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:56.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:56.350+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:56.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:56 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:56.350+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:57.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:56 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:56.839+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:57.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:56 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:56.839+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:57.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:56 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:56.839+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:57.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:56 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:56.839+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:57.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:56 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:56.839+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:57.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:56.907+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:57.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:56.907+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:57.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:56.907+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:57.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:56 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:56.907+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:57.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:57.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:57.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:57.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:57.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:57.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:57.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:57.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:57.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:57.008+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:57.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:57 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:57.311+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:35:58.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:58.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:58.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:58.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:58.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:58.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:57 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:57.810+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:35:58.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:57.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:58.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:57.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:58.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:57.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:58.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:57.893+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:58.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:58.037+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:58.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:58.037+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:58.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:58.037+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:58.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:58.037+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:58.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:58.037+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:35:58.652 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:58.653 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:58.653 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:58.653 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:58.653 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:58.653 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:58 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:58.285+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:35:59.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:35:59.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:35:59.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:35:59.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:35:59.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:35:59.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:58 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:58.796+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:35:59.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:58.896+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.103705+0000 front 2024-09-12T03:34:52.103925+0000 (oldest deadline 2024-09-12T03:35:18.003699+0000) 2024-09-12T03:35:59.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:58.896+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.004260+0000 front 2024-09-12T03:34:58.004158+0000 (oldest deadline 2024-09-12T03:35:22.604779+0000) 2024-09-12T03:35:59.294 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:58.896+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.706653+0000 front 2024-09-12T03:35:07.706712+0000 (oldest deadline 2024-09-12T03:35:31.806827+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:58.896+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:15.307731+0000 front 2024-09-12T03:35:15.307599+0000 (oldest deadline 2024-09-12T03:35:41.207745+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:58 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8[53997]: 2024-09-12T03:35:58.896+0000 7efd78365640 -1 osd.8 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:32.810229+0000 front 2024-09-12T03:35:32.810208+0000 (oldest deadline 2024-09-12T03:35:58.710361+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:59 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:59.071+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:59 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:59.071+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:59 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:59.071+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:59 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:59.071+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:35:59.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:35:59 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:35:59.071+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:35:59.609 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:59 smithi135 podman[71952]: 2024-09-12 03:35:59.334624636 +0000 UTC m=+5.621666254 container died 973b6e7447c59678db1656b9b9da0f557a44c20275bff5485e7e7400dab956bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:35:59.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:35:59 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:35:59.268+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:00.007 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:59 smithi135 podman[71952]: 2024-09-12 03:35:59.715491579 +0000 UTC m=+6.002533197 container cleanup 973b6e7447c59678db1656b9b9da0f557a44c20275bff5485e7e7400dab956bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:36:00.008 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:35:59 smithi135 bash[71952]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:36:00.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:00.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:00.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:00.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:00.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:00.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:35:59 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:35:59.835+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:00.297 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[71983]: 2024-09-12 03:36:00.008477604 +0000 UTC m=+0.664781283 container remove 973b6e7447c59678db1656b9b9da0f557a44c20275bff5485e7e7400dab956bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:00.297 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:00 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:00.088+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:00.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:00 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:00.088+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:00.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:00 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:00.088+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:00.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:00 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:00.088+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:00.298 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:00 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:00.088+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:00.612 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.220366322 +0000 UTC m=+0.015650220 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:36:00.613 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.342999312 +0000 UTC m=+0.138283253 container create 189eeafbde1da8aa52b7cd73974b999fbc8b851d443b267807cae02cf1b4a9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:36:00.613 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.613317172 +0000 UTC m=+0.408601065 container init 189eeafbde1da8aa52b7cd73974b999fbc8b851d443b267807cae02cf1b4a9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:00.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:00 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:00.264+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:01.044 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.616834544 +0000 UTC m=+0.412118442 container start 189eeafbde1da8aa52b7cd73974b999fbc8b851d443b267807cae02cf1b4a9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T03:36:01.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.703903052 +0000 UTC m=+0.499186968 container attach 189eeafbde1da8aa52b7cd73974b999fbc8b851d443b267807cae02cf1b4a9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T03:36:01.045 INFO:journalctl@ceph.osd.8.smithi135.stdout:Sep 12 03:36:00 smithi135 podman[72070]: 2024-09-12 03:36:00.803475386 +0000 UTC m=+0.598759281 container died 189eeafbde1da8aa52b7cd73974b999fbc8b851d443b267807cae02cf1b4a9f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:36:01.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:01.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:01.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:01.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:01.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:01.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:00 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:00.822+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:01.465 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:01.106+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:01.466 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:01.106+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:01.466 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:01.106+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:01.466 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:01.106+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:01.466 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:01.106+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:01.487 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:36:01.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:01.528 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:36:01.528 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:36:01.528 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:01.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:01 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:01.238+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:01.851 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 systemd[1]: Stopping Ceph osd.9 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:02.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:02.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:02.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.851+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.856+0000 7ff49267d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.856+0000 7ff49267d640 -1 osd.9 85 *** Got signal Terminated *** 2024-09-12T03:36:02.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:01 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:01.856+0000 7ff49267d640 -1 osd.9 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:36:02.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:02 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:02.139+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:02.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:02 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:02.139+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:02.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:02 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:02.139+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:02.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:02 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:02.139+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:02.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:02 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:02.139+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:02.551 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:02 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:02.240+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:03.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:03.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:03.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:03.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:03.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:03.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:02 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:02.826+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:03.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:03 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:03.156+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:03.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:03 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:03.156+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:03.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:03 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:03.156+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:03.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:03 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:03.156+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:03.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:03 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:03.156+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:03.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:03.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:03.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:03.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:03.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:03.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:03 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:03.260+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:04.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:04.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:04.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:04.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:04.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:04.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:03 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:03.838+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:04.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:04.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:04.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:04.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:04.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:04.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:04.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:04.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:04.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:04 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:04.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:04.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:04 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:04.233+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:05.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:05.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:05.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:05.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:05.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:05.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:04 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:04.858+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:05.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:05.159+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:05.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:05.159+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:05.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:05.159+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:05.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:05.159+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:05.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:05.159+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:05.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:05 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:05.277+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:06.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:06.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:06.164 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:06.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:06.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:06.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:06.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:05 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:05.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.847281+0000 front 2024-09-12T03:35:40.847324+0000 (oldest deadline 2024-09-12T03:36:05.547270+0000) 2024-09-12T03:36:06.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:06 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:06.133+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:06.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:06 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:06.133+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:06.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:06 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:06.133+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:06.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:06 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:06.133+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:06.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:06 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:06.133+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:06.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:06.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:06 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:06.298+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:52.340356+0000 front 2024-09-12T03:34:52.340251+0000 (oldest deadline 2024-09-12T03:35:18.240220+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:58.240888+0000 front 2024-09-12T03:34:58.240763+0000 (oldest deadline 2024-09-12T03:35:22.940998+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.842304+0000 front 2024-09-12T03:35:08.842396+0000 (oldest deadline 2024-09-12T03:35:31.142282+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.343748+0000 front 2024-09-12T03:35:16.343787+0000 (oldest deadline 2024-09-12T03:35:38.643866+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:24.545254+0000 front 2024-09-12T03:35:24.545352+0000 (oldest deadline 2024-09-12T03:35:50.945575+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.946252+0000 front 2024-09-12T03:35:30.946273+0000 (oldest deadline 2024-09-12T03:35:56.846092+0000) 2024-09-12T03:36:07.165 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:06 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9[64759]: 2024-09-12T03:36:06.840+0000 7ff48ec93640 -1 osd.9 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.847281+0000 front 2024-09-12T03:35:40.847324+0000 (oldest deadline 2024-09-12T03:36:05.547270+0000) 2024-09-12T03:36:07.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:07.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:07.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:07.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:07.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:07.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:07.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:07.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:07.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:07 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:07.172+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:07.648 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:07 smithi019 podman[121235]: 2024-09-12 03:36:07.350670095 +0000 UTC m=+5.619673049 container died a9e1cc426d92f85e795414c81c886442d8127cda1f9cd6b71e22dd314a6a342f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:36:07.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:07.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:07 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:07.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:07.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:07 smithi019 podman[121235]: 2024-09-12 03:36:07.815262764 +0000 UTC m=+6.084265760 container cleanup a9e1cc426d92f85e795414c81c886442d8127cda1f9cd6b71e22dd314a6a342f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:36:07.914 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:07 smithi019 bash[121235]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:36:08.391 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121251]: 2024-09-12 03:36:08.058245793 +0000 UTC m=+0.700793160 container remove a9e1cc426d92f85e795414c81c886442d8127cda1f9cd6b71e22dd314a6a342f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T03:36:08.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:08.129+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:08.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:08.129+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:08.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:08.129+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:08.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:08.129+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:08.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:08 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:08.129+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:08.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:08 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:08.286+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:08.664 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.294647664 +0000 UTC m=+0.016621267 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:36:08.665 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.392302321 +0000 UTC m=+0.114275973 container create e4109c4b68372d90d0309f57a7d4a845710ac3897a1b1a499b4d1c60c255d521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T03:36:08.665 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.620220713 +0000 UTC m=+0.342194315 container init e4109c4b68372d90d0309f57a7d4a845710ac3897a1b1a499b4d1c60c255d521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:08.665 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.623867937 +0000 UTC m=+0.345841530 container start e4109c4b68372d90d0309f57a7d4a845710ac3897a1b1a499b4d1c60c255d521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731) 2024-09-12T03:36:09.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.703337773 +0000 UTC m=+0.425311379 container attach e4109c4b68372d90d0309f57a7d4a845710ac3897a1b1a499b4d1c60c255d521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-12T03:36:09.123 INFO:journalctl@ceph.osd.9.smithi019.stdout:Sep 12 03:36:08 smithi019 podman[121339]: 2024-09-12 03:36:08.819544415 +0000 UTC m=+0.541518009 container died e4109c4b68372d90d0309f57a7d4a845710ac3897a1b1a499b4d1c60c255d521 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:36:09.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:36:09.464 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:09.464 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:36:09.464 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:36:09.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:09.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:09.124+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:09.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 systemd[1]: Stopping Ceph osd.10 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:10.149 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.775+0000 7fb4cffd3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:10.150 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.775+0000 7fb4cffd3640 -1 osd.10 85 *** Got signal Terminated *** 2024-09-12T03:36:10.150 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:09 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:09.775+0000 7fb4cffd3640 -1 osd.10 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:10.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:10 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:10.096+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:10.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:10 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:10.262+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:11.536 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:11 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:11.053+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:11.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:11.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:11.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:11.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:11.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:11.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:11.651 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:11 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:11.275+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:12.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:12 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:12.024+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:12.606 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:12 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:12.266+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:13.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:13.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:13.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:13.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:13.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:13.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:13.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.004+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:13.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:13 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:13.292+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:14.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:13 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:13.958+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:14.649 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:49.951288+0000 front 2024-09-12T03:34:49.951222+0000 (oldest deadline 2024-09-12T03:35:14.051198+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:35:00.452756+0000 front 2024-09-12T03:35:00.452658+0000 (oldest deadline 2024-09-12T03:35:26.352743+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:08.053942+0000 front 2024-09-12T03:35:08.053784+0000 (oldest deadline 2024-09-12T03:35:31.553967+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6814 osd.3 since back 2024-09-12T03:35:16.755624+0000 front 2024-09-12T03:35:16.755545+0000 (oldest deadline 2024-09-12T03:35:38.455552+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:21.356360+0000 front 2024-09-12T03:35:21.356385+0000 (oldest deadline 2024-09-12T03:35:47.756953+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:33.057764+0000 front 2024-09-12T03:35:33.057646+0000 (oldest deadline 2024-09-12T03:35:57.157563+0000) 2024-09-12T03:36:14.650 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:14 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10[59962]: 2024-09-12T03:36:14.270+0000 7fb4cc5e9640 -1 osd.10 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:40.658512+0000 front 2024-09-12T03:35:40.658461+0000 (oldest deadline 2024-09-12T03:36:03.558515+0000) 2024-09-12T03:36:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:15.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:15.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:15.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.009+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:15.442 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:15 smithi121 podman[71693]: 2024-09-12 03:36:15.170366536 +0000 UTC m=+5.496632054 container died e6ef3f0975a0801c8b05b7f7085b8bd760c88b5cf0d886f1d7b4c64e3fb4dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:15.834 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:15 smithi121 podman[71693]: 2024-09-12 03:36:15.541750536 +0000 UTC m=+5.868016056 container cleanup e6ef3f0975a0801c8b05b7f7085b8bd760c88b5cf0d886f1d7b4c64e3fb4dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:36:15.834 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:15 smithi121 bash[71693]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:36:16.141 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:15 smithi121 podman[71711]: 2024-09-12 03:36:15.834554261 +0000 UTC m=+0.657764379 container remove e6ef3f0975a0801c8b05b7f7085b8bd760c88b5cf0d886f1d7b4c64e3fb4dddd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:16.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:15 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:15.960+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:16.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.04424904 +0000 UTC m=+0.016021749 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:36:16.400 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.176927561 +0000 UTC m=+0.148700255 container create fb7fa81c5c2f4502154310d91d49346f477c70c493060c8b2ef72e8668119b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-12T03:36:16.898 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.420919008 +0000 UTC m=+0.392691711 container init fb7fa81c5c2f4502154310d91d49346f477c70c493060c8b2ef72e8668119b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:36:16.898 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.424252411 +0000 UTC m=+0.396025115 container start fb7fa81c5c2f4502154310d91d49346f477c70c493060c8b2ef72e8668119b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:16.899 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.495905437 +0000 UTC m=+0.467678148 container attach fb7fa81c5c2f4502154310d91d49346f477c70c493060c8b2ef72e8668119b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:36:16.899 INFO:journalctl@ceph.osd.10.smithi121.stdout:Sep 12 03:36:16 smithi121 podman[71797]: 2024-09-12 03:36:16.619132895 +0000 UTC m=+0.590905593 container died fb7fa81c5c2f4502154310d91d49346f477c70c493060c8b2ef72e8668119b9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-12T03:36:17.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:36:17.221 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:17.221 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:36:17.221 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:36:17.221 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:36:17.249 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:17.250 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:16 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:16.989+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:17.518 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:17 smithi135 systemd[1]: Stopping Ceph osd.11 for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:17.794 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:17 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:17.518+0000 7f1aa3964640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:17.794 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:17 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:17.518+0000 7f1aa3964640 -1 osd.11 85 *** Got signal Terminated *** 2024-09-12T03:36:17.794 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:17 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:17.518+0000 7f1aa3964640 -1 osd.11 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:36:18.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:18.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:18.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:18.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:18.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:18.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:18.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:18 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:18.006+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:19.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:19 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:19.041+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:20.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:20.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:20.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:20.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:20.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:20.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:20.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:20 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:20.040+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:21.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:21.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:21.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:21.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:21.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:21.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:21.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:21.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:21 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:21.042+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-09-12T03:35:57.464662+0000 front 2024-09-12T03:35:57.464696+0000 (oldest deadline 2024-09-12T03:36:20.964726+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6806 osd.0 since back 2024-09-12T03:34:51.553689+0000 front 2024-09-12T03:34:51.553774+0000 (oldest deadline 2024-09-12T03:35:14.453854+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6804 osd.1 since back 2024-09-12T03:34:59.155108+0000 front 2024-09-12T03:34:59.155136+0000 (oldest deadline 2024-09-12T03:35:22.655127+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6804 osd.2 since back 2024-09-12T03:35:07.956516+0000 front 2024-09-12T03:35:07.956372+0000 (oldest deadline 2024-09-12T03:35:32.056596+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6812 osd.4 since back 2024-09-12T03:35:25.959989+0000 front 2024-09-12T03:35:25.959913+0000 (oldest deadline 2024-09-12T03:35:47.059978+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6812 osd.5 since back 2024-09-12T03:35:30.560993+0000 front 2024-09-12T03:35:30.561010+0000 (oldest deadline 2024-09-12T03:35:55.861125+0000) 2024-09-12T03:36:22.294 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.19:6822 osd.6 since back 2024-09-12T03:35:42.262698+0000 front 2024-09-12T03:35:42.262645+0000 (oldest deadline 2024-09-12T03:36:08.162543+0000) 2024-09-12T03:36:22.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.121:6820 osd.7 since back 2024-09-12T03:35:50.463879+0000 front 2024-09-12T03:35:50.463935+0000 (oldest deadline 2024-09-12T03:36:12.763731+0000) 2024-09-12T03:36:22.295 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11[60563]: 2024-09-12T03:36:22.029+0000 7f1a9bf72640 -1 osd.11 85 heartbeat_check: no reply from 172.21.15.135:6820 osd.8 since back 2024-09-12T03:35:57.464662+0000 front 2024-09-12T03:35:57.464696+0000 (oldest deadline 2024-09-12T03:36:20.964726+0000) 2024-09-12T03:36:23.177 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:22 smithi135 podman[72170]: 2024-09-12 03:36:22.911074681 +0000 UTC m=+5.496673283 container died 583b37a384a5441f5ffc2b95fc9d7ae9e9b0c24f068f48673962d064fa9b85c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-12T03:36:23.543 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:23 smithi135 podman[72170]: 2024-09-12 03:36:23.276242804 +0000 UTC m=+5.861841405 container cleanup 583b37a384a5441f5ffc2b95fc9d7ae9e9b0c24f068f48673962d064fa9b85c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-12T03:36:23.543 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:23 smithi135 bash[72170]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:36:23.794 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:23 smithi135 podman[72185]: 2024-09-12 03:36:23.544154622 +0000 UTC m=+0.630627987 container remove 583b37a384a5441f5ffc2b95fc9d7ae9e9b0c24f068f48673962d064fa9b85c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:36:24.163 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:23 smithi135 podman[72274]: 2024-09-12 03:36:23.795251025 +0000 UTC m=+0.015735516 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:36:24.163 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:23 smithi135 podman[72274]: 2024-09-12 03:36:23.903113748 +0000 UTC m=+0.123598271 container create df78a0335867881d94f639d11cdb3b4ffd512c750b1bdc62204a6394a408b00b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T03:36:24.164 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:24 smithi135 podman[72274]: 2024-09-12 03:36:24.096945213 +0000 UTC m=+0.317429694 container init df78a0335867881d94f639d11cdb3b4ffd512c750b1bdc62204a6394a408b00b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:36:24.164 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:24 smithi135 podman[72274]: 2024-09-12 03:36:24.100128596 +0000 UTC m=+0.320613088 container start df78a0335867881d94f639d11cdb3b4ffd512c750b1bdc62204a6394a408b00b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:36:24.164 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:24 smithi135 podman[72274]: 2024-09-12 03:36:24.163582781 +0000 UTC m=+0.384067264 container attach df78a0335867881d94f639d11cdb3b4ffd512c750b1bdc62204a6394a408b00b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-12T03:36:24.544 INFO:journalctl@ceph.osd.11.smithi135.stdout:Sep 12 03:36:24 smithi135 podman[72274]: 2024-09-12 03:36:24.287350815 +0000 UTC m=+0.507835297 container died df78a0335867881d94f639d11cdb3b4ffd512c750b1bdc62204a6394a408b00b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:36:24.850 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:36:24.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:24.930 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:36:24.930 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:36:24.930 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:36:25.242 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:24 smithi019 systemd[1]: Stopping Ceph mds.a for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:25.501 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: 2024-09-12T03:36:25.241+0000 7f74f91b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:25.501 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:25 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a[71590]: 2024-09-12T03:36:25.241+0000 7f74f91b0640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:36:28.659 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:28 smithi019 podman[121443]: 2024-09-12 03:36:28.392798622 +0000 UTC m=+3.245192420 container died e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-12T03:36:28.915 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:28 smithi019 podman[121443]: 2024-09-12 03:36:28.80845396 +0000 UTC m=+3.660847758 container cleanup e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:36:28.915 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:28 smithi019 bash[121443]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:36:29.246 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:36:29.270 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:29 smithi019 podman[121466]: 2024-09-12 03:36:29.018123106 +0000 UTC m=+0.621370911 container remove e3086e8d88257ff486e6fb6224fa25c1c67e45505bc746fcc4299025e8f45025 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-12T03:36:29.270 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:29 smithi019 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.a.service: Deactivated successfully. 2024-09-12T03:36:29.270 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:29 smithi019 systemd[1]: Stopped Ceph mds.a for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:36:29.270 INFO:journalctl@ceph.mds.a.smithi019.stdout:Sep 12 03:36:29 smithi019 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.a.service: Consumed 21.814s CPU time. 2024-09-12T03:36:29.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:29.284 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:36:29.284 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:36:29.284 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:36:29.629 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:29 smithi019 systemd[1]: Stopping Ceph mds.d for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:29.914 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d[70290]: 2024-09-12T03:36:29.629+0000 7f877fbe8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:29.914 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:29 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d[70290]: 2024-09-12T03:36:29.629+0000 7f877fbe8640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:36:31.364 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:31 smithi019 podman[121575]: 2024-09-12 03:36:31.056174373 +0000 UTC m=+1.551277033 container died 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-12T03:36:31.664 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:31 smithi019 podman[121575]: 2024-09-12 03:36:31.529594114 +0000 UTC m=+2.024696772 container cleanup 2bbd538c6b875e5f91134f2311dea7302a6685fcbdb506e01a8bbbba910cb8b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-12T03:36:31.664 INFO:journalctl@ceph.mds.d.smithi019.stdout:Sep 12 03:36:31 smithi019 bash[121575]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:36:31.972 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:36:32.007 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:32.007 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:36:32.007 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:36:32.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:36:32.342 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:32 smithi019 systemd[1]: Stopping Ceph mds.g for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:32.664 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: 2024-09-12T03:36:32.341+0000 7faa59816640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:32.664 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:32 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g[71178]: 2024-09-12T03:36:32.341+0000 7faa59816640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:36:36.664 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:36 smithi019 podman[121694]: 2024-09-12 03:36:36.354270106 +0000 UTC m=+4.117177064 container died 32c08a4540d59e56536ba4218c83ca78ab664ecb00173b8a1b71b59b6c12ee69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:36:37.164 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:36 smithi019 podman[121694]: 2024-09-12 03:36:36.975523534 +0000 UTC m=+4.738430531 container cleanup 32c08a4540d59e56536ba4218c83ca78ab664ecb00173b8a1b71b59b6c12ee69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-12T03:36:37.164 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:36 smithi019 bash[121694]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:36:37.547 INFO:journalctl@ceph.mds.g.smithi019.stdout:Sep 12 03:36:37 smithi019 podman[121706]: 2024-09-12 03:36:37.285653257 +0000 UTC m=+0.929559010 container remove 32c08a4540d59e56536ba4218c83ca78ab664ecb00173b8a1b71b59b6c12ee69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, ceph=True) 2024-09-12T03:36:37.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:36:37.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:37.605 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:36:37.605 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:36:37.606 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:36:37.914 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:37 smithi019 systemd[1]: Stopping Ceph mds.j for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:38.414 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j[69898]: 2024-09-12T03:36:37.943+0000 7fc523ee2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:38.414 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:37 smithi019 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j[69898]: 2024-09-12T03:36:37.943+0000 7fc523ee2640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:36:39.664 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:39 smithi019 podman[121814]: 2024-09-12 03:36:39.386777106 +0000 UTC m=+1.556014537 container died 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True) 2024-09-12T03:36:40.013 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:39 smithi019 podman[121814]: 2024-09-12 03:36:39.838283641 +0000 UTC m=+2.007521113 container cleanup 3c5242e2b975dd6d677883803079205b524b77e220b43df24ceb06b831f3b571 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3) 2024-09-12T03:36:40.014 INFO:journalctl@ceph.mds.j.smithi019.stdout:Sep 12 03:36:39 smithi019 bash[121814]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:36:40.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:36:40.253 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:40.254 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:36:40.254 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:36:40.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:36:40.556 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:40 smithi121 systemd[1]: Stopping Ceph mds.b for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:40.813 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: 2024-09-12T03:36:40.555+0000 7f3761142640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:40.813 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:40 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b[65460]: 2024-09-12T03:36:40.555+0000 7f3761142640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:36:44.771 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:44 smithi121 podman[71898]: 2024-09-12 03:36:44.441152273 +0000 UTC m=+3.998071706 container died 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-12T03:36:45.125 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:44 smithi121 podman[71898]: 2024-09-12 03:36:44.872297606 +0000 UTC m=+4.429217035 container cleanup 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:36:45.125 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:44 smithi121 bash[71898]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:36:45.354 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:36:45.376 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:45 smithi121 podman[71922]: 2024-09-12 03:36:45.125707905 +0000 UTC m=+0.674001410 container remove 02df0f7f297bd3b262e4cd7952a361b3827f528c035b0e183da7e5688bb53657 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T03:36:45.376 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:45 smithi121 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-12T03:36:45.376 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:45 smithi121 systemd[1]: Stopped Ceph mds.b for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:36:45.376 INFO:journalctl@ceph.mds.b.smithi121.stdout:Sep 12 03:36:45 smithi121 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.b.service: Consumed 5min 44.589s CPU time. 2024-09-12T03:36:45.390 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:45.390 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:36:45.390 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:36:45.390 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:36:45.649 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:45 smithi121 systemd[1]: Stopping Ceph mds.e for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:46.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: 2024-09-12T03:36:45.753+0000 7f9cfeb77640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:46.150 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:45 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e[64847]: 2024-09-12T03:36:45.753+0000 7f9cfeb77640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:36:48.729 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:48 smithi121 podman[72033]: 2024-09-12 03:36:48.440364914 +0000 UTC m=+2.826310894 container died e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:49.120 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:48 smithi121 podman[72033]: 2024-09-12 03:36:48.82810202 +0000 UTC m=+3.214047955 container cleanup e0ff72c82541106d2e9f7e08294248d1a12dce22f59162f88264f5d9d35a4a1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:36:49.120 INFO:journalctl@ceph.mds.e.smithi121.stdout:Sep 12 03:36:48 smithi121 bash[72033]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:36:49.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:36:49.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:49.284 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:36:49.284 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:36:49.284 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:36:49.399 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:49 smithi121 systemd[1]: Stopping Ceph mds.h for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:49.899 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: 2024-09-12T03:36:49.598+0000 7fa7090fb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:49.900 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:49 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h[65145]: 2024-09-12T03:36:49.598+0000 7fa7090fb640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:36:52.715 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:52 smithi121 podman[72155]: 2024-09-12 03:36:52.432689096 +0000 UTC m=+2.934510567 container died a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:36:53.091 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:52 smithi121 podman[72155]: 2024-09-12 03:36:52.815131678 +0000 UTC m=+3.316953156 container cleanup a1b14c53624fe455a45d22ae2cda7ab6a1f6f8df726417b041150978afaa9736 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:36:53.091 INFO:journalctl@ceph.mds.h.smithi121.stdout:Sep 12 03:36:52 smithi121 bash[72155]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:36:53.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:36:53.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:53.250 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:36:53.250 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:36:53.251 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:36:53.649 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:53 smithi121 systemd[1]: Stopping Ceph mds.k for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:53.649 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: 2024-09-12T03:36:53.576+0000 7fb85c14f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:53.650 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:53 smithi121 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k[64531]: 2024-09-12T03:36:53.576+0000 7fb85c14f640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:36:56.702 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:56 smithi121 podman[72278]: 2024-09-12 03:36:56.422276798 +0000 UTC m=+2.945593843 container died 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:36:57.095 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:56 smithi121 podman[72278]: 2024-09-12 03:36:56.793629478 +0000 UTC m=+3.316946513 container cleanup 1ff83e0bca6eef5ae2d0f99ec1e8cc97b52f1727da15c1beadd340b9b59382cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-12T03:36:57.095 INFO:journalctl@ceph.mds.k.smithi121.stdout:Sep 12 03:36:56 smithi121 bash[72278]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:36:57.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:36:57.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:36:57.249 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:36:57.249 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:36:57.249 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:36:57.544 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:36:57 smithi135 systemd[1]: Stopping Ceph mds.c for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:36:57.864 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:36:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: 2024-09-12T03:36:57.600+0000 7f10bc30d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:36:57.864 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:36:57 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c[65698]: 2024-09-12T03:36:57.600+0000 7f10bc30d640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:37:00.778 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:00 smithi135 podman[72390]: 2024-09-12 03:37:00.451381223 +0000 UTC m=+2.978549839 container died 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T03:37:01.034 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:00 smithi135 podman[72390]: 2024-09-12 03:37:00.86681007 +0000 UTC m=+3.393978682 container cleanup 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:37:01.034 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:00 smithi135 bash[72390]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:37:01.264 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:37:01.286 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:01 smithi135 podman[72413]: 2024-09-12 03:37:01.034677794 +0000 UTC m=+0.574762916 container remove 3ecb57a1b6bd2d2c7803d7b1fee3eb58bd5dde0cc090abf4e2c41d354d41ceeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:37:01.286 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:01 smithi135 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.c.service: Deactivated successfully. 2024-09-12T03:37:01.286 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:01 smithi135 systemd[1]: Stopped Ceph mds.c for 45fb56a2-70b4-11ef-bcea-c7b262605968. 2024-09-12T03:37:01.286 INFO:journalctl@ceph.mds.c.smithi135.stdout:Sep 12 03:37:01 smithi135 systemd[1]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.c.service: Consumed 5min 15.452s CPU time. 2024-09-12T03:37:01.301 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:37:01.301 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:37:01.302 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:37:01.302 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:37:01.544 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:01 smithi135 systemd[1]: Stopping Ceph mds.f for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:37:02.044 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f[65399]: 2024-09-12T03:37:01.637+0000 7fb2041c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:37:02.044 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:01 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f[65399]: 2024-09-12T03:37:01.637+0000 7fb2041c6640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:37:04.751 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:04 smithi135 podman[72520]: 2024-09-12 03:37:04.453911534 +0000 UTC m=+2.925854815 container died 8ec1c147ce0a65bad4d49ffab95777d52a5da8b062a9f6b6cde0b4488966485a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:37:05.044 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:04 smithi135 podman[72520]: 2024-09-12 03:37:04.83737066 +0000 UTC m=+3.309313940 container cleanup 8ec1c147ce0a65bad4d49ffab95777d52a5da8b062a9f6b6cde0b4488966485a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:37:05.044 INFO:journalctl@ceph.mds.f.smithi135.stdout:Sep 12 03:37:04 smithi135 bash[72520]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:37:05.215 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:37:05.249 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:37:05.250 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:37:05.250 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:37:05.250 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:37:05.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:05 smithi135 systemd[1]: Stopping Ceph mds.i for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:37:05.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:37:05.582+0000 7f426c111640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:37:05.794 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:05 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i[66012]: 2024-09-12T03:37:05.582+0000 7f426c111640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:37:08.674 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:08 smithi135 podman[72641]: 2024-09-12 03:37:08.423981423 +0000 UTC m=+2.945584140 container died 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-12T03:37:09.044 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:08 smithi135 podman[72641]: 2024-09-12 03:37:08.907736789 +0000 UTC m=+3.429339519 container cleanup 1e37e1e7664a8bbbf81b8f4c5405bf6f0d383686fa2444c36507620988e68b10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:37:09.044 INFO:journalctl@ceph.mds.i.smithi135.stdout:Sep 12 03:37:08 smithi135 bash[72641]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:37:09.282 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:37:09.315 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:37:09.316 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:37:09.316 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:37:09.316 DEBUG:teuthology.orchestra.run.smithi135:> sudo systemctl stop ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:37:09.544 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:09 smithi135 systemd[1]: Stopping Ceph mds.l for 45fb56a2-70b4-11ef-bcea-c7b262605968... 2024-09-12T03:37:10.044 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: 2024-09-12T03:37:09.627+0000 7f88177c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:37:10.044 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:09 smithi135 ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l[65096]: 2024-09-12T03:37:09.627+0000 7f88177c3640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:37:12.770 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:12 smithi135 podman[72763]: 2024-09-12 03:37:12.425841633 +0000 UTC m=+2.893836063 container died 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-12T03:37:13.036 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:12 smithi135 podman[72763]: 2024-09-12 03:37:12.844480388 +0000 UTC m=+3.312474800 container cleanup 5ebb9a7c69774436f23d57f36a0af81bb2b073a3c7c6595840807e7265a06931 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:37:13.036 INFO:journalctl@ceph.mds.l.smithi135.stdout:Sep 12 03:37:12 smithi135 bash[72763]: ceph-45fb56a2-70b4-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:37:13.239 DEBUG:teuthology.orchestra.run.smithi135:> sudo pkill -f 'journalctl -f -n 0 -u ceph-45fb56a2-70b4-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:37:13.271 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:37:13.271 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:37:13.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:37:13.483 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:37:16.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:37:16.409 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:37:18.973 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:37:19.173 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T03:37:21.585 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:21.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:21.653 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:37:21.681 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:37:21.683 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi019/crash 2024-09-12T03:37:21.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:37:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:tar: /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:37:21.714 INFO:teuthology.orchestra.run.smithi019.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:37:21.716 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi121/crash 2024-09-12T03:37:21.717 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:37:21.761 INFO:teuthology.orchestra.run.smithi121.stderr:tar: /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:37:21.762 INFO:teuthology.orchestra.run.smithi121.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:37:21.763 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi135/crash 2024-09-12T03:37:21.763 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:37:21.799 INFO:teuthology.orchestra.run.smithi135.stderr:tar: /var/lib/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:37:21.799 INFO:teuthology.orchestra.run.smithi135.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:37:21.800 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:37:21.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:37:21.837 INFO:teuthology.orchestra.run.smithi019.stdout:2024-09-12T03:22:38.269247+0000 mds.i (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details 2024-09-12T03:37:21.838 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-12T03:37:21.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[SEC\]' /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:37:21.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]' /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:37:21.995 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[WRN\]' /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:37:22.070 INFO:teuthology.orchestra.run.smithi019.stdout:2024-09-12T03:22:38.269247+0000 mds.i (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details 2024-09-12T03:37:22.071 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:37:22.071 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:37:22.114 DEBUG:teuthology.orchestra.run.smithi121:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:37:22.117 DEBUG:teuthology.orchestra.run.smithi135:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:37:22.138 INFO:teuthology.orchestra.run.smithi019.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:37:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:37:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:37:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:37:22.139 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr: 82.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:37:22.143 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:37:22.152 INFO:teuthology.orchestra.run.smithi135.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-12T03:37:22.153 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:37:22.157 INFO:teuthology.orchestra.run.smithi135.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:37:22.157 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:37:22.157 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:37:22.157 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:37:22.157 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:37:22.158 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:37:22.159 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:37:22.161 INFO:teuthology.orchestra.run.smithi121.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log: No such file or directory 2024-09-12T03:37:22.162 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-12T03:37:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:37:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:37:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log: /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:37:22.162 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:37:22.163 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:37:22.163 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:37:22.163 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:37:22.163 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:37:22.163 INFO:teuthology.orchestra.run.smithi019.stderr: 89.1% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:37:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:37:22.170 INFO:teuthology.orchestra.run.smithi121.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log: /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:37:22.171 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:37:22.172 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:37:22.172 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:37:22.174 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:37:22.175 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:37:22.200 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:37:22.202 INFO:teuthology.orchestra.run.smithi121.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.1% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:37:22.290 INFO:teuthology.orchestra.run.smithi135.stderr:/var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 82.3% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:37:22.355 INFO:teuthology.orchestra.run.smithi135.stderr: 90.9% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:37:22.378 INFO:teuthology.orchestra.run.smithi121.stderr: 89.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:37:22.420 INFO:teuthology.orchestra.run.smithi135.stderr: 90.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:37:22.440 INFO:teuthology.orchestra.run.smithi135.stderr: 93.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:37:22.451 INFO:teuthology.orchestra.run.smithi121.stderr: 82.4% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:37:22.453 INFO:teuthology.orchestra.run.smithi121.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:37:22.497 INFO:teuthology.orchestra.run.smithi019.stderr: 90.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:37:22.502 INFO:teuthology.orchestra.run.smithi121.stderr: 90.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:37:22.536 INFO:teuthology.orchestra.run.smithi121.stderr: 90.9% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:37:22.544 INFO:teuthology.orchestra.run.smithi135.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:37:23.279 INFO:teuthology.orchestra.run.smithi019.stderr: 90.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:37:23.887 INFO:teuthology.orchestra.run.smithi135.stderr: 89.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:37:25.229 INFO:teuthology.orchestra.run.smithi121.stderr: 91.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:37:26.746 INFO:teuthology.orchestra.run.smithi135.stderr: 91.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:37:28.938 INFO:teuthology.orchestra.run.smithi019.stderr: 95.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:37:32.445 INFO:teuthology.orchestra.run.smithi019.stderr: 95.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:37:37.351 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:37:45.014 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:37:45.970 INFO:teuthology.orchestra.run.smithi019.stderr: 92.6% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:37:49.786 INFO:teuthology.orchestra.run.smithi019.stderr: 92.4% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:37:49.788 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-12T03:37:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m27.661s 2024-09-12T03:37:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:user 1m44.617s 2024-09-12T03:37:49.788 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m3.282s 2024-09-12T03:38:03.887 INFO:teuthology.orchestra.run.smithi121.stderr: 95.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:38:20.698 INFO:teuthology.orchestra.run.smithi121.stderr: 92.0% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:38:39.889 INFO:teuthology.orchestra.run.smithi121.stderr: 92.3% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:39:25.876 INFO:teuthology.orchestra.run.smithi121.stderr: 92.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:39:28.351 INFO:teuthology.orchestra.run.smithi121.stderr: 95.9% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:39:53.829 INFO:teuthology.orchestra.run.smithi121.stderr: 92.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:41:13.636 INFO:teuthology.orchestra.run.smithi135.stderr: 92.6% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:41:28.002 INFO:teuthology.orchestra.run.smithi135.stderr: 95.6% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:41:51.305 INFO:teuthology.orchestra.run.smithi135.stderr: 92.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:43:20.135 INFO:teuthology.orchestra.run.smithi135.stderr: 92.2% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:44:03.367 INFO:teuthology.orchestra.run.smithi135.stderr: 92.1% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:45:42.497 INFO:teuthology.orchestra.run.smithi121.stderr: 94.6% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:46:24.550 INFO:teuthology.orchestra.run.smithi121.stderr: 94.7% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:46:24.552 INFO:teuthology.orchestra.run.smithi121.stderr: 2024-09-12T03:46:24.552 INFO:teuthology.orchestra.run.smithi121.stderr:real 9m2.419s 2024-09-12T03:46:24.552 INFO:teuthology.orchestra.run.smithi121.stderr:user 5m50.451s 2024-09-12T03:46:24.552 INFO:teuthology.orchestra.run.smithi121.stderr:sys 0m15.686s 2024-09-12T04:01:53.960 INFO:teuthology.orchestra.run.smithi135.stderr: 94.5% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T04:03:20.314 INFO:teuthology.orchestra.run.smithi135.stderr: 94.8% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T04:05:06.298 INFO:teuthology.orchestra.run.smithi135.stderr: 94.1% -- replaced with /var/log/ceph/45fb56a2-70b4-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T04:05:06.328 INFO:teuthology.orchestra.run.smithi135.stderr: 2024-09-12T04:05:06.328 INFO:teuthology.orchestra.run.smithi135.stderr:real 27m44.196s 2024-09-12T04:05:06.329 INFO:teuthology.orchestra.run.smithi135.stderr:user 8m22.995s 2024-09-12T04:05:06.329 INFO:teuthology.orchestra.run.smithi135.stderr:sys 0m26.564s 2024-09-12T04:05:06.329 INFO:tasks.cephadm:Archiving logs... 2024-09-12T04:05:06.329 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi019/log 2024-09-12T04:05:06.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T04:05:20.469 DEBUG:teuthology.misc:Transferring archived files from smithi121:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi121/log 2024-09-12T04:05:20.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T04:06:12.022 DEBUG:teuthology.misc:Transferring archived files from smithi135:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi135/log 2024-09-12T04:06:12.024 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T04:07:25.213 INFO:tasks.cephadm:Removing cluster... 2024-09-12T04:07:25.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T04:07:25.427 INFO:teuthology.orchestra.run.smithi019.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T04:07:25.823 DEBUG:teuthology.orchestra.run.smithi121:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T04:07:26.134 INFO:teuthology.orchestra.run.smithi121.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T04:07:26.718 DEBUG:teuthology.orchestra.run.smithi135:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 45fb56a2-70b4-11ef-bcea-c7b262605968 --force 2024-09-12T04:07:27.454 INFO:teuthology.orchestra.run.smithi135.stdout:Deleting cluster with fsid: 45fb56a2-70b4-11ef-bcea-c7b262605968 2024-09-12T04:07:28.064 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T04:07:28.064 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T04:07:28.085 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T04:07:28.105 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T04:07:28.121 INFO:tasks.cephadm:Teardown complete 2024-09-12T04:07:28.121 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T04:07:28.133 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T04:07:28.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T04:07:28.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T04:07:28.147 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T04:07:28.331 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T04:07:28.342 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T04:07:28.342 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T04:07:28.345 DEBUG:teuthology.orchestra.run.smithi121:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T04:07:28.348 DEBUG:teuthology.orchestra.run.smithi135:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T04:07:28.360 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-12T04:07:28.362 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-09-12T04:07:28.362 INFO:teuthology.orchestra.run.smithi135.stderr:bash: line 1: ntpq: command not found 2024-09-12T04:07:28.365 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T04:07:28.366 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-12T04:07:28.366 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -2269us[-2277us] +/- 53ms 2024-09-12T04:07:28.366 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 +1511us[+1511us] +/- 84ms 2024-09-12T04:07:28.366 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +2016us[+2016us] +/- 78ms 2024-09-12T04:07:28.366 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 -2330us[-2199us] +/- 53ms 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 17 +1418us[+1418us] +/- 84ms 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 18 +1954us[+1954us] +/- 78ms 2024-09-12T04:07:28.395 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T04:07:28.416 INFO:teuthology.orchestra.run.smithi135.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T04:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout:=============================================================================== 2024-09-12T04:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 -2280us[-2199us] +/- 53ms 2024-09-12T04:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 +1483us[+1483us] +/- 84ms 2024-09-12T04:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +2026us[+2107us] +/- 78ms 2024-09-12T04:07:28.417 INFO:teuthology.orchestra.run.smithi135.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T04:07:28.431 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T04:07:28.443 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T04:07:28.443 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T04:07:28.530 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T04:07:28.624 DEBUG:teuthology.orchestra.run.smithi121:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T04:07:28.716 DEBUG:teuthology.orchestra.run.smithi135:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T04:07:28.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T04:07:28.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:28.852 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T04:07:28.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:28.883 DEBUG:teuthology.orchestra.run.smithi135:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T04:07:28.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:28.912 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T04:07:28.923 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T04:07:28.964 INFO:teuthology.task.internal:Duration was 3895.617675 seconds 2024-09-12T04:07:28.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T04:07:28.978 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T04:07:28.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T04:07:28.981 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T04:07:28.983 DEBUG:teuthology.orchestra.run.smithi135:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T04:07:29.023 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T04:07:29.041 INFO:teuthology.orchestra.run.smithi135.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T04:07:29.047 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T04:07:29.457 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T04:07:29.457 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-09-12T04:07:29.457 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T04:07:29.481 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi121.front.sepia.ceph.com 2024-09-12T04:07:29.481 DEBUG:teuthology.orchestra.run.smithi121:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T04:07:29.519 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi135.front.sepia.ceph.com 2024-09-12T04:07:29.520 DEBUG:teuthology.orchestra.run.smithi135:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T04:07:29.547 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T04:07:29.547 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T04:07:29.550 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T04:07:29.563 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T04:07:29.620 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T04:07:29.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T04:07:29.623 DEBUG:teuthology.orchestra.run.smithi121:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T04:07:29.634 DEBUG:teuthology.orchestra.run.smithi135:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T04:07:31.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T04:07:31.182 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T04:07:31.182 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T04:07:31.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T04:07:31.255 DEBUG:teuthology.orchestra.run.smithi135:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T04:07:31.287 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T04:07:31.300 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.305 DEBUG:teuthology.orchestra.run.smithi135:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.324 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-09-12T04:07:31.333 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = core 2024-09-12T04:07:31.381 INFO:teuthology.orchestra.run.smithi135.stdout:kernel.core_pattern = core 2024-09-12T04:07:31.400 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:31.417 DEBUG:teuthology.orchestra.run.smithi121:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:31.434 DEBUG:teuthology.orchestra.run.smithi135:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T04:07:31.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T04:07:31.451 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T04:07:31.464 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T04:07:31.466 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi019 2024-09-12T04:07:31.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T04:07:31.537 DEBUG:teuthology.misc:Transferring archived files from smithi121:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi121 2024-09-12T04:07:31.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T04:07:31.577 DEBUG:teuthology.misc:Transferring archived files from smithi135:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901665/remote/smithi135 2024-09-12T04:07:31.577 DEBUG:teuthology.orchestra.run.smithi135:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T04:07:31.617 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T04:07:31.617 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T04:07:31.620 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T04:07:31.622 DEBUG:teuthology.orchestra.run.smithi135:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T04:07:31.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T04:07:31.682 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T04:07:31.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T04:07:31.720 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T04:07:31.720 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T04:07:31.723 DEBUG:teuthology.orchestra.run.smithi121:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T04:07:31.725 DEBUG:teuthology.orchestra.run.smithi135:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T04:07:31.736 INFO:teuthology.orchestra.run.smithi019.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 04:07 /home/ubuntu/cephtest 2024-09-12T04:07:31.739 INFO:teuthology.orchestra.run.smithi121.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 04:07 /home/ubuntu/cephtest 2024-09-12T04:07:31.742 INFO:teuthology.orchestra.run.smithi135.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 04:07 /home/ubuntu/cephtest 2024-09-12T04:07:31.743 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T04:07:31.769 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T04:07:31.929 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/blogbench}} duration: 3895.617674589157 failure_reason: '"2024-09-12T03:22:38.269247+0000 mds.i (mds.0) 24 : cluster [WRN] Scrub error on inode 0x10000002ac1 (/volumes/qa/sv_1/68e383da-dc2e-4231-8b0e-a1f655e088d6/client.0/tmp/blogbench-1.0/src/blogtest_in/blog-76) see mds.i log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-09-12T04:07:31.929 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T04:07:32.022 INFO:teuthology.run:FAIL